opensubscriber
   Find in this group all groups
 
Unknown more information…

j : java-user@lucene.apache.org 9 March 2005 • 4:16AM -0500

Re: Document lazy-loading WAS [Re: Fast access to a random page of the search results.]
by markharw00d

REPLY TO AUTHOR
 
REPLY TO GROUP



So this is just the old problem of avoiding reading large, less
frequently accessed fields when you are trying to read just the smaller
more frequently accessed fields eg titles.
You can achieve this by:

a)  Modifying Lucene using something like the code I  originally posted
which stops reading a document's fields if the required list has been
found (hoping the stored field order is favourable to all of your app's
access patterns)
b) Storing any large-but-rarely-accessed fields externally eg in files
or an RDBMS and keeping a primary key in Lucene to retrieve the  content
when required.
c) If you dont want to use an RDBMS and want to keep it all Lucene-based
you could store any large-but-rarely-accessed fields in another index
alongside your main one to avoid the hit.. Each large field could be a
separate document in this index and you could retrieve it when required
by accessing largeFieldStoreIndexReader.document(x) where x is
1) calculated by taking the mainSearchIndexReader document id and
multiplying by the number of externally stored large fields per main
document or
2) Querying largeFieldStoreIndex for a unique key which is stored in
the mainSearchIndex.

I would favour b) as a and c seem messy.




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@luce...
For additional commands, e-mail: java-user-help@luce...

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.