opensubscriber
   Find in this group all groups
 
Unknown more information…

l : lucene-user@jakarta.apache.org 1 March 2005 • 12:59AM -0500

Re: Fast access to a random page of the search results.
by Erik Hatcher

REPLY TO AUTHOR
 
REPLY TO GROUP



On Feb 28, 2005, at 10:39 AM, Stanislav Jordanov wrote:
> > What did you do in your private investigation?
> 1. empirical tests with an index of nearly 75,000 docs (I am attaching
> the test source)

Only certain (.txt?) attachments are allowed to come through on the
mailing list.

> > Sorted by descending relevance (the default), or in some other way?
> In some other way - sorted by some column (asc or desc - doesn't
> matter)

Using IndexSearcher(query, sort)?

>  > If a search is fast enough, as you report, then you can simply start
> > your access to Hits at the appropriate spot.  For the current systems
> > I'm working on, this is the approach I've used - start iterating hits
> > at (pageNumber - 1) * numberOfItemsPerPage.
> >
> > Is that approach insufficient?
> I'm afraid this is not sufficient;
> Either I am doing something wrong,
> or it is not that simple:
> following is a log from my test session;
> It appears that IndexSearcher.search(...) finishes rather fast
> compared to the time it takes to fetch the last document from the Hits
> object.

I assume you are only accessing the documents you wish to display
rather than all of them up to where you need.   Also keep in mind that
accessing a Document is when the document is pulled from the index.  If
you have a large amount of data in a document it will take a
corresponding amount of time to load it.  You may need to restructure
what you store in a document to reduce the load times.  Or perhaps you
need to investigate the (is it in the codebase already?) patch to load
fields lazily upon demand instead.

Erik

>

> The log starts here:
>
> pa
>
> Found 74222 document(s) that matched query 'pa'
>
> Sorting by "sfile_name"
>
> query executed in 16ms
>
> Last doc accessed in 375ms
>
> us
>
> Found 74222 document(s) that matched query 'us'
>
> Sorting by "sfile_name"
>
> query executed in 31ms
>
> Last doc accessed in 219ms
>
> 1
>
> Found 74222 document(s) that matched query '1'
>
> Sorting by "sfile_name"
>
> query executed in 15ms
>
> Last doc accessed in 235ms
>
> 5
>
> Found 74222 document(s) that matched query '5'
>
> Sorting by "sfile_name"
>
> query executed in 422ms
>
> Last doc accessed in 219ms
>
> 6
>
> Found 72759 document(s) that matched query '6'
>
> Sorting by "sfile_name"
>
> query executed in 344ms
>
> Last doc accessed in 250ms
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jaka...
> For additional commands, e-mail: lucene-user-help@jaka...

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jaka...
For additional commands, e-mail: lucene-user-help@jakarta.apache.o

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.