Mailing List Archive: Lucene: Java-Dev

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery

Index | Next | Previous | View Threaded

bugzilla at apache

Jan 8, 2005, 9:21 AM

Post #1 of 15 (151 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965

paul.elschot [at] xs4all changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|Use filter bits for next() |[PATCH] Use filter bits for
|and skipTo() in |next() and skipTo() in
|FilteredQuery |FilteredQuery

------- Additional Comments From paul.elschot [at] xs4all 2005-01-08 17:21 -------
This 5 Jan 2005 version of FilteredQuery.java could be simplified
by deriving an intIterator directly from a BitSet.
In the useIterFilter() method this is done with a SortedVIntList
in between, but it could easily be done directly.

Perhaps the intIterator attachment should be renamed to DocNrIter
or DocNrIterator.

With the current Filter only a java.lang.BitSet
can be used as the underlying data structure for the filter.
IterFilter allows the use of other data structures,
eg. an int array of document numbers, or a SortedVIntList.

Regards,
Paul Elschot

--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe [at] jakarta
For additional commands, e-mail: lucene-dev-help [at] jakarta

bugzilla at apache

Jan 8, 2005, 9:24 AM

Post #2 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

bugzilla at apache

Jan 8, 2005, 2:40 PM

Post #3 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965

paul.elschot [at] xs4all changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #13940|0 |1
is obsolete| |

------- Additional Comments From paul.elschot [at] xs4all 2005-01-08 22:40 -------
Created an attachment (id=13944)
--> (http://issues.apache.org/bugzilla/attachment.cgi?id=13944&action=view)
Refactoring of FilteredQuery, 2nd version of 8 Jan 2005.

FilteredScorer now has separate methods for next() and skipTo()
for the filter (abstract filterNext() and abstract filterSkipTo())
and for the query scorer (scorerNext() and scorerSkipTo()).
The next() and skipTo() methods work like ConjunctionScorer,
but the filter and query scorer are separately called.
The filter is always used before the query scorer to minimize I/O.

The inline subclasses of FilteredScorer for Filter and IterFilter
only differ for the filterNext() and filterSkipTo() methods and on a helper
method for explain(), so they now have a readable size.

Regards,
Paul Elschot

--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe [at] jakarta
For additional commands, e-mail: lucene-dev-help [at] jakarta

bugzilla at apache

Jan 9, 2005, 6:58 AM

Post #4 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

bugzilla at apache

Jan 16, 2005, 10:11 AM

Post #5 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

bugzilla at apache

Jan 16, 2005, 10:16 AM

Post #6 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

bugzilla at apache

Jan 16, 2005, 10:22 AM

Post #7 of 15 (149 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32965>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965

------- Additional Comments From paul.elschot [at] xs4all 2005-01-16 18:22 -------
The patch in IndexSearcher is independent of the rest of these
attachments.

SkipFilter.java also contains a method to replace the Filter by
a SkipFilter. This method is for testing only.

With and without the use of this method, and with the patch to
IndexSearcher, all current tests pass.
This could indicate that the combination of BooleanQuery and Filter
is is missing from the current tests.

FilteredQuery is also considerably simplified wrt. to the version
of 8 Jan 2005.

Regards,
Paul Elschot

--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe [at] jakarta
For additional commands, e-mail: lucene-dev-help [at] jakarta

bugzilla at apache

Jan 16, 2005, 10:22 AM

Post #8 of 15 (150 views)
Permalink

DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

ab at getopt

Apr 4, 2005, 5:18 AM

Post #9 of 15 (148 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

Erik Hatcher wrote:
> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert

Erhm.. you meant 1.5 (five), right?

--
Best regards,
Andrzej Bialecki
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

erik at ehatchersolutions

Apr 4, 2005, 6:18 AM

Post #10 of 15 (149 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

On Apr 4, 2005, at 8:18 AM, Andrzej Bialecki wrote:
> Erik Hatcher wrote:
>> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
>
> Erhm.. you meant 1.5 (five), right?

No, 1.4. Assert's were added in JDK 1.4. I live in a Mac-centric
world and 1.5 (errr, 5.0!), and I've yet to run 1.5 since its not
freely available for Mac OS X yet.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

chuck at allthingslocal

Apr 4, 2005, 9:02 AM

Post #11 of 15 (148 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

Erik Hatcher writes (4/4/2005 2:36 AM):

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.

As a 1.5 user, I'd love to see Lucene at least at 1.4. Assert's are a
good thing.

Chuck

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

bernhard.messer at intrafind

Apr 4, 2005, 9:32 AM

Post #12 of 15 (148 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

Erik Hatcher wrote:

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?

I'm not a fan of outdated software or historical systems. So i think the
best would be to keep lucene still backward compatible with version 1.9
and perform the switch to JDK 1.4 with lucene 2.0.

Bernhard

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

bmesser at apache

Apr 4, 2005, 9:33 AM

Post #13 of 15 (148 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

Erik Hatcher wrote:

> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?
>
> Erik

I'm not a fan of outdated software or historical systems. So i think the
best would be to keep lucene still backward compatible with version 1.9
and perform the switch to JDK 1.4 with lucene 2.0.

Bernhard

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

paul.elschot at xs4all

Apr 4, 2005, 10:14 AM

Post #14 of 15 (150 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

On Monday 04 April 2005 11:34, Erik Hatcher wrote:
> I added Paul's SkipFilter and overwrote my FilteredQuery class from the
> code here. However, your FilteredQuery depends on two additional
> classes: DocNrSkipper and SortedVIntList that are not provided.
> Please attach those classes to this issue.

They are in one of the bugs the filter depends on:
http://issues.apache.org/bugzilla/show_bug.cgi?id=32921

>
> I'm not personally fond of the abbreviation of "Number" to "Nr" - any
> objections to spelling out "Number" entirely?

Not at all. My tradeoff between programming line length and
identifier readability is just slightly different. For me, number is
general enough to abbreviate in an identifier.

> Any others have objections to Paul's patches here? I'll commit them
> once I have everything working locally, unless I hear otherwise.

Please note the other dependency: this filter only works correctly when
documents are always scored in document number order.
This means that the bucket table of the 1.4.3 BooleanScorer can not
be used anymore, except when sorting is added.
See also the last comments I posted at that bug:
http://issues.apache.org/bugzilla/show_bug.cgi?id=33019
The (non ordered) bucket scoring mechanism the fastest one that is
available, anything else is slower, alhough not by much.

As for the assert statements: could you comment them out in case
the dependency on java 1.4 is not acceptable? That would
leave them useable for later.

Regards,
Paul Elschot

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

paul.elschot at xs4all

Apr 4, 2005, 10:41 AM

Post #15 of 15 (148 views)
Permalink

Re: DO NOT REPLY [Bug 32965] - [PATCH] Use filter bits for next() and skipTo() in FilteredQuery [In reply to]

On Monday 04 April 2005 11:36, Erik Hatcher wrote:
> Oh, and one other thing.... Paul's code relies on JDK 1.4's assert
> keyword. It seems this is an unnecessary reason to jump to 1.4
> dependence.
>
> What do folks think about JDK 1.4 as a minimum Lucene requirement?

The FilteredQuery as posted there requires jdk 1.4
because it uses BitSet.nextSetBit():

http://issues.apache.org/bugzilla/show_bug.cgi?id=32965#c2

Regards,
Paul Elschot

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads