Mailing List Archive: Lucene: Java-User

Sort.INDEXORDER works incorrectly?

Index | Next | Previous | View Threaded

xonixx at gmail

Apr 17, 2012, 4:27 PM

Post #1 of 3 (174 views)
Permalink

Sort.INDEXORDER works incorrectly?

Hi, dear Lucene specialists,

I've updated in my project the version of Lucene to the latest (3.6).
I wanted at last to add near realtime support to my indexing
subsystem. Everything seemed to work pretty fine except that I
suddenly encountered that the sorting by indexorder (actually,
reversed order as I'm using new Sort(new SortField(null,
SortField.DOC, true))) not works as before.

Please let me explain. I am sorting by reversed index order because it
coinsides with the order of the dates of my documents (actually, forum
posts) but sorting by actual dates is much more RAM-intensive and
slow.

Now what I'm seen is the resulting order looks like random.
Interesting enough is that when I search by *:* (matchall) I'm getting
nearly correct order (though at 1 results page it is also little
broken).

The only explanation I could think of is the new TieredMergePolicy
instead of old LogMergePolicy. Could it be that because of
TieredMergePolicy merges not adjacent segments - this results in not
preserving index order or this is total nonsense?

Please advise.

Sincerely yours,
Vladimir

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

trejkaz at trypticon

Apr 17, 2012, 4:33 PM

Post #2 of 3 (168 views)
Permalink

Re: Sort.INDEXORDER works incorrectly? [In reply to]

On Wed, Apr 18, 2012 at 9:27 AM, Vladimir Gubarkov <xonixx [at] gmail> wrote:
> Hi, dear Lucene specialists,
>
> The only explanation I could think of is the new TieredMergePolicy
> instead of old LogMergePolicy. Could it be that because of
> TieredMergePolicy merges not adjacent segments - this results in not
> preserving index order or this is total nonsense?

TieredMergePolicy will definitely shuffle the order of docs in your
index. But unless you provided a Version of Version.LUCENE_32 or later
when constructing your IndexWriterConfig, it shouldn't be using that
(and this won't happen automatically when you upgrade, unless you're
doing something odd like using LUCENE_LATEST...)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

xonixx at gmail

Apr 17, 2012, 4:42 PM

Post #3 of 3 (169 views)
Permalink

Re: Sort.INDEXORDER works incorrectly? [In reply to]

Thank you, Trejkaz for your quick reply!

I definitelly updated my global constant
public static final Version VERSION = Version.LUCENE_30;
to
public static final Version VERSION = Version.LUCENE_36;

And performed total reindex as is recommended.

So the only option is to use old mergepolicy? Btw, I wonder is it
compatible with near realtime indexing?

On Wed, Apr 18, 2012 at 3:33 AM, Trejkaz <trejkaz [at] trypticon> wrote:
> On Wed, Apr 18, 2012 at 9:27 AM, Vladimir Gubarkov <xonixx [at] gmail> wrote:
>> Hi, dear Lucene specialists,
>>
>> The only explanation I could think of is the new TieredMergePolicy
>> instead of old LogMergePolicy. Could it be that because of
>> TieredMergePolicy merges not adjacent segments - this results in not
>> preserving index order or this is total nonsense?
>
> TieredMergePolicy will definitely shuffle the order of docs in your
> index. But unless you provided a Version of Version.LUCENE_32 or later
> when constructing your IndexWriterConfig, it shouldn't be using that
> (and this won't happen automatically when you upgrade, unless you're
> doing something odd like using LUCENE_LATEST...)
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads