Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-User

Readers synchronization

 

 

Lucene java-user RSS feed   Index | Next | Previous | View Threaded


ediaz666 at yahoo

Jul 8, 2008, 8:39 AM

Post #1 of 6 (4593 views)
Permalink
Readers synchronization

According to SVN history on the next version this will be available:

LUCENE-1044: IndexWriter with autoCommit=true now commits (such
that a reader can see the changes) far less often than it used to.
Previously, every flush was also a commit. You can always force a
commit by calling IndexWriter.commit(). Furthermore, in 3.0,
autoCommit will be hardwired to false (IndexWriter constructors
that take an autoCommit argument have been deprecated) (Mike
McCandless)

Does this mean that I won't need to reopen all the readers in order to see the index changes?

Thanks




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


lucene at mikemccandless

Jul 8, 2008, 9:12 AM

Post #2 of 6 (4451 views)
Permalink
Re: Readers synchronization [In reply to]

No, that's not changed. You must still reopen an IndexReader to see
changes to the index. An IndexReader always searches a point-in-time
snapshot of the index.

LUCENE-1044 does mean that you should call IndexWriter.commit() (or,
close the writer) to ensure all changes you've made become visible to
the reader.

Mike

Eric Diaz wrote:

> According to SVN history on the next version this will be available:
>
> LUCENE-1044: IndexWriter with autoCommit=true now commits (such
> that a reader can see the changes) far less often than it used to.
> Previously, every flush was also a commit. You can always force a
> commit by calling IndexWriter.commit(). Furthermore, in 3.0,
> autoCommit will be hardwired to false (IndexWriter constructors
> that take an autoCommit argument have been deprecated) (Mike
> McCandless)
>
> Does this mean that I won't need to reopen all the readers in order
> to see the index changes?
>
> Thanks
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


ediaz666 at yahoo

Jul 8, 2008, 9:40 AM

Post #3 of 6 (4431 views)
Permalink
Re: Readers synchronization [In reply to]

Besides the warm up that the faq section suggests (used on solr), is there another technique or solution to have an IndexReader/Search with an updated view of an index under a concurrent scenario (web app)?

Thanks

--- On Tue, 7/8/08, Michael McCandless <lucene [at] mikemccandless> wrote:

> From: Michael McCandless <lucene [at] mikemccandless>
> Subject: Re: Readers synchronization
> To: java-user [at] lucene
> Date: Tuesday, July 8, 2008, 11:12 AM
> No, that's not changed. You must still reopen an
> IndexReader to see
> changes to the index. An IndexReader always searches a
> point-in-time
> snapshot of the index.
>
> LUCENE-1044 does mean that you should call
> IndexWriter.commit() (or,
> close the writer) to ensure all changes you've made
> become visible to
> the reader.
>
> Mike
>
> Eric Diaz wrote:
>
> > According to SVN history on the next version this will
> be available:
> >
> > LUCENE-1044: IndexWriter with autoCommit=true now
> commits (such
> > that a reader can see the changes) far less often
> than it used to.
> > Previously, every flush was also a commit. You can
> always force a
> > commit by calling IndexWriter.commit().
> Furthermore, in 3.0,
> > autoCommit will be hardwired to false (IndexWriter
> constructors
> > that take an autoCommit argument have been
> deprecated) (Mike
> > McCandless)
> >
> > Does this mean that I won't need to reopen all the
> readers in order
> > to see the index changes?
> >
> > Thanks
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> java-user-unsubscribe [at] lucene
> > For additional commands, e-mail:
> java-user-help [at] lucene
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe [at] lucene
> For additional commands, e-mail:
> java-user-help [at] lucene




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


lucene at mikemccandless

Jul 8, 2008, 9:58 AM

Post #4 of 6 (4429 views)
Permalink
Re: Readers synchronization [In reply to]

No other techniques that I know of...

But there is ongoing discussions/work towards making reopening a
reader much less costly. EG repopulating the field cache after reopen
is a costly operation now, but this issue:

https://issues.apache.org/jira/browse/LUCENE-1231

would make that cost be proportional to the number & size of the
changed segments since you last reopened.

There has also been discussions on creating an IndexReader
implementation that can directly search the RAM buffer in IndexWriter,
which should give very fast turnaround in searching just-indexed
documents, but that is quite a ways off...

Mike

Eric Diaz wrote:

> Besides the warm up that the faq section suggests (used on solr), is
> there another technique or solution to have an IndexReader/Search
> with an updated view of an index under a concurrent scenario (web
> app)?
>
> Thanks
>
> --- On Tue, 7/8/08, Michael McCandless <lucene [at] mikemccandless>
> wrote:
>
>> From: Michael McCandless <lucene [at] mikemccandless>
>> Subject: Re: Readers synchronization
>> To: java-user [at] lucene
>> Date: Tuesday, July 8, 2008, 11:12 AM
>> No, that's not changed. You must still reopen an
>> IndexReader to see
>> changes to the index. An IndexReader always searches a
>> point-in-time
>> snapshot of the index.
>>
>> LUCENE-1044 does mean that you should call
>> IndexWriter.commit() (or,
>> close the writer) to ensure all changes you've made
>> become visible to
>> the reader.
>>
>> Mike
>>
>> Eric Diaz wrote:
>>
>>> According to SVN history on the next version this will
>> be available:
>>>
>>> LUCENE-1044: IndexWriter with autoCommit=true now
>> commits (such
>>> that a reader can see the changes) far less often
>> than it used to.
>>> Previously, every flush was also a commit. You can
>> always force a
>>> commit by calling IndexWriter.commit().
>> Furthermore, in 3.0,
>>> autoCommit will be hardwired to false (IndexWriter
>> constructors
>>> that take an autoCommit argument have been
>> deprecated) (Mike
>>> McCandless)
>>>
>>> Does this mean that I won't need to reopen all the
>> readers in order
>>> to see the index changes?
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> java-user-unsubscribe [at] lucene
>>> For additional commands, e-mail:
>> java-user-help [at] lucene
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> java-user-unsubscribe [at] lucene
>> For additional commands, e-mail:
>> java-user-help [at] lucene
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


ediaz666 at yahoo

Jul 8, 2008, 10:38 AM

Post #5 of 6 (4410 views)
Permalink
Re: Readers synchronization [In reply to]

Is there any plan to change this behavior? meaning that by default a reader will see the current index?

Thanks in advance

--- On Tue, 7/8/08, Michael McCandless <lucene [at] mikemccandless> wrote:

> From: Michael McCandless <lucene [at] mikemccandless>
> Subject: Re: Readers synchronization
> To: java-user [at] lucene, ediaz666 [at] yahoo
> Date: Tuesday, July 8, 2008, 11:58 AM
> No other techniques that I know of...
>
> But there is ongoing discussions/work towards making
> reopening a
> reader much less costly. EG repopulating the field cache
> after reopen
> is a costly operation now, but this issue:
>
> https://issues.apache.org/jira/browse/LUCENE-1231
>
> would make that cost be proportional to the number &
> size of the
> changed segments since you last reopened.
>
> There has also been discussions on creating an IndexReader
>
> implementation that can directly search the RAM buffer in
> IndexWriter,
> which should give very fast turnaround in searching
> just-indexed
> documents, but that is quite a ways off...
>
> Mike
>
> Eric Diaz wrote:
>
> > Besides the warm up that the faq section suggests
> (used on solr), is
> > there another technique or solution to have an
> IndexReader/Search
> > with an updated view of an index under a concurrent
> scenario (web
> > app)?
> >
> > Thanks
> >
> > --- On Tue, 7/8/08, Michael McCandless
> <lucene [at] mikemccandless>
> > wrote:
> >
> >> From: Michael McCandless
> <lucene [at] mikemccandless>
> >> Subject: Re: Readers synchronization
> >> To: java-user [at] lucene
> >> Date: Tuesday, July 8, 2008, 11:12 AM
> >> No, that's not changed. You must still reopen
> an
> >> IndexReader to see
> >> changes to the index. An IndexReader always
> searches a
> >> point-in-time
> >> snapshot of the index.
> >>
> >> LUCENE-1044 does mean that you should call
> >> IndexWriter.commit() (or,
> >> close the writer) to ensure all changes you've
> made
> >> become visible to
> >> the reader.
> >>
> >> Mike
> >>
> >> Eric Diaz wrote:
> >>
> >>> According to SVN history on the next version
> this will
> >> be available:
> >>>
> >>> LUCENE-1044: IndexWriter with autoCommit=true
> now
> >> commits (such
> >>> that a reader can see the changes) far less
> often
> >> than it used to.
> >>> Previously, every flush was also a commit.
> You can
> >> always force a
> >>> commit by calling IndexWriter.commit().
> >> Furthermore, in 3.0,
> >>> autoCommit will be hardwired to false
> (IndexWriter
> >> constructors
> >>> that take an autoCommit argument have been
> >> deprecated) (Mike
> >>> McCandless)
> >>>
> >>> Does this mean that I won't need to reopen
> all the
> >> readers in order
> >>> to see the index changes?
> >>>
> >>> Thanks
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail:
> >> java-user-unsubscribe [at] lucene
> >>> For additional commands, e-mail:
> >> java-user-help [at] lucene
> >>>
> >>
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail:
> >> java-user-unsubscribe [at] lucene
> >> For additional commands, e-mail:
> >> java-user-help [at] lucene
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> java-user-unsubscribe [at] lucene
> > For additional commands, e-mail:
> java-user-help [at] lucene
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe [at] lucene
> For additional commands, e-mail:
> java-user-help [at] lucene




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


lucene at mikemccandless

Jul 8, 2008, 10:41 AM

Post #6 of 6 (4393 views)
Permalink
Re: Readers synchronization [In reply to]

Not that I know of.

Mike

Eric Diaz wrote:

> Is there any plan to change this behavior? meaning that by default a
> reader will see the current index?
>
> Thanks in advance
>
> --- On Tue, 7/8/08, Michael McCandless <lucene [at] mikemccandless>
> wrote:
>
>> From: Michael McCandless <lucene [at] mikemccandless>
>> Subject: Re: Readers synchronization
>> To: java-user [at] lucene, ediaz666 [at] yahoo
>> Date: Tuesday, July 8, 2008, 11:58 AM
>> No other techniques that I know of...
>>
>> But there is ongoing discussions/work towards making
>> reopening a
>> reader much less costly. EG repopulating the field cache
>> after reopen
>> is a costly operation now, but this issue:
>>
>> https://issues.apache.org/jira/browse/LUCENE-1231
>>
>> would make that cost be proportional to the number &
>> size of the
>> changed segments since you last reopened.
>>
>> There has also been discussions on creating an IndexReader
>>
>> implementation that can directly search the RAM buffer in
>> IndexWriter,
>> which should give very fast turnaround in searching
>> just-indexed
>> documents, but that is quite a ways off...
>>
>> Mike
>>
>> Eric Diaz wrote:
>>
>>> Besides the warm up that the faq section suggests
>> (used on solr), is
>>> there another technique or solution to have an
>> IndexReader/Search
>>> with an updated view of an index under a concurrent
>> scenario (web
>>> app)?
>>>
>>> Thanks
>>>
>>> --- On Tue, 7/8/08, Michael McCandless
>> <lucene [at] mikemccandless>
>>> wrote:
>>>
>>>> From: Michael McCandless
>> <lucene [at] mikemccandless>
>>>> Subject: Re: Readers synchronization
>>>> To: java-user [at] lucene
>>>> Date: Tuesday, July 8, 2008, 11:12 AM
>>>> No, that's not changed. You must still reopen
>> an
>>>> IndexReader to see
>>>> changes to the index. An IndexReader always
>> searches a
>>>> point-in-time
>>>> snapshot of the index.
>>>>
>>>> LUCENE-1044 does mean that you should call
>>>> IndexWriter.commit() (or,
>>>> close the writer) to ensure all changes you've
>> made
>>>> become visible to
>>>> the reader.
>>>>
>>>> Mike
>>>>
>>>> Eric Diaz wrote:
>>>>
>>>>> According to SVN history on the next version
>> this will
>>>> be available:
>>>>>
>>>>> LUCENE-1044: IndexWriter with autoCommit=true
>> now
>>>> commits (such
>>>>> that a reader can see the changes) far less
>> often
>>>> than it used to.
>>>>> Previously, every flush was also a commit.
>> You can
>>>> always force a
>>>>> commit by calling IndexWriter.commit().
>>>> Furthermore, in 3.0,
>>>>> autoCommit will be hardwired to false
>> (IndexWriter
>>>> constructors
>>>>> that take an autoCommit argument have been
>>>> deprecated) (Mike
>>>>> McCandless)
>>>>>
>>>>> Does this mean that I won't need to reopen
>> all the
>>>> readers in order
>>>>> to see the index changes?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>> java-user-unsubscribe [at] lucene
>>>>> For additional commands, e-mail:
>>>> java-user-help [at] lucene
>>>>>
>>>>
>>>>
>>>>
>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>> java-user-unsubscribe [at] lucene
>>>> For additional commands, e-mail:
>>>> java-user-help [at] lucene
>>>
>>>
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> java-user-unsubscribe [at] lucene
>>> For additional commands, e-mail:
>> java-user-help [at] lucene
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> java-user-unsubscribe [at] lucene
>> For additional commands, e-mail:
>> java-user-help [at] lucene
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Lucene java-user RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.