Mailing List Archive: Lucene: Java-Dev

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force)

Index | Next | Previous | View Threaded

jira at apache

Jun 23, 2008, 10:14 AM

Post #1 of 13 (560 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force)

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

First cut at this, seems to work as desired. Need to come up with a test case for it.

Does anyone know how to turn off Eclipse automatically changing the import statements? I am not making it reformat but if I edit some code in a file it sees fit to reformat the imports.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Priority: Minor
> Attachments: lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

mike.klaas at gmail

Jun 23, 2008, 1:07 PM

Post #2 of 13 (534 views)
Permalink

Re: [jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

On 23-Jun-08, at 10:14 AM, Jason Rutherglen (JIRA) wrote:
>
> Does anyone know how to turn off Eclipse automatically changing the
> import statements? I am not making it reformat but if I edit some
> code in a file it sees fit to reformat the imports.

http://www.google.com/search?q=turn%20off%20eclipse%20changing%20import%20statements

----------------
I'm running into a problem where Organize Imports is removing all of
my import statements. I had to turn off "Keep Imports Organized"
because I noticed that ...
----------------

-Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jason.rutherglen at gmail

Jun 23, 2008, 3:52 PM

Post #3 of 13 (531 views)
Permalink

Re: [jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

Project -> Properties -> Java Editor -> Save Actions Check "Perform the
selected actions on save" Uncheck Organize Imports is the correct answer now
in Google

On Mon, Jun 23, 2008 at 4:07 PM, Mike Klaas <mike.klaas[at]gmail.com> wrote:

> On 23-Jun-08, at 10:14 AM, Jason Rutherglen (JIRA) wrote:
>
>>
>> Does anyone know how to turn off Eclipse automatically changing the import
>> statements? I am not making it reformat but if I edit some code in a file
>> it sees fit to reformat the imports.
>>
>
>
> http://www.google.com/search?q=turn%20off%20eclipse%20changing%20import%20statements
>
> ----------------
> I'm running into a problem where Organize Imports is removing all of my
> import statements. I had to turn off "Keep Imports Organized" because I
> noticed that ...
> ----------------
>
> -Mike
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
> For additional commands, e-mail: java-dev-help[at]lucene.apache.org
>
>

jira at apache

Jun 24, 2008, 8:38 AM

Post #4 of 13 (516 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Included the changes mentioned to allow a subclass of SegmentReader to not load fieldsReader on every reopen. Added the IMPL based instantiation in reopenSegment. SegmentReader.doClose does not close fieldsReader if openNewFieldsReader is false.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jun 24, 2008, 9:56 AM

Post #5 of 13 (516 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Added SegmentReader protected BitVector cloneDeletedDocs() that allows subclasses to do the cloning. This will allow for pooling reuse if many reopens occur and many BitVectors are created.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jun 29, 2008, 8:17 AM

Post #6 of 13 (475 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Added fieldsReaderLocal to SegmentReader. reopenSegment passed the fieldsReaderLocal and termVectorsLocal to the new SegmentReader. In the current version, a new termVectorsLocal is created for each SegmentReader which made reuse of the previously created TermVectorsReaders for a thread unavailable. The same is implemented for fieldsReaderLocal.

The locking needs a default mechanism. For my own purposes I will likely turn it off, the bug I posted was really the fault of the Ocean code since fixed.

Would like to be able to optionally have this line run in DirectoryIndexReader in reopen. Does it need to be run on a clone?

SegmentInfos infos = new SegmentInfos();
infos.read(directory, segmentFileName);

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 2, 2008, 3:06 PM

Post #7 of 13 (427 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

SegmentReader.doReopen throws an IOException when deletedDocsDirty or normsDirty is true. DirectoryIndexReader getLock and setLock removed. Added isCurrent check in DirectoryIndexReader.doCommit, as per a few comments ago regarding making sure the commit only happens to the latest index.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 3, 2008, 4:48 AM

Post #8 of 13 (415 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-1314:
---------------------------------------

Attachment: lucene-1314.patch

I don't think we need to check isCurrent() in doCommit() because since
the reader holds the write lock it is necessarily current?

I can't compile with your patch. It seems like you have changes to
BitVector.java which did't make it into the patch? Eg the getBits()
method.

I attached a new version with these changes:

* Throw IllegalStateException if you try to clone a reader that has
pending changes.

* Clone the SegmentInfos, instead of just taking a reference, in
DirectoryIndexReader

* Changed "public abstract Object clone()" in IndexReader to be a
method that throws UnsupportedOperationException instead

* Renamed a few things; removed some whitespace only diffs.

* Factored up doReopenOrClone into IndexReader

One difference between clone() and reopen() is you force the
deletedDocs BitVector to be cloned in SegmentReader during clone(),
but not during reopen(). With reopen() we "declared" that if you make
changes to your reopened reader, it's undefined what happens to your
old readers. Ie, it's a "don't do that" situation.

But with clone() the situation is reversed: the whole reason why you
make a clone() is to isolate any changes in the new reader from being
visible to the old reader. Given that, I think you also must clone()
the norms right?

Jason could you add cloning of norms, and add some unit tests, to the
patch? Thanks.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 7, 2008, 6:11 AM

Post #9 of 13 (373 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Implemented copy on write for norms and deletedDocs when a cloned SegmentReader is created. A copy on write ref count is used to insure the correct number of copies are created.

Includes a test case.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 8, 2008, 10:50 AM

Post #10 of 13 (353 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

docStoreOffset is now cloned in FieldsReader.clone() fixing the bug encountered by TestStressIndexing2.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 8, 2008, 10:52 AM

Post #11 of 13 (353 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Same as previous, with TestIndexReaderClone

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 10, 2008, 7:12 AM

Post #12 of 13 (320 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Added protected DirectoryIndexReader.allowCloneWithChanges which allows clones with changes to be made. Added protected IndexReader.decRef(boolean flush) and protected IndexReader.close(boolean flush) which allows subclasses to optionally flush changes.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Jul 16, 2008, 7:44 AM

Post #13 of 13 (258 views)
Permalink

[jira] Updated: (LUCENE-1314) IndexReader.reopen(boolean force) [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1314:
-------------------------------------

Attachment: lucene-1314.patch

lucene-1314.patch

Fixed bug in SegmentReader.reopenSegment that caused the ref count of deleted docs and norms to reset after the second clone. Added test in TestIndexReaderClone to test for this.

Answering the question from a few comments ago, I don't think this patch needs to wait on IndexReader becoming read only.

> IndexReader.reopen(boolean force)
> ---------------------------------
>
> Key: LUCENE-1314
> URL: https://issues.apache.org/jira/browse/LUCENE-1314
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Affects Versions: 2.3.1
> Reporter: Jason Rutherglen
> Assignee: Michael McCandless
> Priority: Minor
> Attachments: lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch, lucene-1314.patch
>
>
> Based on discussion http://www.nabble.com/IndexReader.reopen-issue-td18070256.html. The problem is reopen returns the same reader if there are no changes, so if docs are deleted from the new reader, they are also reflected in the previous reader which is not always desired behavior.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com