Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jira at apache

Feb 8, 2010, 12:21 PM

Post #1 of 4 (383 views)
Permalink
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles

[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831095#action_12831095 ]

Jason Rutherglen commented on LUCENE-2255:
------------------------------------------

You're not the first to report this... I wonder if we should also do a weak ref type of dealie that auto closes the reader for users who aren't calling IR.close?

To answer the question, yes, close on IR must be called. I can't directly do anything about the javadocs so I'll leave that for Mike.

> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene


jira at apache

Feb 8, 2010, 12:43 PM

Post #2 of 4 (365 views)
Permalink
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831107#action_12831107 ]

Simon Willnauer commented on LUCENE-2255:
-----------------------------------------

I see this coming up multiple times, we should document this properly in the javadoc and on the wiki. Jason, aren't you the NRT specialist here. What keeps you from attaching a patch for the IW javadoc?

simon

> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene


jira at apache

Feb 8, 2010, 12:59 PM

Post #3 of 4 (365 views)
Permalink
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831116#action_12831116 ]

Jason Rutherglen commented on LUCENE-2255:
------------------------------------------

bq. What keeps you from attaching a patch for the IW javadoc?

Laziness and lack of motivation... :)

I'll post something to the issue, how's that?

> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene


jira at apache

Feb 8, 2010, 1:17 PM

Post #4 of 4 (366 views)
Permalink
[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831124#action_12831124 ]

Michael McCandless commented on LUCENE-2255:
--------------------------------------------

This has already been fixed on trunk... we added this to the javadocs a while back:

* <p>You must close the {@link IndexReader} returned by
* this method once you are done using it.</p>

While the returned reader used IndexWriter to determine which segments to open, the returned reader is fully independent of the writer. EG you can close the writer and continue using the reader. And you must close the reader.

I'll backport that fix to 3.0 & 2.9.

> IndexWriter.getReader() allocates file handles
> ----------------------------------------------
>
> Key: LUCENE-2255
> URL: https://issues.apache.org/jira/browse/LUCENE-2255
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Environment: Ubuntu 9.10, Java 6
> Reporter: Mikkel Kamstrup Erlandsen
> Attachments: LuceneManyCommits.java
>
>
> I am not sure if this is a "bug" or really just me not reading the Javadocs right...
> The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a *new* reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.
> So perhaps the docs should be amended to clearly state that this is a caller-owns reader that *must* be closed? Attaching a simple app that illustrates the problem.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.