Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-User

IndexWriteConfig ignored?

 

 

Lucene java-user RSS feed   Index | Next | Previous | View Threaded


clemensdev at mysign

Apr 16, 2012, 1:46 AM

Post #1 of 3 (240 views)
Permalink
IndexWriteConfig ignored?

We limit the memory consumption of our IndexWriters by setting RAMBufferSizeMB to 5MB (IndexWriterConfig.setRAMBufferSizeMB).
Inspecting a heapdump unveils that I still have writers wich consume/retain more than 35MB!
How come?
Any help/advice appreciated
Clemens

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


lucene at mikemccandless

Apr 16, 2012, 5:55 AM

Post #2 of 3 (229 views)
Permalink
Re: IndexWriteConfig ignored? [In reply to]

RAM can be used in IndexWriter for other reasons: merge is running,
near-real-time reader was opened.

The RAMBufferSizeMB only applies to buffered postings (indexed documents)

If you turn on IndexWriter's infoStream, do you see output saying it's
flushing a new segment because RAM is > 5.0 MB?

Mike McCandless

http://blog.mikemccandless.com

On Mon, Apr 16, 2012 at 4:46 AM, Clemens Wyss <clemensdev [at] mysign> wrote:
> We limit the memory consumption of our IndexWriters by setting RAMBufferSizeMB to 5MB (IndexWriterConfig.setRAMBufferSizeMB).
> Inspecting a heapdump unveils that I still have writers wich consume/retain more than 35MB!
> How come?
> Any help/advice appreciated
> Clemens
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


clemensdev at mysign

Apr 17, 2012, 2:33 AM

Post #3 of 3 (220 views)
Permalink
AW: IndexWriteConfig ignored? [In reply to]

No nrt-reader opened.

My heapdump says that the freeCharBlocks-member of DocumentsWriter is holding >20MB. I must say that I keep the IndexWriter open "forever", which (memorywise) does not seem to be "optimal".

-----Ursprüngliche Nachricht-----
Von: Michael McCandless [mailto:lucene [at] mikemccandless]
Gesendet: Montag, 16. April 2012 14:56
An: java-user [at] lucene
Betreff: Re: IndexWriteConfig ignored?

RAM can be used in IndexWriter for other reasons: merge is running, near-real-time reader was opened.

The RAMBufferSizeMB only applies to buffered postings (indexed documents)

If you turn on IndexWriter's infoStream, do you see output saying it's flushing a new segment because RAM is > 5.0 MB?

Mike McCandless

http://blog.mikemccandless.com

On Mon, Apr 16, 2012 at 4:46 AM, Clemens Wyss <clemensdev [at] mysign> wrote:
> We limit the memory consumption of our IndexWriters by setting RAMBufferSizeMB to 5MB (IndexWriterConfig.setRAMBufferSizeMB).
> Inspecting a heapdump unveils that I still have writers wich consume/retain more than 35MB!
> How come?
> Any help/advice appreciated
> Clemens
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
> For additional commands, e-mail: java-user-help [at] lucene
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe [at] lucene
For additional commands, e-mail: java-user-help [at] lucene

Lucene java-user RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.