Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

[jira] Commented: (LUCENE-1574) PooledSegmentReader, pools SegmentReader underlying byte arrays

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jira at apache

Nov 10, 2009, 10:04 PM

Post #1 of 3 (219 views)
Permalink
[jira] Commented: (LUCENE-1574) PooledSegmentReader, pools SegmentReader underlying byte arrays

[ https://issues.apache.org/jira/browse/LUCENE-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776299#action_12776299 ]

Jason Rutherglen commented on LUCENE-1574:
------------------------------------------

Yonik,

Do you recommend using the method in SimpleStringInterner for lockless pooling?

> PooledSegmentReader, pools SegmentReader underlying byte arrays
> ---------------------------------------------------------------
>
> Key: LUCENE-1574
> URL: https://issues.apache.org/jira/browse/LUCENE-1574
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/*
> Affects Versions: 2.4.1
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 3.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> PooledSegmentReader pools the underlying byte arrays of deleted docs and norms for realtime search. It is designed for use with IndexReader.clone which can create many copies of byte arrays, which are of the same length for a given segment. When pooled they can be reused which could save on memory.
> Do we want to benchmark the memory usage comparison of PooledSegmentReader vs GC? Many times GC is enough for these smaller objects.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene


jira at apache

Nov 10, 2009, 10:50 PM

Post #2 of 3 (196 views)
Permalink
[jira] Commented: (LUCENE-1574) PooledSegmentReader, pools SegmentReader underlying byte arrays [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776321#action_12776321 ]

Jason Rutherglen commented on LUCENE-1574:
------------------------------------------

I suppose as we're on Java 1.5, ConcurrentLinkedQueue can be used.

> PooledSegmentReader, pools SegmentReader underlying byte arrays
> ---------------------------------------------------------------
>
> Key: LUCENE-1574
> URL: https://issues.apache.org/jira/browse/LUCENE-1574
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/*
> Affects Versions: 2.4.1
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 3.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> PooledSegmentReader pools the underlying byte arrays of deleted docs and norms for realtime search. It is designed for use with IndexReader.clone which can create many copies of byte arrays, which are of the same length for a given segment. When pooled they can be reused which could save on memory.
> Do we want to benchmark the memory usage comparison of PooledSegmentReader vs GC? Many times GC is enough for these smaller objects.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene


jira at apache

Nov 11, 2009, 11:02 AM

Post #3 of 3 (186 views)
Permalink
[jira] Commented: (LUCENE-1574) PooledSegmentReader, pools SegmentReader underlying byte arrays [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776576#action_12776576 ]

Jason Rutherglen commented on LUCENE-1574:
------------------------------------------

A likely optimization for this patch is we'll only pool if the doc count is above a threshold, 100,000 seems like a good number. Also pooling will be optional.

> PooledSegmentReader, pools SegmentReader underlying byte arrays
> ---------------------------------------------------------------
>
> Key: LUCENE-1574
> URL: https://issues.apache.org/jira/browse/LUCENE-1574
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/*
> Affects Versions: 2.4.1
> Reporter: Jason Rutherglen
> Priority: Minor
> Fix For: 3.1
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> PooledSegmentReader pools the underlying byte arrays of deleted docs and norms for realtime search. It is designed for use with IndexReader.clone which can create many copies of byte arrays, which are of the same length for a given segment. When pooled they can be reused which could save on memory.
> Do we want to benchmark the memory usage comparison of PooledSegmentReader vs GC? Many times GC is enough for these smaller objects.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe [at] lucene
For additional commands, e-mail: java-dev-help [at] lucene

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.