jira at apache
Jun 30, 2008, 3:14 PM
Post #3 of 13
(479 views)
Permalink
|
|
[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
[In reply to]
|
|
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609383#action_12609383 ]
Michael McCandless commented on LUCENE-753:
-------------------------------------------
OK I re-ran only PooledPread, SeparateFile and ChannelPread since they
are the "leading contenders" on all platforms.
Also, I changed to serial=false.
Now the results are very close on all but windows, but on windows I'm
seeing the opposite of what Yonik saw: PooledPread is slowest, and
SeparateFile is fastest. But this is a laptop (Win XP Pro), and it is
JRE 1.4. Also I ran with pool size == number of threads == 4.
Mac OS X 10.5.3, single WD Velociraptor hard drive, Sun JRE 1.6.0_05
{code}
config: impl=PooledPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=120807, MB/sec=222.20190551871994
config: impl=SeparateFile serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830326, ms=119641, MB/sec=224.36744594244448
config: impl=ChannelPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=119217, MB/sec=225.1654176837196
{code}
Linux 2.6.22.1, 6-drive RAID 5 array, Sun JRE 1.6.0_06
{code}
config: impl=PooledPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=52613, MB/sec=510.2074696367818
config: impl=SeparateFile serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=52715, MB/sec=509.22025230010433
config: impl=ChannelPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=53792, MB/sec=499.0248661511005
{code}
WIN XP PRO, laptop, Sun JRE 1.4.2_15:
{code}
config: impl=PooledPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=209956, MB/sec=127.85319590771401
config: impl=SeparateFile serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=89101, MB/sec=301.27098012367986
config: impl=ChannelPread serial=false nThreads=4 iterations=100 bufsize=1024 poolsize=4 filelen=67108864
answer=-23830370, ms=184087, MB/sec=145.81988733587923
{code}
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java, FSIndexInput.patch, FSIndexInput.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the underlying file.
> This could mitigate any MT performance drop caused by reducing the number of files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org
|
