Mailing List Archive: Lucene: Java-Dev

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Index | Next | Previous | View Threaded

jira at apache

Nov 6, 2009, 2:10 PM

Post #1 of 11 (194 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joey Surls updated LUCENE-2041:
-------------------------------

Attachment: LUCENE-2041.patch

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 6, 2009, 3:46 PM

Post #2 of 11 (183 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

Fix Version/s: (was: 3.0)
3.1
Assignee: Simon Willnauer

moving to 3.1 for now. we will see how this turns out

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 7, 2009, 4:00 PM

Post #3 of 11 (175 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

Attachment: LUCENE_2041.patch

I drove this a bit further and refactored the whole code to use the java 1.5 concurrent utils. I seem to be somewhat cleaner with executors / callable and a little refactoring.

simon

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 9, 2009, 6:16 AM

Post #4 of 11 (164 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Attachment: LUCENE-2041.patch

Updated patch.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 9, 2009, 6:18 AM

Post #5 of 11 (164 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

Attachment: LUCENE-2041.patch

Fixed the Java5 issue.
Damned I should use java 5 on my notebook.

simon

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 9, 2009, 3:28 PM

Post #6 of 11 (161 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Attachment: LUCENE-2041.patch

Advanced patch:
- Removed @SuppressWarnings by duplicating some code, but it is type-safe now
- Added support for maxScore in sorted mode
- Refactoring: made all internal classes static inner classes
- Fix thread safety: hq.setFields() synchronized

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 9, 2009, 3:53 PM

Post #7 of 11 (161 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Attachment: LUCENE-2041-refactor.patch

Here another possibility, removing more duplicated code: MultiSearcher contains now the merge Callables as static inner classes. These Callables are used directly without Executor in MultiSearcher, but with Executor in ParallelMultiSearcher.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 10, 2009, 7:57 AM

Post #8 of 11 (138 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

Attachment: LUCENE-2041.patch

Added some more final keywords and generalized HitQueue to PriorityQueue<ScoreDoc> in search (no sort)

I guess we are set with this patch, good team work Uwe! Lets get this out for heavy committing! :)

I get the impression that we should think about moving stuff like the Function interface out to utils. This could be useful in many situations though. Thoughts?

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 10, 2009, 8:15 AM

Post #9 of 11 (138 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Attachment: LUCENE-2041-refactor.patch

Here my latest patch, will commit soon.

Uwe

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 10, 2009, 8:19 AM

Post #10 of 11 (138 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Fix Version/s: (was: 3.1)
3.0

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

jira at apache

Nov 10, 2009, 8:47 AM

Post #11 of 11 (134 views)
Permalink

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

Attachment: LUCENE-2041-final.patch

Merged our patches somehow, but left HitQueue generic param (I like this more, because it conforms to the sort inner class). I think this is final now.

Ready to commit, all tests are running.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-final.patch, LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe[at]lucene.apache.org
For additional commands, e-mail: java-dev-help[at]lucene.apache.org

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com