Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Lucene: Java-Dev

[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains

 

 

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded


jira at apache

Apr 8, 2012, 6:35 PM

Post #1 of 7 (108 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-3969:
--------------------------------

Attachment: LUCENE-3969.patch

First cut at a patch: supports all of our analysis components, which means basically every time you run this test now, it finds a bug.

The test ignores any invocations of UOE or IAE, and i fixed most/all trivial issues related to that, but there are serious problems...

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 8, 2012, 9:13 PM

Post #2 of 7 (104 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-3969:
--------------------------------

Attachment: LUCENE-3969.patch

updated patch... just fixing some more bugs.

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 9, 2012, 1:43 AM

Post #3 of 7 (103 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-3969:
----------------------------------

Attachment: LUCENE-3969.patch

Patch with the above mentioned cleanups. I also changed some code to be more consistent (isAnnotationPresent also for classes, Class.getName instead toString).

This thing fails almost always, in most cases with "too many tokens".

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 9, 2012, 2:09 AM

Post #4 of 7 (104 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-3969:
----------------------------------

Attachment: LUCENE-3969.patch

New patch:
- I now have found out where the UOE comes from, it's the random parameter generator. But this is nasty and should be solved better. I readded the catch block with a comment.

In general, the code should be refactored to not call getConstrcutors all the time. Instead the global list of List<Class<T>> should be replaced by List<Constructor<T>>, then we only have one list where to choose the ctor from (the class is implicit).

Will work on a patch.

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 9, 2012, 3:35 AM

Post #5 of 7 (102 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-3969:
----------------------------------

Attachment: LUCENE-3969.patch

New patch with the Constructors moved up the chain to be top-level citizens, analysis classes are no longer explicitely used.

It would be goot to fix the randomParameter generator to never fail but instead the reflection code in beforClass() to check the constructor args against a Set<Class<?>> validArgs of valid parameters and throw away all invalid ctors from the beginning: validArgs.containsAll(Arrays.asList(ctor.getParameterTypes()))

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 9, 2012, 6:03 AM

Post #6 of 7 (104 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-3969:
----------------------------------

Attachment: LUCENE-3969.patch

Here a patch with a more flexible argument generator. Its still a little bit ugly how the three special cases are handled (and the Sets need to be kept in sync!), but at least the standard types are created very simple.

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene


jira at apache

Apr 11, 2012, 9:37 AM

Post #7 of 7 (104 views)
Permalink
[jira] [Updated] (LUCENE-3969) Use all (non-deprecated) analysis ctors in TestRandomChains [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-3969:
--------------------------------

Fix Version/s: 3.6.1
4.0

> Use all (non-deprecated) analysis ctors in TestRandomChains
> -----------------------------------------------------------
>
> Key: LUCENE-3969
> URL: https://issues.apache.org/jira/browse/LUCENE-3969
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Robert Muir
> Fix For: 4.0, 3.6.1
>
> Attachments: LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch, LUCENE-3969.patch
>
>
> We made TestRandomChains in LUCENE-3919, which reflects all
> analysis components from the classpath and builds analyzers from them,
> then checks consistency.
> but currently it only supports some tokenizers/tokenfilters/charfilters,
> because it hardcodes at certain ctors e.g. Tokenizer(Reader) and Tokenizer(Version+Reader).
> Instead we should use all ctors, just filling them in with random data of
> whatever argument type they take.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

Lucene java-dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.