Mailing List Archive: Lucene: Java-Dev

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.

Index | Next | Previous | View Threaded

jira at apache

Apr 24, 2012, 3:10 PM

Post #1 of 12 (91 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261075#comment-13261075 ]

Dawid Weiss commented on LUCENE-4020:
-------------------------------------

I think the "reproduce with" line should explicitly state the timezone and locale picked for the test. Granted, certain combinations will be invalid on other JVMs/ systems, but at least it's explicit what should be used?

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 24, 2012, 3:38 PM

Post #2 of 12 (96 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261101#comment-13261101 ]

Dawid Weiss commented on LUCENE-4020:
-------------------------------------

Oops, sorry, wrong thinking. It has to be the actual Locale, not the passed property.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 24, 2012, 4:24 PM

Post #3 of 12 (93 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261132#comment-13261132 ]

Robert Muir commented on LUCENE-4020:
-------------------------------------

don't we need to also take care to still consume the same number of random bits
when these params are specified?

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 12:13 AM

Post #4 of 12 (88 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261354#comment-13261354 ]

Dawid Weiss commented on LUCENE-4020:
-------------------------------------

Good point, yes it should be identical. I'll update in the evening.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 3:28 PM

Post #5 of 12 (89 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262204#comment-13262204 ]

Robert Muir commented on LUCENE-4020:
-------------------------------------

+1 looks great

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 3:30 PM

Post #6 of 12 (87 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262206#comment-13262206 ]

Dawid Weiss commented on LUCENE-4020:
-------------------------------------

It'll fail on 1.7 and those darn script variant locales should they be picked for the test... The first time it does, I'll think what to do about it.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 4:02 PM

Post #7 of 12 (88 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262228#comment-13262228 ]

Hoss Man commented on LUCENE-4020:
----------------------------------

straw man suggestion:

change the impl of randomLocale(Random) to ensure that it only returns Locale objects that can be round tripped...

{code}
public static Locale randomLocale(final Random outerRandom) {
// consume a fixed amount of randomness from outerRandom
final Random random = new Random(outerRandom.nextLong());

Locale locales[] = Locale.getAvailableLocales();
Locale result = null;
for (int i = 0; i < 100; i++) {
Locale candidate = locales[random.nextInt(locales.length)];
try {
Locale roundTrip = localeForName(candidate.toString());
if (candidate.equals(roundTrip)) {
result = candidate;
break;
}
} catch (Exception e) {
// :NOOP: ... go around again
}
}
if (null == result) {
throw new YourJvmIsFuckedException("Gave up trying to pick a random Locale")
}
return result;
}
{code}

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 4:10 PM

Post #8 of 12 (87 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262231#comment-13262231 ]

Robert Muir commented on LUCENE-4020:
-------------------------------------

but your jvm isnt fucked, and we should test the new java 7 locales.

Its just this idea of roundtripping without using anything but java7 BCP47 apis is fucked.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 4:14 PM

Post #9 of 12 (88 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262232#comment-13262232 ]

Hoss Man commented on LUCENE-4020:
----------------------------------

so the exception name isn't great ... but the point is there should be a decent number of Locales that work, and we should be able to try N times to find one, nad if we really don't then give up because we can't run a reproducible test.

(or hell: don't give up .. try 100 times and if we still don't get one that round trips just use whatever we did get and let the user deal with the fact that they can't pass that locale back in to reproduce if it fails)

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 4:18 PM

Post #10 of 12 (88 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262235#comment-13262235 ]

Robert Muir commented on LUCENE-4020:
-------------------------------------

I can make them all work.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 25, 2012, 4:36 PM

Post #11 of 12 (86 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262244#comment-13262244 ]

Robert Muir commented on LUCENE-4020:
-------------------------------------

Patch is on 4021 for round trip parsing (in 6 and 7). It should also work across 6 and 7,
e.g. in on java6 we emit th_TH_TH, on java7 th_TH_TH_#u-nu-thai, but both are constructed as th_TH_TH.

This is ok because of the backwards mechanism (Special Cases) listed in http://docs.oracle.com/javase/7/docs/api/java/util/Locale.htm

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

jira at apache

Apr 26, 2012, 1:32 AM

Post #12 of 12 (87 views)
Permalink

[jira] [Commented] (LUCENE-4020) Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked. [In reply to]

[ https://issues.apache.org/jira/browse/LUCENE-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262465#comment-13262465 ]

Dawid Weiss commented on LUCENE-4020:
-------------------------------------

Fantastic, thanks.

> Tests may not be repeatable across different platforms/ JVMs due to different locale/ timezone being picked.
> ------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-4020
> URL: https://issues.apache.org/jira/browse/LUCENE-4020
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4020.diff, LUCENE-4020.patch
>
>
> This is because the source array can be/ is different for each system/ JVM. So this pick is not repeatable for example:
> {code}
> /**
> * Return a random Locale from the available locales on the system.
> */
> public static Locale randomLocale(Random random) {
> Locale locales[] = Locale.getAvailableLocales();
> return locales[random.nextInt(locales.length)];
> }
> {code}
> I don't think there is much we can do to make it repeatable (other than maybe enforcing locale using system property).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe [at] lucene
For additional commands, e-mail: dev-help [at] lucene

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads