Mailing List Archive: Python: Dev

Re: please consider changing --enable-unicode default to ucs4

Index | Next | Previous | View Threaded

solipsis at pitrou

Sep 20, 2009, 9:36 AM

Post #1 of 6 (670 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4

Le Sun, 20 Sep 2009 10:17:45 -0600, Zooko O'Whielacronx a écrit :
>
>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4
>> setting.
>
> That would be an improvement! Unfortunately we instead get mysterious
> misbehavior of the module, e.g.:
>
> http://bugs.python.org/setuptools/msg309
> http://allmydata.org/trac/tahoe/ticket/704#comment:5

The bug reports in themselves aren't very explicit, and they don't seem
to be related to any native extension. So I'm not sure why you're talking
about "mysterious memory corruption errors" in your original mail,
because there doesn't seem to be such a thing happening at all.

Please note that there's a bug related to a non-portable peephole
optimization of some unicode constants, perhaps it may explain the
aforementioned problems (perhaps not) :
http://bugs.python.org/issue5057

I expect the solution to this bug to be rather easy (just disable the
optimization, since it isn't really useful), but someone has to care
enough to produce a patch.

>> For information, all Mandriva versions I've used until now have had
>> their Python's built with UCS2 (maxunicode == 65535).
>
> Thank you for the data point. This means that binary extension modules
> built on Mandriva can't be ported to Ubuntu or vice versa.

"Ported" they can certainly be, you just have to recompile.

> However, is
> this an argument for or against changing the default setting to UCS4?
> Changing the default setting wouldn't interfere with Mandriva's
> decision, right?

Well, let's put it this way:
- either you expect the default setting to be observed by everyone, and
it *will* interfere with someone's current decision
- or you don't expect the default setting to be observed by everyone, and
then there's no point in changing it because it won't stop your problems

Either way, my mentioning of Mandriva was just meant as an additional
data point to those you already provided ;-)

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Sep 20, 2009, 9:42 AM

Post #2 of 6 (631 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4 [In reply to]

Le Sun, 20 Sep 2009 10:33:23 -0600, Zooko O'Whielacronx a écrit :
>
> By the way, I was investigating this, and discovered an issue on the
> Mandriva tracker which suggests that they intend to switch to UCS4 in
> the next release in order to avoid compatibility problems like these.

Trying to use a Fedora or Suse RPM under Mandriva (or the other way
round) isn't reasonable and is certainly not supported.
I don't understand why this so-called "compatibility problem" should be
taken seriously by anyone.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Sep 29, 2009, 8:33 AM

Post #3 of 6 (592 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4 [In reply to]

Hello,

> I've also encountered this trap multiple times. Obviously, the problem
> is not rebuilding Python which is quick, but to figure out the correct
> configure option to use (--enable-unicode=ucs4). Others have also
> spent some time scratching their heads over the strange
> PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as
> Zooko's links show.

Isn't this overrated?

First, if you have a Python version that has the wrong version, just print out
its sys.maxunicode and choose the right version according to that (if
sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an
UCS-2 version).

Second, once you have encountered this issue, you know what you need the
subsequent times. There are only two possibilities after all.

> If Python can't infer the unicode setting from the width of the
> platforms wchar_t, then perhaps it should be mandatory to specify to
> configure whether you want UCS2 or UCS4? For someone clueless like me,
> it would be easier to deal with the problem upfront than (much)
> further down the line.

I'm not sure why someone "clueless" (your word :-)) wants to compile his own
Python, though.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

martin at v

Sep 29, 2009, 11:36 AM

Post #4 of 6 (591 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4 [In reply to]

>> I've also encountered this trap multiple times. Obviously, the problem
>> is not rebuilding Python which is quick, but to figure out the correct
>> configure option to use (--enable-unicode=ucs4). Others have also
>> spent some time scratching their heads over the strange
>> PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as
>> Zooko's links show.
>
> Isn't this overrated?
>
> First, if you have a Python version that has the wrong version, just print out
> its sys.maxunicode and choose the right version according to that (if
> sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an
> UCS-2 version).

To do so, you have to know that there is such a configuration option in
the first place, and that the error message you get (missing symbols)
has anything to do with it. This is quite puzzling to users.

Once people know what the problem is, fixing it is indeed easy.

> I'm not sure why someone "clueless" (your word :-)) wants to compile his own
> Python, though.

People install a lot of software that they don't understand. In fact,
most people who ever install software don't know how it is written,
and cannot enumerate the meaning of all configuration options that the
software possesses. In Unix, there is a long tradition that "installing
software" means "building from source"; if you have a configure script,
you expect that it either works out of the box, or gives an error
message if it finds that something is wrong with the environment.

So it is quite normal that people who don't understand how the Python
interpreter works (or that it has a Unicode type) install Python.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

jan.matejek at novell

Oct 5, 2009, 10:18 AM

Post #5 of 6 (540 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4 [In reply to]

Dne 20.9.2009 18:42, Antoine Pitrou napsal(a):
> Le Sun, 20 Sep 2009 10:33:23 -0600, Zooko O'Whielacronx a écrit :
>>
>> By the way, I was investigating this, and discovered an issue on the
>> Mandriva tracker which suggests that they intend to switch to UCS4 in
>> the next release in order to avoid compatibility problems like these.
>
> Trying to use a Fedora or Suse RPM under Mandriva (or the other way
> round) isn't reasonable and is certainly not supported.
> I don't understand why this so-called "compatibility problem" should be
> taken seriously by anyone.

You're not making sense. No distro is an island - plus, upstream
distributors have a nasty habit of providing RPMs only for Fedora. I
don't see what is bad about improving compatibility in a place where the
setting doesn't hurt one way or the other.
Besides, the more compatibility we achieve now, the easier time we'll
have once python makes it into LSB

regards
m.

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jmatejek%40suse.cz
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Oct 5, 2009, 11:09 AM

Post #6 of 6 (549 views)
Permalink

Re: please consider changing --enable-unicode default to ucs4 [In reply to]

Le lundi 05 octobre 2009 à 19:18 +0200, Jan Matejek a écrit :
>
> I
> don't see what is bad about improving compatibility in a place where the
> setting doesn't hurt one way or the other.

I can't speak for Mandriva, but I'm sure they care more about not
breaking user installs when they upgrade to Mandriva X + 1, than about
making it possible to use Fedora RMPs on Mandriva.

In any case, this is quite off-topic for python-dev. If you are
motivated about this, you should try to convinve the Mandriva developers
instead.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads