Mailing List Archive: Python: Python

Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception

Index | Next | Previous | View Threaded

heikki at osafoundation

Jul 23, 2008, 11:42 AM

Post #1 of 5 (423 views)
Permalink

Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception

I was debugging M2Crypto function written in C which changed behavior
between Python 2.6 and earlier Python versions. In an error condition
the function was supposed to raise exception type A, but with 2.6 it
raised type B, and further, there was no string value for the exception.

I tracked this down to the C code incorrectly returning Py_None when it
should have returned NULL. Changing the C code to return NULL made it
behave correctly in 2.6.

I don't know how common a mistake it is to return Py_None when NULL
should have been returned, but it might be worth a note in the list of
changes for 2.6 that this behavior changed, don't you think?

--
Heikki Toivonen
--
http://mail.python.org/mailman/listinfo/python-list

fredrik at pythonware

Jul 23, 2008, 12:45 PM

Post #2 of 5 (392 views)
Permalink

Re: Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception [In reply to]

Heikki Toivonen wrote:

> I was debugging M2Crypto function written in C which changed behavior
> between Python 2.6 and earlier Python versions. In an error condition
> the function was supposed to raise exception type A, but with 2.6 it
> raised type B, and further, there was no string value for the exception.
>
> I tracked this down to the C code incorrectly returning Py_None when it
> should have returned NULL. Changing the C code to return NULL made it
> behave correctly in 2.6.
>
> I don't know how common a mistake it is to return Py_None when NULL
> should have been returned, but it might be worth a note in the list of
> changes for 2.6 that this behavior changed, don't you think?

the behaviour when setting the exception status without returning NULL
has never been well-defined; it pretty much depends on what kind of
error checking the code that runs later happens to use.

(this has been an entertaining source of obscure errors over the years;
code that uses PyErr_Clear may mask errors, and code that uses
PyErr_Occurred to check if something went wrong might raise the wrong
error, often at an unexpected location).

</F>

--
http://mail.python.org/mailman/listinfo/python-list

martin at v

Jul 23, 2008, 3:31 PM

Post #3 of 5 (385 views)
Permalink

Re: Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception [In reply to]

Heikki Toivonen wrote:
> I was debugging M2Crypto function written in C which changed behavior
> between Python 2.6 and earlier Python versions. In an error condition
> the function was supposed to raise exception type A, but with 2.6 it
> raised type B, and further, there was no string value for the exception.
>
> I tracked this down to the C code incorrectly returning Py_None when it
> should have returned NULL. Changing the C code to return NULL made it
> behave correctly in 2.6.

Can you please be specific what function you are talking about?

> I don't know how common a mistake it is to return Py_None when NULL
> should have been returned, but it might be worth a note in the list of
> changes for 2.6 that this behavior changed, don't you think?

Perhaps. OTOH, perhaps the change is completely erroneous. In that case,
rather than documenting it, it should be reverted.

Unfortunately, as you keep the specific issue secret, none of this will
happen, as we have no clue what you are talking about.

I'm sure there are tons of silent changes, in this release, all past
releases, and all future releases, not only in Python, but in any
software.

Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

heikki at osafoundation

Jul 23, 2008, 10:33 PM

Post #4 of 5 (374 views)
Permalink

Re: Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception [In reply to]

Martin v. Löwis wrote:
> Heikki Toivonen wrote:
>> I tracked this down to the C code incorrectly returning Py_None when it
>> should have returned NULL. Changing the C code to return NULL made it
>> behave correctly in 2.6.
>
> Can you please be specific what function you are talking about?

Perhaps it wasn't clear that I was referring to the C code in an
extension, M2Crypto. I assumed that this affected all extension code
like this, which is why I didn't mention the actual lines. It appears
from Fredrik's comment that this might not be just a 2.6 issue, but that
this problem has cropped up in the past as well more or less randomly.

>> I don't know how common a mistake it is to return Py_None when NULL
>> should have been returned, but it might be worth a note in the list of
>> changes for 2.6 that this behavior changed, don't you think?
>
> Perhaps. OTOH, perhaps the change is completely erroneous. In that case,
> rather than documenting it, it should be reverted.
>
> Unfortunately, as you keep the specific issue secret, none of this will
> happen, as we have no clue what you are talking about.

I don't know what change in Python caused the change in M2Crypto
behavior. I can only point you to the change I made in M2Crypto if you
are interested:
http://viewcvs.osafoundation.org/m2crypto/trunk/SWIG/_pkcs7.i?rev=531&r1=611&r2=531

If you revert that change and run the M2Crypto unit tests you will see
the single error in the tests.

> I'm sure there are tons of silent changes, in this release, all past
> releases, and all future releases, not only in Python, but in any
> software.

Given that there is a long document showing the changes in each Python
release, I would hope all intended changes of significance would be
listed. Of course mistakes can happen, which was why I posted in the
first place.

I have no plans to track down the exact change in Python code that
caused this. There does not seem to be much point, since according to
Fredrik this seems to be an area that is practically undefined and the
M2Crypto code was clearly buggy.

--
Heikki Toivonen
--
http://mail.python.org/mailman/listinfo/python-list

martin at v

Jul 24, 2008, 1:21 PM

Post #5 of 5 (376 views)
Permalink

Re: Undocumented Python 2.6 change: Py_None vs NULL when C implementation raises exception [In reply to]

> I have no plans to track down the exact change in Python code that
> caused this. There does not seem to be much point, since according to
> Fredrik this seems to be an area that is practically undefined and the
> M2Crypto code was clearly buggy.

I see, and I agree with Fredrik's analysis. It might actually be that
there was *no* change in 2.6 causing this change in behavior, but just
a difference in data returned in the actual application. E.g. if a
PyInt_FromLong returns -1, the caller also needs to check for
PyErr_Occurred, which would then detect an earlier exception. So it
might be that if you have -1 in your data, you see the exception, but
if you have -2 instead, you won't see it.

Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads