Mailing List Archive: Python: Dev

Why is type_modified() in typeobject.c not a public function?

Index | Next | Previous | View Threaded

stefan_ml at behnel

May 27, 2008, 9:47 AM

Post #1 of 7 (335 views)
Permalink

Why is type_modified() in typeobject.c not a public function?

[.reposting this to python-dev, as it affects both 2.6 and 3.0]

Hi,

when we build extension classes in Cython, we have to first build the type to
make it available to user code, and then update the type's tp_dict while we
run the class body code (PyObject_SetAttr() does not work here). In Py2.6+,
this requires invalidating the method cache after each attribute change, which
Python does internally using the type_modified() function.

Could this function get a public interface? I do not think Cython is the only
case where C code wants to modify a type after its creation, and copying the
code over seems like a hack to me.

Stefan

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

guido at python

May 27, 2008, 10:38 AM

Post #2 of 7 (309 views)
Permalink

Re: Why is type_modified() in typeobject.c not a public function? [In reply to]

I'm fine with giving it a public interface. Please submit a patch with
docs included.

On Tue, May 27, 2008 at 9:47 AM, Stefan Behnel <stefan_ml [at] behnel> wrote:
> [.reposting this to python-dev, as it affects both 2.6 and 3.0]
>
> Hi,
>
> when we build extension classes in Cython, we have to first build the type to
> make it available to user code, and then update the type's tp_dict while we
> run the class body code (PyObject_SetAttr() does not work here). In Py2.6+,
> this requires invalidating the method cache after each attribute change, which
> Python does internally using the type_modified() function.
>
> Could this function get a public interface? I do not think Cython is the only
> case where C code wants to modify a type after its creation, and copying the
> code over seems like a hack to me.
>
> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>

--
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

stefan_ml at behnel

May 27, 2008, 10:47 PM

Post #3 of 7 (310 views)
Permalink

Re: Why is type_modified() in typeobject.c not a public function? [In reply to]

Hi,

Guido van Rossum wrote:
> On Tue, May 27, 2008 at 9:47 AM, Stefan Behnel <stefan_ml [at] behnel> wrote:
>> Could this function get a public interface? I do not think Cython is the only
>> case where C code wants to modify a type after its creation, and copying the
>> code over seems like a hack to me.
>>
> I'm fine with giving it a public interface. Please submit a patch with
> docs included.

Straight forward patch is attached (against 3.0a5). Works for me in Cython. I
thought about a name like "Taint(t)" or "ClearTypeCache(t)", but then went
with the coward solution of calling the function PyType_Modified() as it was
(almost) named internally.

BTW, I noticed that the code in typeobject.c uses "DECREF before set" two
times, like this:

method_cache[h].version = type->tp_version_tag;
method_cache[h].value = res; /* borrowed */
Py_INCREF(name);
Py_DECREF(method_cache[h].name);
method_cache[h].name = name;

During the call to Py_DECREF, the cache content is incorrect, so can't this
run into the same problem that Py_CLEAR() aims to solve? I attached a patch
for that, too, just in case.

Stefan

Attachments:

pytype_modified.patch (3.02 KB)

possible-decref-before-set-fix.patch (1.28 KB)

stefan_ml at behnel

May 28, 2008, 4:41 AM

Post #4 of 7 (309 views)
Permalink

Re: Why is type modified() in typeobject.c not a public function? [In reply to]

Stefan Behnel wrote:
> Straight forward patch is attached (against 3.0a5).
[...]
> BTW, I noticed that the code in typeobject.c uses "DECREF before set" two
> times.

Filed on the bug tracker as issues 2989 and 2990.

http://bugs.python.org/issue2989
http://bugs.python.org/issue2990

Stefan

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

May 28, 2008, 4:43 AM

Post #5 of 7 (308 views)
Permalink

Re: Why is type_modified() in typeobject.c not a public function? [In reply to]

Stefan Behnel schrieb:
> Straight forward patch is attached (against 3.0a5). Works for me in Cython. I
> thought about a name like "Taint(t)" or "ClearTypeCache(t)", but then went
> with the coward solution of calling the function PyType_Modified() as it was
> (almost) named internally.
>
> BTW, I noticed that the code in typeobject.c uses "DECREF before set" two
> times, like this:
>
> method_cache[h].version = type->tp_version_tag;
> method_cache[h].value = res; /* borrowed */
> Py_INCREF(name);
> Py_DECREF(method_cache[h].name);
> method_cache[h].name = name;
>
> During the call to Py_DECREF, the cache content is incorrect, so can't this
> run into the same problem that Py_CLEAR() aims to solve? I attached a patch
> for that, too, just in case.

Please create two tickets in the bug tracker and append the patches.

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

May 29, 2008, 2:22 AM

Post #6 of 7 (304 views)
Permalink

Re: Why is type modified() in typeobject.c not a public function? [In reply to]

Stefan Behnel <stefan_ml <at> behnel.de> writes:
> BTW, I noticed that the code in typeobject.c uses "DECREF before set" two
> times, like this:
>
> method_cache[h].version = type->tp_version_tag;
> method_cache[h].value = res; /* borrowed */
> Py_INCREF(name);
> Py_DECREF(method_cache[h].name);
> method_cache[h].name = name;

Since this is so common, would it be useful to define a macro for the correct
construct?

Something like

#define Py_SETREF(var, obj) \
{ PyObject *tmp = (var); Py_INCREF(obj); \
(var) = (obj); Py_XDECREF(tmp); }

Or, if we want to allow more optimizations, make two versions of it:

#define Py_SETREF(var, obj) \
{ PyObject *tmp = (var); Py_INCREF(obj); \
(var) = (obj); Py_DECREF(tmp); }

#define Py_XSETREF(var, obj) \
{ PyObject *tmp = (var); Py_INCREF(obj); \
(var) = (obj); Py_XDECREF(tmp); }

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

stefan_ml at behnel

May 29, 2008, 2:55 AM

Post #7 of 7 (290 views)
Permalink

Re: Py_SETREF/Py_XSETREF (was: Why is type_modified() in typeobject.c not a public function?) [In reply to]

Antoine Pitrou wrote:
> Stefan Behnel <stefan_ml <at> behnel.de> writes:
>> BTW, I noticed that the code in typeobject.c uses "DECREF before set" two
>> times, like this:
>>
>> method_cache[h].version = type->tp_version_tag;
>> method_cache[h].value = res; /* borrowed */
>> Py_INCREF(name);
>> Py_DECREF(method_cache[h].name);
>> method_cache[h].name = name;
>
> Since this is so common, would it be useful to define a macro for the correct
> construct?
>
> Something like
>
> #define Py_SETREF(var, obj) \
> { PyObject *tmp = (var); Py_INCREF(obj); \
> (var) = (obj); Py_XDECREF(tmp); }
>
> Or, if we want to allow more optimizations, make two versions of it:
>
> #define Py_SETREF(var, obj) \
> { PyObject *tmp = (var); Py_INCREF(obj); \
> (var) = (obj); Py_DECREF(tmp); }
>
> #define Py_XSETREF(var, obj) \
> { PyObject *tmp = (var); Py_INCREF(obj); \
> (var) = (obj); Py_XDECREF(tmp); }

Both sound like a good idea to me. Having to think about whether or not DECREF
poses a problem in a specific case is just cumbersome.

Stefan

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads