Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Calling Methods from Pythons C API with Keywords

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


cbarton at metavr

Jun 19, 2007, 7:21 AM

Post #1 of 10 (443 views)
Permalink
Calling Methods from Pythons C API with Keywords

Hey Guys,

My first post on this list so I hope this is the right place to post and
relevant.
Im rewriting parts of the Blender3D python API that has got a bit old
and needs an update.

Im making a PyList subtype with the C/Python API, this involves
intercepting calls to standard list methods to make sure Blenders array
data is in Sync with the list's data.

Iv got it working for tp_as_sequence, tp_as_mapping, iter and dealloc
etc but methods are a problem.

I want to add my own call's before and after PyLists standard functions
but have a proplem with functons that use keywords and have no API
equivalent.
For example, I cant use the API's PyList_Sort because that dosnt support
keywords like...

ls.sort(key=lambda a: a.foo))

And the Problem with PyObject_CallMethod is that it dosnt accept keywords.

PyObject_CallMethod((PyObject *)mylist, "sort", "O", args);


Looking at abstract.c, PyObject_CallMethod uses call_function_tail,
which calls "PyObject_Call(callable, args, NULL);" - so Its not
currently possible with PyObject_CallMethod.

But I cant find any way to do this in a few lines.

I could use PyEval_CallObjectWithKeywords but that would mean Id need to
get the method from the list manually which Ill look into, but unless Im
missing something here, it seems PyObject_CallMethodWithKeywords would
be a nice addition to the Python API that cant be done in a straight
forward way at the moment.

- Thanks


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


hrvoje.niksic at avl

Jun 20, 2007, 12:34 AM

Post #2 of 10 (421 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
> I want to add my own call's before and after PyLists standard functions
> but have a proplem with functons that use keywords and have no API
> equivalent.
> For example, I cant use the API's PyList_Sort because that dosnt support
> keywords like...
>
> ls.sort(key=lambda a: a.foo))
>
> And the Problem with PyObject_CallMethod is that it dosnt accept keywords.

Note that you can always simply call PyObject_Call on the bound method
object retrieved using PyObject_GetAttrString. The hardest part is
usually constructing the keywords dictionary, a job best left to
Py_BuildValue and friends. When I need that kind of thing in more than
one place, I end up with a utility function like this one:

/* Equivalent to PyObject_CallMethod but accepts keyword args. The
format... arguments should produce a dictionary that will be passed
as keyword arguments to obj.method.

Usage example:
PyObject *res = call_method(lst, "sort", "{s:O}", "key", keyfun));
*/

PyObject *
call_method(PyObject *obj, const char *methname, char *format, ...)
{
va_list va;
PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL;

args = PyTuple_New(0);
if (!args)
goto out;
meth = PyObject_GetAttrString(obj, methname);
if (!meth)
goto out;

va_start(va, format);
kwds = Py_VaBuildValue(format, va);
va_end(va);
if (!kwds)
goto out;

ret = PyObject_Call(meth, args, kwds);
out:
Py_XDECREF(meth);
Py_XDECREF(args);
Py_XDECREF(kwds);
return ret;
}

It would be nice for the Python C API to support a more convenient way
of calling objects and methods with keyword arguments.


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


cbarton at metavr

Jun 20, 2007, 3:17 AM

Post #3 of 10 (424 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

Hrvoje Nikšić wrote:
> On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
>> I want to add my own call's before and after PyLists standard functions
>> but have a proplem with functons that use keywords and have no API
>> equivalent.
>> For example, I cant use the API's PyList_Sort because that dosnt support
>> keywords like...
>>
>> ls.sort(key=lambda a: a.foo))
>>
>> And the Problem with PyObject_CallMethod is that it dosnt accept keywords.
>
> Note that you can always simply call PyObject_Call on the bound method
> object retrieved using PyObject_GetAttrString. The hardest part is
> usually constructing the keywords dictionary, a job best left to
> Py_BuildValue and friends. When I need that kind of thing in more than
> one place, I end up with a utility function like this one:
>
> /* Equivalent to PyObject_CallMethod but accepts keyword args. The
> format... arguments should produce a dictionary that will be passed
> as keyword arguments to obj.method.
>
> Usage example:
> PyObject *res = call_method(lst, "sort", "{s:O}", "key", keyfun));
> */
>
> PyObject *
> call_method(PyObject *obj, const char *methname, char *format, ...)
> {
> va_list va;
> PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL;
>
> args = PyTuple_New(0);
> if (!args)
> goto out;
> meth = PyObject_GetAttrString(obj, methname);
> if (!meth)
> goto out;
>
> va_start(va, format);
> kwds = Py_VaBuildValue(format, va);
> va_end(va);
> if (!kwds)
> goto out;
>
> ret = PyObject_Call(meth, args, kwds);
> out:
> Py_XDECREF(meth);
> Py_XDECREF(args);
> Py_XDECREF(kwds);
> return ret;
> }
>
> It would be nice for the Python C API to support a more convenient way
> of calling objects and methods with keyword arguments.


Thanks for the hint, I ended up using PyObject_Call.
This seems to work, EXPP_PyTuple_New_Prepend - is a utility function
that returns a new tuple with self at the start (needed so args starts
with self)

I dont think I can use PyObject_GetAttrString because the subtype would
return a reference to this function - rather then the lists original
function, Id need an instance of a list and dont have one at that point.
______________________

static PyObject * MaterialList_sort(BPy_MaterialList *self, PyObject
*args, PyObject *keywds )
{
PyObject *ret;
PyObject *newargs = EXPP_PyTuple_New_Prepend(args, (PyObject *)self);

sync_list_from_materials__internal(self); # makes sure the list matches
blenders materials

ret = PyObject_Call(PyDict_GetItemString(PyList_Type.tp_dict, "sort"),
newargs, keywds);
Py_DECREF(newargs);

if (ret)
sync_materials_from_list__internal(self); # makes blenders materials
match the lists

return ret;
}

_____________________

Later on Ill probably avoid using PyDict_GetItemString on
PyList_Type.tp_dict all the time since the methods for lists does not
change during python running. - Can probably be assigned to a constant.


--
Campbell J Barton (ideasman42)
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


hrvoje.niksic at avl

Jun 20, 2007, 4:38 AM

Post #4 of 10 (425 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

[. Note that this discussion, except maybe for the suggestion to add a
simpler way to call a method with keyword args, is off-topic to
python-dev. ]

On Wed, 2007-06-20 at 20:17 +1000, Campbell Barton wrote:
> I dont think I can use PyObject_GetAttrString because the subtype would
> return a reference to this function - rather then the lists original
> function, Id need an instance of a list and dont have one at that point.

Note that PyList_Type is a full-fledged PyObject, so
PyObject_GetAttrString works on it just fine. Of course, you would also
need to add the "self" argument before the keywords, but that's a
trivial change to the function.

Calling PyObject_GetAttrString feels cleaner than accessing tp_dict
directly, and most importantly call_method as written delegates creation
of the dictionaty to Py_BuildValue.


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


cbarton at metavr

Jun 20, 2007, 5:12 AM

Post #5 of 10 (422 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

Hrvoje Nikšić wrote:
> [. Note that this discussion, except maybe for the suggestion to add a
> simpler way to call a method with keyword args, is off-topic to
> python-dev. ]

Is there a list for this kind of discussion?
Iv tried asking questions on the freenode python chat room but almost
very few people there do C/Python api development.

--
Campbell J Barton (ideasman42)
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


hrvoje.niksic at avl

Jun 20, 2007, 6:00 AM

Post #6 of 10 (420 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

On Wed, 2007-06-20 at 22:12 +1000, Campbell Barton wrote:
> Hrvoje Nikšić wrote:
> > [. Note that this discussion, except maybe for the suggestion to add a
> > simpler way to call a method with keyword args, is off-topic to
> > python-dev. ]
>
> Is there a list for this kind of discussion?

I believe the appropriate list would be the general Python
list/newsgroup. I agree that response about the Python/C API tends to
be sparse on general-purpose lists, though.

If there is a forum dedicated to discussing the *use* of Python at the C
level, I'd like to know about it as well.


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

Jun 20, 2007, 10:26 AM

Post #7 of 10 (428 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

Campbell Barton schrieb:
> Hrvoje Nikšić wrote:
>> [. Note that this discussion, except maybe for the suggestion to add a
>> simpler way to call a method with keyword args, is off-topic to
>> python-dev. ]
>
> Is there a list for this kind of discussion?

Hrvoje wasn't explicit on *why* this discussion is inappropriate here,
so I just add that for better understanding:

python-dev is for the development *of* Python, not for the development
*with* Python. So you post here if you propose an enhancement or discuss
the resolution of a bug. Question of the "how do I" kind are off-topic -
posters are expected to know and understand the options, and then
discuss the flaws of these options, rather than asking what they are.

As Hrvoje says: try python-list (aka comp.lang.python). If you don't
get an answer, you didn't phrase your question interestingly enough,
or nobody knows the answer, or nobody has the time to tell you.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


hrvoje.niksic at avl

Jun 21, 2007, 4:33 AM

Post #8 of 10 (424 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

On Wed, 2007-06-20 at 19:26 +0200, "Martin v. Löwis" wrote:
> As Hrvoje says: try python-list (aka comp.lang.python). If you don't
> get an answer, you didn't phrase your question interestingly enough,
> or nobody knows the answer, or nobody has the time to tell you.

The thing with comp.lang.python is that it is followed by a large number
of Python users, but a much smaller number of the C API users -- which
is only natural, since the group is about Python, not about C. For most
users the Python/C API is an implementation detail which they never have
to worry about.

Futrhermore, questions about the C API often concern CPython
implementation details and so they don't feel like they would belong in
comp.lang.python. As an experiment, it might make sense to open a
mailing list dedicated to the Python C API. It could become a useful
support forum for extension writers (a group very useful to Python) and
maybe even a melting pot for new ideas regarding CPython, much like
comp.lang.python historically provided ideas for Python the language.


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


cbarton at metavr

Jun 21, 2007, 4:59 AM

Post #9 of 10 (419 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

The reason I asked on this in the first place is I had looked through
the python source to make sure PyObject_Call had no equivalent that
supported keywords, and since I needed to do this I figured it might be
worth considering for Pythons API.

Im sure everyone could write their own PyObject_Call, if they had to but
thats what the API's for.

Hrvoje Nikšić wrote:
> On Wed, 2007-06-20 at 19:26 +0200, "Martin v. Löwis" wrote:
>> As Hrvoje says: try python-list (aka comp.lang.python). If you don't
>> get an answer, you didn't phrase your question interestingly enough,
>> or nobody knows the answer, or nobody has the time to tell you.
>
> The thing with comp.lang.python is that it is followed by a large number
> of Python users, but a much smaller number of the C API users -- which
> is only natural, since the group is about Python, not about C. For most
> users the Python/C API is an implementation detail which they never have
> to worry about.
>
> Futrhermore, questions about the C API often concern CPython
> implementation details and so they don't feel like they would belong in
> comp.lang.python. As an experiment, it might make sense to open a
> mailing list dedicated to the Python C API. It could become a useful
> support forum for extension writers (a group very useful to Python) and
> maybe even a melting pot for new ideas regarding CPython, much like
> comp.lang.python historically provided ideas for Python the language.

Agree a Python/C API List would be great, in fact I cant see any reasons
not to have it- likely the pure python users dont want to know about
refcounting problems.. etc anyway.

http://mail.python.org/mailman/listinfo
http://www.python.org/community/sigs/
There are lists/newsgroup for py2exe and pyrex, Python-ObjectiveC etc,

Python/C API seems much more generic, and its also fairly tricky to use
at times - when doing more advanced stuff (subtyping has been tricky for
me anyway). I expect the dev's of pyrex, pygame etc might also need to
discuss C API spesific issues as well.

Iv had roughly this conversation in IRC...

Q. Hi, Id like to know how wrap python subtype methods in the C API
A. C dosnt have classes, use C++
Q. no I want to use pythons C API,
A. Subtypes are easy to do in python..
...... you get the idea...

Quite a few "python only" users dont understand where the Python/C API
fits in and its annoying to have to explain the question each time (yes,
Iv had these conversations more then once)


- Cam
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

Jun 21, 2007, 10:25 AM

Post #10 of 10 (420 views)
Permalink
Re: Calling Methods from Pythons C API with Keywords [In reply to]

> Futrhermore, questions about the C API often concern CPython
> implementation details and so they don't feel like they would belong in
> comp.lang.python. As an experiment, it might make sense to open a
> mailing list dedicated to the Python C API. It could become a useful
> support forum for extension writers (a group very useful to Python) and
> maybe even a melting pot for new ideas regarding CPython, much like
> comp.lang.python historically provided ideas for Python the language.

In the past, we created special-interest groups for such discussion.
Would you like to coordinate a C sig? See

http://www.python.org/community/sigs/

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.