mheieis at alois
Jan 18, 2014, 11:22 AM
Re: Porting c extension - PyBuffer_New() deprecated in python3. What's the replacement?
[In reply to]
Thank-you for the reply. I hadn't considered cpython, unfortunately the
extension is too large a project to port at the moment. I ended up
replacing the PyBuffer_New() segment with malloc() and passing back an
object from PyByteArray_FromStringAndSize(). It seems to work.
On 2014-01-11 01:10, Stefan Behnel wrote:
> Mark Heieis, 11.01.2014 06:47:
>> I need to convert the following existing c extension code to support Python 3.
>> // --- existing code ------
>> // PyBuffer_New() deprecated in python3
>> if (!(pyBuf = PyBuffer_New(len)))
>> return NULL;
>> // should use memoryview object in python3
>> if (PyObject_AsWriteBuffer(pyBuf, &cbuf, &len))
>> return NULL ;
>> // fill in cbuf
>> return pyBuf ;
>> I'm somewhat confounded in finding an equivalent (PyBuffer_New()) for
>> creating a buffer of size len that has continuous memory in the c extension
>> function for python3. cbuf is manipulated/filled in using c, after which
>> the created pyBuf is then returned. So far, I haven't found much in the way
>> of examples/doc for porting the deprecated Python-/C-level buffer API calls
>> to the new C-level buffer API/memoryview object model.
>> Any guidance or direction to existing doc/example is much appreciated.
> If the extension isn't huge, you should consider rewriting it in Cython.
> That can usually be done quite quickly - the main thing is to figure out
> what the verbose C code actually does and write it down in much simpler
> Python code. And it will make it easy to make the code portable and fast.
> Also likely safer and more generic and versatile, because Cython covers
> away a lot of the annoying boilerplate, ref-counting issues, type
> conversions, etc.
> For your specific problem at hand, you could use Cython's memory views:
> They allow you to convert the input value to a 1-dim char buffer (or
> whatever you need, but you mentioned the old Py2 buffer interface, which
> can't do much more) by saying
> cdef char[:] my_memview = some_python_object
> If you need to pass the unpacked buffer into C code, you can get the
> address as "&my_memview" (i.e. the address of the first item in the
> buffer). Memory views themselves support fast slicing and indexing, so you
> can efficiently work with them using the normal Python slicing/indexing syntax.
> In case what you actually receive are not arbitrary buffers but simple byte
> strings or bytearray instances, you can even use the normal byte string
> coercion in Cython and simply say
> cdef char* c_string = some_python_byte_string_object
> and then use that pointer to pass it on into C.
> I've written a string processing tutorial for Cython here:
> These things may take a moment to learn, especially if you are used to
> doing everything in excessive manual detail in C code, but once you are
> through that, you should get things done much more quickly than when trying
> to do them by hand.