Mailing List Archive: Python: Bugs

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error

Index | Next | Previous | View Threaded

report at bugs

Jun 26, 2012, 8:25 PM

Post #1 of 9 (134 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error

New submission from Ian Bell <ian.h.bell [at] gmail>:

When a non-picklable object is sent through a multiprocessing.Pipe, no exception is raised, instead when trying to read the other end of the pipe, a TypeError is raised:

TypeError: Required argument 'handle' (pos 1) not found

----------
components: Windows
messages: 164118
nosy: Ian.Bell
priority: normal
severity: normal
status: open
title: multiprocessing Pipe send of non-picklable objects doesn't raise error
versions: Python 2.7

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 26, 2012, 8:26 PM

Post #2 of 9 (124 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Changes by Ian Bell <ian.h.bell [at] gmail>:

----------
type: -> crash

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 27, 2012, 4:17 AM

Post #3 of 9 (120 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Changes by Antoine Pitrou <pitrou [at] free>:

----------
nosy: +sbt

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 30, 2012, 12:25 PM

Post #4 of 9 (115 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Richard Oudkerk <shibturn [at] gmail> added the comment:

What type of object did you try to send, and how can the problem be reproduced?

There are plenty of types which don't support pickling, and where pickling only succeeds in producing invalid data which cannot be successfully unpickled. The exception raised when unpickling will depend on the type of the object.

----------
type: crash -> behavior

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jun 30, 2012, 12:40 PM

Post #5 of 9 (115 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Ian Bell <ian.h.bell [at] gmail> added the comment:

I had issues with a class that I wrote myself. It is a rather involved data structure with all kinds of interesting things going on. Unfortunately I cannot put together a minimal working example that will cause a Python hang.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 1:16 AM

Post #6 of 9 (116 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Richard Oudkerk <shibturn [at] gmail> added the comment:

Then I doubt this is a bug in Python.

If your class does not override __reduce__, __reduce_ex__ or __getstate__/__setstate__, then it is probably one of the attributes of your instance which is causing the problem. You could try to find out which one by trying

cPickle.loads(cPickle.dumps(attribute, -1))

for each attribute of an instance of your class to see if it raises an error.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 1:26 AM

Post #7 of 9 (120 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Senthil Kumaran <senthil [at] uthcode> added the comment:

On Sun, Jul 1, 2012 at 1:16 AM, Richard Oudkerk <report [at] bugs> wrote:
> Then I doubt this is a bug in Python.

I guess, you meant, this is NOT a bug in Python.

----------
nosy: +orsenthil

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 10:22 AM

Post #8 of 9 (122 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Ian Bell <ian.h.bell [at] gmail> added the comment:

I have repaired my class so that it pickles properly, but that does not resolve the issue that if you send a non-picklable object through a pipe, it should raise an error, rather than hang.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 1, 2012, 11:10 AM

Post #9 of 9 (116 views)
Permalink

[issue15198] multiprocessing Pipe send of non-picklable objects doesn't raise error [In reply to]

Richard Oudkerk <shibturn [at] gmail> added the comment:

> I have repaired my class so that it pickles properly, but that does not
> resolve the issue that if you send a non-picklable object through a pipe,
> it should raise an error, rather than hang.

What do you mean by hang? Do you mean that send() succeeds and recv() raises an error. That might cause your *program* to hang, but send() itself should not hang.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15198>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads