Mailing List Archive: Python: Bugs

[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe

Index | Next | Previous | View Threaded

report at bugs

Jul 13, 2009, 10:11 AM

Post #1 of 3 (308 views)
Permalink

[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe

New submission from Jose Fonseca <jose.r.fonseca [at] gmail>:

MSVCRT's implementation of _spawnve, _spawnvpe, or any version that
takes an environ as paramater is not thread-safe, because it stores a
temporary environment string into a global variable.

_spawnve, _spawnvpe, and friends call a function named _cenvarg which
concatenate the environment strings into a global variable called
_aenvptr, which gets free'd and zero'd after each invocation.

This was causing random build failures in scons when parallel build (-j)
was enabled.

The sample application evidences this problem. It also includes a simple
workaround in python, by acquiring a global lock around os.spawnve, and
simulating P_WAIT with P_NOWAIT to avoid holding the global lock while
the child process is running. I believe something along these lines
should be done for CPython on Windows.

----------
components: Interpreter Core
files: spawnve.py
messages: 90495
nosy: jfonseca
severity: normal
status: open
title: MSVCRT's spawnve/spawnvpe are not thread safe
type: crash
versions: Python 2.4, Python 2.5, Python 2.6
Added file: http://bugs.python.org/file14495/spawnve.py

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue6476>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 13, 2009, 12:02 PM

Post #2 of 3 (277 views)
Permalink

[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe [In reply to]

Amaury Forgeot d'Arc <amauryfa [at] gmail> added the comment:

Indeed. But shouldn't you use the subprocess module instead?

----------
nosy: +amaury.forgeotdarc
resolution: -> wont fix
status: open -> pending

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue6476>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Jul 13, 2009, 2:09 PM

Post #3 of 3 (290 views)
Permalink

[issue6476] MSVCRT's spawnve/spawnvpe are not thread safe [In reply to]

Jose Fonseca <jose.r.fonseca [at] gmail> added the comment:

Perhaps. I'm not a scons developer -- just an user -- and I don't know
what versions of python far back in time they want support, but it
appears it would make sense to use subprocess where available indeed. I
already I've filled an issue with scons at
http://scons.tigris.org/issues/show_bug.cgi?id=2449 and linked back to
this issue and I trust the developers to do whatever they see fit to
address this problem.

Instead of just marking this issue as won't fix, shouldn't at least some
documentation be added, or something sensible like that? In
http://docs.python.org/library/os.html#process-management os.spawn* are
not marked as deprecated -- just says that "the subprocess module
provides more powerful facilities" and is "preferable" --, and it
certainly doesn't say these functions are not thread safe. It would be a
pity if other users would have to invest as much time as I did to find
this race condition (it was rare, and happened in a build farm so we
couldn't see the windows access violation dialog), and multithreading is
getting more and more common.

Also, if the only reason not to fix this is the lack of a patch I don't
mind in producing one FWIW.

----------
status: pending -> open

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue6476>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads