report at bugs
Jul 25, 2008, 2:06 PM
Post #1 of 2
(241 views)
Permalink
|
|
[issue3448] Multi-process 2to3
|
|
New submission from Nick Edds <nedds [at] uchicago>:
Here is a working, multiprocess version of 2to3 with a few caveats.
First, you need to already have the processing module installed for this
to work. If we don't want to include processing in some way, I think I
can modify this to only import processing and use the Process method if
the user wants to run it with more than one process. Also, I know that
logger is supposed to be thread safe, so I am correct in assuming that
this means it is also multi-process safe? This fix delegates the fixing
of files to a designated number of processes, so on a multi-core
machine, it is significantly faster. It may be appropriate to add a test
suite for it, but I have not yet done so. I've tested it on my dual-core
laptop running Ubuntu and a quad-core mac and it appears to be working
fine. I don't know if the use of tempfile was the right choice, but it
seemed reasonable. Another possibility is to instead using a processing
Queue and pass it the output, filename, and input rather than calling
write_file. Also, the join timeout I used is completely arbitrary
because I don't really have a sense of what a reasonable value for it
should be.
----------
assignee: collinwinter
components: 2to3 (2.x to 3.0 conversion tool)
files: multiProcess.diff
keywords: patch
messages: 70277
nosy: collinwinter, nedds
severity: normal
status: open
title: Multi-process 2to3
type: performance
Added file: http://bugs.python.org/file10985/multiProcess.diff
_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue3448>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
|
