google at mrabarnett
May 16, 2008, 6:21 PM
Post #11 of 14
(201 views)
Permalink
|
|
Re: can't delete from a dictionary in a loop
[In reply to]
|
|
On May 16, 10:22 pm, "Dan Upton" <up...@virginia.edu> wrote:
> This might be more information than necessary, but it's the best way I
> can think of to describe the question without being too vague.
>
> The task:
>
> I have a list of processes (well, strings to execute said processes)
> and I want to, roughly, keep some number N running at a time. If one
> terminates, I want to start the next one in the list, or otherwise,
> just wait.
>
> The attempted solution:
>
> Using subprocess, I Popen the next executable in the list, and store
> it in a dictionary, with keyed on the pid:
> (outside the loop)
> procs_dict={}
>
> (inside a while loop)
> process = Popen(benchmark_exstring[num_started], shell=true)
> procs_dict[process.pid]=process
>
> Then I sleep for a while, then loop through the dictionary to see
> what's terminated. For each one that has terminated, I decrement a
> counter so I know how many to start next time, and then try to remove
> the record from the dictionary (since there's no reason to keep
> polling it since I know it's terminated). Roughly:
>
> for pid in procs_dict:
> if procs_dict[pid].poll() != None
> # do the counter updates
> del procs_dict[pid]
>
> The problem:
>
> RuntimeError: dictionary changed size during iteration
>
> So, the question is: is there a way around this? I know that I can
> just /not/ delete from the dictionary and keep polling each time
> around, but that seems sloppy and like it could keep lots of memory
> around that I don't need, since presumably the dictionary holding a
> reference to the Popen object means the garbage collector could never
> reclaim it. Is the only reasonable solution to do something like
> append all of those pids to a list, and then after I've iterated over
> the dictionary, iterate over the list of pids to delete?
>
> (Also, from the implementation side, is there a reason the dictionary
> iterator can't deal with that? If I was deleting from in front of the
> iterator, maybe, but since I'm deleting from behind it...)
>
Why do you need a counter? len(procs_dict) will tell you how many are
in the dictionary.
You can rebuild the dictionary, excluding those that are no longer
active, with:
procs_dict = dict((id, process) for id, process in
procs_dict.iteritems() if process.poll() != None)
and then start N - len(procs_dict) new processes.
--
http://mail.python.org/mailman/listinfo/python-list
|
