Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

Why does this leak memory?

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


no at spam

Jun 7, 2012, 11:48 AM

Post #1 of 7 (493 views)
Permalink
Why does this leak memory?

When I run this program:

import configparser, sys, gc
def test():
config=configparser.ConfigParser()
#del(config)
#gc.collect()
test()
sys.stderr.write(sys.version)
gc.set_debug(gc.DEBUG_LEAK|gc.DEBUG_STATS)

It reports:

3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)]gc:
collecting generation 2...
gc: objects in each generation: 453 258 4553
gc: collectable <ConfigParser 026C0050>
gc: collectable <dict 0264DE40>
gc: collectable <OrderedDict 0200FB78>
gc: collectable <_Link 02713300>
gc: collectable <dict 0264D930>
gc: collectable <weakproxy 026BC180>
gc: collectable <OrderedDict 0200FC10>
gc: collectable <_Link 02713350>
gc: collectable <dict 0264D9C0>
gc: collectable <weakproxy 026BC1B0>
gc: collectable <OrderedDict 0200FCA8>
gc: collectable <_Link 02713378>
gc: collectable <dict 0264DC90>
gc: collectable <weakproxy 026BC210>
gc: collectable <SectionProxy 026C0030>
gc: collectable <dict 0264D5D0>
gc: collectable <_Link 02713328>
gc: collectable <dict 0269D8A0>
gc: collectable <weakproxy 026BC150>
gc: done, 19 unreachable, 0 uncollectable, 0.0000s elapsed.

The leaks can be removed by uncommenting both lines shown.

This strikes me as very odd behaviour. Can anyone explain it, or is it a
bug?

Thanks,

S.




--
http://mail.python.org/mailman/listinfo/python-list


gordon at panix

Jun 7, 2012, 1:46 PM

Post #2 of 7 (488 views)
Permalink
Re: Why does this leak memory? [In reply to]

In <jqqt5s$ejd$1 [at] speranza> "Steve" <no [at] spam> writes:

> gc: objects in each generation: 453 258 4553
> gc: collectable <ConfigParser 026C0050>
> gc: collectable <dict 0264DE40>
> gc: collectable <OrderedDict 0200FB78>
> gc: collectable <_Link 02713300>
> gc: collectable <dict 0264D930>
> gc: collectable <weakproxy 026BC180>
> gc: collectable <OrderedDict 0200FC10>
> gc: collectable <_Link 02713350>
> gc: collectable <dict 0264D9C0>
> gc: collectable <weakproxy 026BC1B0>
> gc: collectable <OrderedDict 0200FCA8>
> gc: collectable <_Link 02713378>
> gc: collectable <dict 0264DC90>
> gc: collectable <weakproxy 026BC210>
> gc: collectable <SectionProxy 026C0030>
> gc: collectable <dict 0264D5D0>
> gc: collectable <_Link 02713328>
> gc: collectable <dict 0269D8A0>
> gc: collectable <weakproxy 026BC150>
> gc: done, 19 unreachable, 0 uncollectable, 0.0000s elapsed.

> The leaks can be removed by uncommenting both lines shown.

> This strikes me as very odd behaviour. Can anyone explain it, or is it a
> bug?

I'm unfamiliar with gc output, but just glancing over it I don't see
anything that looks like a leak. It reported that there were 19 objects
which are unreachable and therefore are candidates for being collected.

What makes you think there is a leak?

--
John Gordon A is for Amy, who fell down the stairs
gordon [at] panix B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

--
http://mail.python.org/mailman/listinfo/python-list


ian.g.kelly at gmail

Jun 7, 2012, 3:31 PM

Post #3 of 7 (485 views)
Permalink
Re: Why does this leak memory? [In reply to]

On Thu, Jun 7, 2012 at 12:48 PM, Steve <no [at] spam> wrote:
> The leaks can be removed by uncommenting both lines shown.

That's just a matter of timing. You call the function before you call
set_debug, so when you uncomment the lines, the garbage collector is
explicitly run before the debug flags are set. When it runs again
later, there are no collectable objects for it to find, as they've
already been collected. With the lines commented, the garbage
collector doesn't get run until after the set_debug call, so it finds
the collectable objects at that time. Either way the objects get
collected sooner or later, so I don't see any leaks here.

Cheers,
Ian
--
http://mail.python.org/mailman/listinfo/python-list


ian.g.kelly at gmail

Jun 7, 2012, 3:41 PM

Post #4 of 7 (488 views)
Permalink
Re: Why does this leak memory? [In reply to]

For comparison, here is what a leaking program would look like:

class Foo(object):

def __init__(self, other=None):
if other is None:
other = Foo(self)
self.other = other

def __del__(self):
pass

import gc
gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_LEAK)

for i in range(5):
foo = Foo()
del foo
gc.collect()
print "len(gc.garbage) ==", len(gc.garbage)


gc: collecting generation 2...
gc: objects in each generation: 167 3363 0
gc: uncollectable <Foo 00B818F0>
gc: uncollectable <Foo 00B81930>
gc: uncollectable <dict 00B80ED0>
gc: uncollectable <dict 00B86270>
gc: done, 4 unreachable, 4 uncollectable, 0.0000s elapsed.
len(gc.garbage) == 4
gc: collecting generation 2...
gc: objects in each generation: 4 0 3463
gc: uncollectable <Foo 00B81910>
gc: uncollectable <Foo 00B81970>
gc: uncollectable <dict 00B861E0>
gc: uncollectable <dict 00B860C0>
gc: done, 4 unreachable, 4 uncollectable, 0.0000s elapsed.
len(gc.garbage) == 8
gc: collecting generation 2...
gc: objects in each generation: 4 0 3467
gc: uncollectable <Foo 00B81950>
gc: uncollectable <Foo 00B819B0>
gc: uncollectable <dict 00B80F60>
gc: uncollectable <dict 00B86030>
gc: done, 4 unreachable, 4 uncollectable, 0.0000s elapsed.
len(gc.garbage) == 12
gc: collecting generation 2...
gc: objects in each generation: 4 0 3471
gc: uncollectable <Foo 00B81990>
gc: uncollectable <Foo 00B819F0>
gc: uncollectable <dict 00B86390>
gc: uncollectable <dict 00B86150>
gc: done, 4 unreachable, 4 uncollectable, 0.0000s elapsed.
len(gc.garbage) == 16
gc: collecting generation 2...
gc: objects in each generation: 4 0 3475
gc: uncollectable <Foo 00B819D0>
gc: uncollectable <Foo 00B81A30>
gc: uncollectable <dict 00B866F0>
gc: uncollectable <dict 00B86420>
gc: done, 4 unreachable, 4 uncollectable, 0.0000s elapsed.
len(gc.garbage) == 20
gc: collecting generation 2...
gc: objects in each generation: 0 0 3475
gc: done, 0.0000s elapsed.


Note that on each iteration, it reports only the four new
uncollectable objects, not any of the previous uncollectable objects,
because when they are found to be uncollectable they are added to
gc.garbage and so become reachable again.

Cheers,
Ian
--
http://mail.python.org/mailman/listinfo/python-list


no at spam

Jun 8, 2012, 9:02 AM

Post #5 of 7 (479 views)
Permalink
Re: Why does this leak memory? [In reply to]

> "John Gordon" wrote in message news:jqr3v5$src$1 [at] reader1
>
> I'm unfamiliar with gc output, but just glancing over it I don't see
> anything that looks like a leak. It reported that there were 19 objects
> which are unreachable and therefore are candidates for being collected.
>
> What makes you think there is a leak?

Well, I guess I was confused by the terminology. I thought there were leaked
objects _after_ a garbage collection had been run (as it said "collecting
generation 2"). Also, "unreachable" actually appears to mean "unreferenced".
You live n learn...

Cheers.


--
http://mail.python.org/mailman/listinfo/python-list


ian.g.kelly at gmail

Jun 8, 2012, 10:04 AM

Post #6 of 7 (483 views)
Permalink
Re: Why does this leak memory? [In reply to]

On Fri, Jun 8, 2012 at 10:02 AM, Steve <no [at] spam> wrote:
> Well, I guess I was confused by the terminology. I thought there were leaked
> objects _after_ a garbage collection had been run (as it said "collecting
> generation 2").

That means that it's going to check all objects. The garbage
collector divides the objects up into "generations", based on how many
collection attempts they've so far survived (due to not being
unreachable). For efficiency, higher generation objects are checked
less frequently than lower generation objects, and generation 2 is the
highest.

> Also, "unreachable" actually appears to mean "unreferenced".

Not exactly. CPython uses reference counting as well as periodic
garbage collection, so an unreferenced object is deallocated
immediately. "Unreachable" means that the object is referenced but
cannot be reached via direct or indirect reference from any stack
frame -- i.e., the object is only referenced in unreachable reference
cycles.
--
http://mail.python.org/mailman/listinfo/python-list


ulrich.eckhardt at dominolaser

Jun 11, 2012, 12:56 AM

Post #7 of 7 (472 views)
Permalink
Re: Why does this leak memory? [In reply to]

Am 08.06.2012 18:02, schrieb Steve:
> Well, I guess I was confused by the terminology. I thought there were
> leaked objects _after_ a garbage collection had been run (as it said
> "collecting generation 2"). Also, "unreachable" actually appears to mean
> "unreferenced". You live n learn...

Actually I understand that differently. If you have circular references
between two objects they are both referenced. If neither is referenced
(directly or indirectly) by the current context, they are unreachable
and can be garbage-collected. Being unreferenced implies that it is
unreachable, but not vice-versa.

Uli
--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.