Mailing List Archive: Python: Dev

PEP 442 aftermath: module globals at shutdown

Index | Next | Previous | View Threaded

solipsis at pitrou

Jul 30, 2013, 11:42 AM

Post #1 of 7 (77 views)
Permalink

PEP 442 aftermath: module globals at shutdown

Hello,

PEP 442 has now been committed in time for testing in Alpha 1.

This paves the way for the removal of another well-known annoyance: the
behaviour of module globals at shutdown. Now that reference cycles
aren't a barrier to object finalization anymore, we shouldn't need
to set module globals to None before trying to reclaim modules.
(and then, we don't need to cache global functions for use in
finalizers, either)

I have a patch to suppress the hack in
http://bugs.python.org/issue18214
Once I get to add some tests, I would like to commit it soon too!

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

guido at python

Jul 30, 2013, 12:58 PM

Post #2 of 7 (74 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

I'm very excited to see this happening! It's been a constant pain and
one of the things I've always regretted. Thanks Antoine!

--Guido

On Tue, Jul 30, 2013 at 11:42 AM, Antoine Pitrou <solipsis [at] pitrou> wrote:
>
> Hello,
>
> PEP 442 has now been committed in time for testing in Alpha 1.
>
> This paves the way for the removal of another well-known annoyance: the
> behaviour of module globals at shutdown. Now that reference cycles
> aren't a barrier to object finalization anymore, we shouldn't need
> to set module globals to None before trying to reclaim modules.
> (and then, we don't need to cache global functions for use in
> finalizers, either)
>
> I have a patch to suppress the hack in
> http://bugs.python.org/issue18214
> Once I get to add some tests, I would like to commit it soon too!
>
> Regards
>
> Antoine.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Jul 30, 2013, 2:32 PM

Post #3 of 7 (74 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

On Tue, 30 Jul 2013 12:58:58 -0700
Guido van Rossum <guido [at] python> wrote:
> I'm very excited to see this happening! It's been a constant pain and
> one of the things I've always regretted. Thanks Antoine!

Note this is currently imperfect. I've identified two reasons why a
pure Python module could stay alive even after being removed from
sys.modules:

- it is held alive by a C extension: the main example is the locale
module, which is held alive by _io and in turn keeps alive other
Python modules (such as collections or re). But there is also the
readline module, whose completer can keep alive other stuff (this
is aggravated by the fact the readline module currently doesn't
have a proper dealloc routine).

- it is held alive through builtins: the site module patches builtins
with additional objects, which themselves keep references to the site
module's globals, and with it other modules

Regards

Antoine.

>
> --Guido
>
> On Tue, Jul 30, 2013 at 11:42 AM, Antoine Pitrou <solipsis [at] pitrou> wrote:
> >
> > Hello,
> >
> > PEP 442 has now been committed in time for testing in Alpha 1.
> >
> > This paves the way for the removal of another well-known annoyance: the
> > behaviour of module globals at shutdown. Now that reference cycles
> > aren't a barrier to object finalization anymore, we shouldn't need
> > to set module globals to None before trying to reclaim modules.
> > (and then, we don't need to cache global functions for use in
> > finalizers, either)
> >
> > I have a patch to suppress the hack in
> > http://bugs.python.org/issue18214
> > Once I get to add some tests, I would like to commit it soon too!
> >
> > Regards
> >
> > Antoine.
>

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

greg.ewing at canterbury

Jul 30, 2013, 6:30 PM

Post #4 of 7 (72 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

Antoine Pitrou wrote:
> - it is held alive through builtins: the site module patches builtins
> with additional objects, which themselves keep references to the site
> module's globals,

The module probably *should* stay alive in that case, since
it's still accessible via those patched builtins.

--
Greg
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Jul 30, 2013, 11:21 PM

Post #5 of 7 (68 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

On Wed, 31 Jul 2013 13:30:58 +1200
Greg Ewing <greg.ewing [at] canterbury> wrote:
> Antoine Pitrou wrote:
> > - it is held alive through builtins: the site module patches builtins
> > with additional objects, which themselves keep references to the site
> > module's globals,
>
> The module probably *should* stay alive in that case, since
> it's still accessible via those patched builtins.

Of course. I'm not gonna kill reachable modules like a cowboy on the
loose :-)
The problem is that its global objects will get finalized very late.

Anyway, the solution is to unpatch builtins through an atexit handler
in the site module.

(my patch still has a last step where surviving modules get their
globals wiped, though)

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

martin at v

Aug 1, 2013, 8:03 AM

Post #6 of 7 (60 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

Am 30.07.13 23:32, schrieb Antoine Pitrou:
> - it is held alive by a C extension: the main example is the locale
> module, which is held alive by _io and in turn keeps alive other
> Python modules (such as collections or re).

If the _locale module would use PEP 3121 (issue15662), this problem
should go away.

Regards,
Martin

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Aug 1, 2013, 8:11 AM

Post #7 of 7 (60 views)
Permalink

Re: PEP 442 aftermath: module globals at shutdown [In reply to]

Le Thu, 01 Aug 2013 17:03:03 +0200,
"Martin v. Löwis" <martin [at] v> a écrit :
> Am 30.07.13 23:32, schrieb Antoine Pitrou:
> > - it is held alive by a C extension: the main example is the locale
> > module, which is held alive by _io and in turn keeps alive other
> > Python modules (such as collections or re).
>
> If the _locale module would use PEP 3121 (issue15662), this problem
> should go away.

Not really: I'm talking about the pure Python locale module.
However, I've got another solution for this one (using weakrefs,
unsurprisingly):
http://bugs.python.org/issue18608

cheers

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads