Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue13959] Re-implement parts of imp in pure Python

 

 

First page Previous page 1 2 3 Next page Last page  View All Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Apr 21, 2012, 6:15 PM

Post #26 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset 085cf1480cfe by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.find_module() in Lib/imp.py.
http://hg.python.org/cpython/rev/085cf1480cfe

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 12:41 AM

Post #27 of 56 (301 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

Ported _imp.reload() (Python/import.c) to Lib/imp.py. Included is the change to PyImport_ReloadModule() to make it simply a wrapper around the pure Python imp.reload(). There's a good chance I don't have this right or that I have some reference leak. I haven't worked a ton on the C side of Python (sounds tropical). This patch also removes 'modules_reloading' from the interpreter state, since it's no longer used anywhere (see issue14618).

----------
Added file: http://bugs.python.org/file25302/issue13959_reload.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 3:40 AM

Post #28 of 56 (303 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

A patch for magic and tag. It's not quite finished, but I wanted to see if the approach was palatable. FYI, I'm also trying to push forward the sys.implementation stuff, which would help on the pyc tag.

----------
Added file: http://bugs.python.org/file25303/issue13959_magic.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 8:38 AM

Post #29 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

After thinking about it, is MAGIC an implementation detail? It certainly reflects changes specific to the CPython interpreter. I'm much more comfortable with leaving implementation details in Python/import.c.

On the other hand, there's already no small amount of rather CPython-specific stuff in Lib/importlib/_bootstrap.py, which belongs there. The pyc magic bytes are tightly coupled with it. Because of that, I realize, I'm still fine with the patch.

But it makes me wonder if it might be worth having a clear separation between the general and CPython-specific stuff in _bootstrap.py, for the sake of people who look at the code for the first (or tenth) time. That's the same rationale I have for advocating moving as much over from import.c as relates to the importlib implementation.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 10:02 AM

Post #30 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset 4e853913054c by Brett Cannon in branch 'default':
Issue #13959: Continue to try to accomodate altsep in importlib by not
http://hg.python.org/cpython/rev/4e853913054c

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 12:54 PM

Post #31 of 56 (301 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

First off, you should separate the patches for get_magic() and get_tag(). Second, why is there _get_pyc_magic_int() when it is never called? Third, all of this would be greatly simplified if you just had a _RAW_MAGIC_NUMBER of 3220, did the bytes object creation for _MAGIC_NUMBER in-place (i.e. no separate function), and then in the C code just got _RAW_MAGIC_NUMBER and did the MAGIC macro work there.

As for what is CPython-specific and what isn't, only the other VMs can state that officially, so I'm not going to worry about that yet (but I will ask before Python 3.3 goes out so as to minimize backporting patches in the future). But importlib needs to stabilize more before that can happen.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 12:58 PM

Post #32 of 56 (301 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

Good feedback. The some of that code was the result of directly translating the C. I'll get a new, simpler patch up probably tomorrow night. Thanks.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 5:53 PM

Post #33 of 56 (301 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

How consistent do the semantics of reload() need to remain? (The C version does more type checking than the Python version probably needs to worry about. reload() seems to be one of those bits that doesn't have much test coverage.)

Also, what's the best way to exercise the changes one makes to the C code?

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 7:42 PM

Post #34 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

Loosening the type restrictions is fine.

As for testing, that's what the test suite is supposed to do. So if need be just write tests in Python that exercise the C code.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 9:22 PM

Post #35 of 56 (301 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

To try and narrow down the issue, I now have separate patches for an explicit sys.meta_path and a sys.path_hooks. Both fail on tests, but for different reasons.

The explicit sys.meta_path is still failing on the __main__ issue on three of the tests and for threaded imports on the other.

The explicit sys.path_hooks patch fails on two tests: one on threading and then test_cmd_line_script on apparently lacking an absolute file path thanks to issue #8202 (and why an explicit sys.path_hooks triggers that I don't know).

Both failures Nick might know about.

----------
Added file: http://bugs.python.org/file25308/explicit_meta_path.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 22, 2012, 9:23 PM

Post #36 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Changes by Brett Cannon <brett [at] python>:


Added file: http://bugs.python.org/file25309/explicit_path_hooks.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 23, 2012, 10:38 PM

Post #37 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

updated patch for magic number support in imp/importlib

----------
Added file: http://bugs.python.org/file25335/issue13959_magic.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 23, 2012, 10:40 PM

Post #38 of 56 (302 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

updated patch for moving TAG to importlib/imp.py

----------
Added file: http://bugs.python.org/file25336/issue13959_tag.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 29, 2012, 9:50 AM

Post #39 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset eb5c5c23ca9b by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.NullImporter in Lib/imp.py.
http://hg.python.org/cpython/rev/eb5c5c23ca9b

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 29, 2012, 10:20 AM

Post #40 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

Update time!

With NullImporter dealt with, that leaves get_magic(), get_tag(), reload(), and get_suffixes() as things to potentially move to Lib/imp.py. I would also like to re-implement PyImport_ExecCodeModuleObject() as it's keeping a lot of C code alive just for its personal use.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 29, 2012, 10:23 AM

Post #41 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Éric Araujo <merwok [at] netwok> added the comment:

1659 lines less than 3.2’s import.c so far!

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 29, 2012, 11:40 AM

Post #42 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset eb68502731dd by Brett Cannon in branch 'default':
Issues #13959, 14647: Re-implement imp.reload() in Lib/imp.py.
http://hg.python.org/cpython/rev/eb68502731dd

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 29, 2012, 12:38 PM

Post #43 of 56 (283 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

So here is the deal with PyImport_ExecCodeModuleObject(): bootstrapping and Barry has made this a little annoying. =)

First off, PyImport_ImportFrozenModuleObject() uses PyImport_ExecCodeModuleObject(), so that precludes just directly importing imp to handle this function entirely. OK, so that means just trying to chop out the path manipulation stuff since that is duplicating code found in imp/importlib.

The problem, though, is that PyImport_ExecCodeModuleWithPathnames() will take its pathname argument and try to get a source path from it if it points to some .pyc file (PEP 3147 or sourceless .pyc, and if that new path exists then it is used instead of the given path. Unfortunately that API was introduced in Python 3.2, so there is a backwards-compatibility issue in that one can't just rip out the code w/o supporting it. But those semantics are the reason the equivalent of imp.source_from_cache() continues to exist in Python/import.c.

I see two options here. One is to simply leave the C code in, but that has the drawback of duplicated Python and C code. Two is to stick in a call to imp.source_from_cache() between PyImport_ExecCodeModuleWithPathnames() and PyImport_ExecCodeModuleObject() so the former retains the semantics and the latter doesn't pick up the bad habit before 3.3 is released.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 1, 2012, 7:10 AM

Post #44 of 56 (284 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Benjamin Peterson <benjamin [at] python> added the comment:

Windows is currently failing test_imp:

http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/214/steps/test/logs/stdio

----------
nosy: +benjamin.peterson

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 1, 2012, 7:32 AM

Post #45 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

That test is going to stay intermittent until issue #14657 gets resolved else the exact reason for the failure is going to be hard to debug remotely.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 4, 2012, 12:20 PM

Post #46 of 56 (284 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset 257cbd2fac38 by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.get_suffixes() in Lib/imp.py.
http://hg.python.org/cpython/rev/257cbd2fac38

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 4, 2012, 12:23 PM

Post #47 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Brett Cannon <brett [at] python> added the comment:

OK, I'm waiting on issue #14657 (avoiding the _frozen_importlib/importlib dichotomy) is resolved before I document the new imp.extension_suffixes() as it would be good to know where I should pull in the source and bytecode suffixes.

I think this only leaves get_magic() and get_tag() as the only things to move + trying to figure out how to handle PyImport_ExecCodeModuleObject() and its grip on various bits of C code.

----------
dependencies: +Avoid two importlib copies

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 4, 2012, 1:13 PM

Post #48 of 56 (283 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset 22b0689346f9 by Brett Cannon in branch 'default':
Issue #13959: Move module type constants to Lib/imp.py.
http://hg.python.org/cpython/rev/22b0689346f9

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 11, 2012, 9:59 AM

Post #49 of 56 (284 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset b81ddaf0db47 by Brett Cannon in branch 'default':
Issue #13959: Deprecate imp.get_suffixes() for new attributes on
http://hg.python.org/cpython/rev/b81ddaf0db47

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

May 11, 2012, 10:41 AM

Post #50 of 56 (285 views)
Permalink
[issue13959] Re-implement parts of imp in pure Python [In reply to]

Eric Snow <ericsnowcurrently [at] gmail> added the comment:

Question on this one:

<snip>
@@ -126,7 +131,7 @@ def load_compiled(name, pathname, file=N
# XXX deprecate
def load_package(name, path):
if os.path.isdir(path):
- extensions = _bootstrap._SOURCE_SUFFIXES + [_bootstrap._BYTECODE_SUFFIX]
+ extensions = machinery.SOURCE_SUFFIXES[:] + [machinery.BYTECODE_SUFFIXES]
for extension in extensions:
path = os.path.join(path, '__init__'+extension)
if os.path.exists(path):
</snip>

Should that be the following?

extensions = machinery.SOURCE_SUFFIXES[:] + machinery.BYTECODE_SUFFIXES[:]

Also, why the "[:]"?

Finally, in a couple spots you use the first element of the list (like the old case of "machinery.SOURCE_SUFFIXES[0]" in source_from_cache() and the new one in find_module()). Will this be a problem where the source file has one of the other suffixes? I'm not sure it's a big enough deal for the moment to worry about, but thought I'd ask. :)

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue13959>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

First page Previous page 1 2 3 Next page Last page  View All Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.