Mailing List Archive: Python: Bugs

[issue15316] runpy swallows ImportError information with relative imports

Index | Next | Previous | View Threaded

report at bugs

Aug 21, 2012, 5:15 PM

Post #1 of 15 (192 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports

Chris Jerdonek added the comment:

I randomly ran into this issue again. I'm not sure this was ever resolved (i.e. I think it may always have been different from issue 15111).

I still get the above behavior in the default branch.

And here is what I get in the 3.2 branch (the error information is not swallowed):

$ ./python.exe -m foo
***
Traceback (most recent call last):
File ".../Lib/runpy.py", line 161, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File ".../Lib/runpy.py", line 74, in _run_code
exec(code, run_globals)
File ".../foo/__main__.py", line 1, in <module>
from foo import bar
File "foo/bar.py", line 2, in <module>
raise ImportError('test...')
ImportError: test...

----------
nosy: +brett.cannon, ncoghlan
resolution: duplicate ->
status: closed -> open
superseder: Wrong ImportError message with importlib ->
type: -> behavior

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 21, 2012, 5:25 PM

Post #2 of 15 (190 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Chris Jerdonek added the comment:

I ran into this again because an error while running `./python.exe -m test` was getting masked. The use of __main__.py in the package may be the distinguishing characteristic.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 21, 2012, 7:25 PM

Post #3 of 15 (186 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Changes by Chris Jerdonek <chris.jerdonek [at] gmail>:

----------
keywords: +3.2regression

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 12:21 AM

Post #4 of 15 (187 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Chris Jerdonek added the comment:

Should this issue be fixed before the release? If it is not fixed, certain problems found after the release may become harder to report and diagnose (because the true source of error will be masked).

Two months ago issue 15111 which was thought to be the same was given a priority of "high".

----------
nosy: +georg.brandl, r.david.murray

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 8:02 AM

Post #5 of 15 (186 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Changes by Jeff Knupp <jknupp [at] gmail>:

----------
nosy: +Jeff.Knupp

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 9:33 AM

Post #6 of 15 (185 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Brett Cannon added the comment:

It has nothing to do with runpy and __main__.py and everything to do with rev 78619:0d52f125dd32 (which fixed issue #15715) which was done to ignore bogus names in fromlist. If you simply change the import line to "import foo.bar" then you get the expected result.

The options I see to solve this without invalidating issue #15715 is:

1) Parse the ImportError message to notice that it was a lookup failure and not some other ImportError (ewww)

2) Importlib has to tag every exception is raises because of a finder failure to know when an ImportError was triggered because the module wasn't found (less eww, but still brittle as it means only importlib or people aware of the flag will have the expected semantics)

3) Create a ModuleNotFoundError exception that subclasses ImportError and catch that (breaks doctests and introduces a new exception that people will need to be aware of, plus the question of whether it should just exist in importlib or be a builtin)

4) change importlib._bootstrap._handle_fromlist (and various other bits of importlib code) to call _find_module() and silently ignore when no finder is found as desired (probably need to add a flag to _find_and_load() to signal whether finder failure just returns None or raises ImportError)

While I prefer 3, I think it's a bit late in the release to try to introduce a new exception to begin separating the meaning of 16 different ``raise ImportError`` cases in importlib._bootstrap based on inheritance. My gut says 4 is the best solution given the timeframe. I should be able to get a patch in on Friday if that works for people.

----------
assignee: -> brett.cannon
priority: normal -> release blocker
stage: -> test needed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 12:08 PM

Post #7 of 15 (184 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Chris Jerdonek added the comment:

Here is a formal unit test case that passes in 3.2 but not in 3.3 (a "simpler" case not using __main__.py).

(script_helper.create_empty_file() doesn't seem to be available in 3.2.)

----------
keywords: +patch
Added file: http://bugs.python.org/file26967/issue-15316-failing-test-1.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 12:12 PM

Post #8 of 15 (184 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Brett Cannon added the comment:

Thanks for the test, Chris. It can probably be simplified using the utilities in test_importlib (e.g. managing the cleanup of sys.path, using mocked loaders to raise the exception instead of having to write to the file system, etc.), but otherwise the idea of the test is accurate.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 12:18 PM

Post #9 of 15 (181 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Chris Jerdonek added the comment:

You're welcome, Brett. I'll let you or someone else recast the test using the latest preferred techniques. I was just using the style of the immediately surrounding tests.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 22, 2012, 12:46 PM

Post #10 of 15 (182 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Eric Snow added the comment:

> While I prefer 3, I think it's a bit late in the release to try to
> introduce a new exception to begin separating the meaning of 16
> different ``raise ImportError`` cases in importlib._bootstrap based on
> inheritance. My gut says 4 is the best solution given the timeframe.

Agreed. I still think option 3 would be suitable and have created issue 15767 to track it.

----------
nosy: +eric.snow

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 24, 2012, 8:57 AM

Post #11 of 15 (176 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Georg Brandl added the comment:

I don't agree that this is a blocker; would be nice to fix it, of course.

----------
priority: release blocker -> critical

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 24, 2012, 9:07 AM

Post #12 of 15 (176 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Brett Cannon added the comment:

It will get fixed today.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 24, 2012, 2:44 PM

Post #13 of 15 (174 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Brett Cannon added the comment:

I am running the test suite now using the "secret" attribute on ImportError. I tried to pass a flag, but locking became a bit messy/complicated. And I also realized that if I didn't do this then using different implementation of import_ in importlib wouldn't work because I would be hard-coding in what import implementation was used which would bypass the accelerated code in import.c. Plus with the compatibility issues I have had in the passed, I didn't want to skip a step of import that someone was probably relying on.

Once Python 3.4 comes out I will create a new exception that is raised when no module is found and catch that class specifically to avoid this hack.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 24, 2012, 3:26 PM

Post #14 of 15 (179 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Roundup Robot added the comment:

New changeset ca4bf8e10bc0 by Brett Cannon in branch 'default':
Issue #15316: Let exceptions raised during imports triggered by the
http://hg.python.org/cpython/rev/ca4bf8e10bc0

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Aug 24, 2012, 3:27 PM

Post #15 of 15 (176 views)
Permalink

[issue15316] runpy swallows ImportError information with relative imports [In reply to]

Changes by Brett Cannon <brett [at] python>:

----------
resolution: -> fixed
stage: test needed -> committed/rejected
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue15316>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads