Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue18706] test failure in test_codeccallbacks

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Aug 10, 2013, 5:03 PM

Post #1 of 12 (38 views)
Permalink
[issue18706] test failure in test_codeccallbacks

New submission from Antoine Pitrou:

Since issue #18681, the following test sequence fails:

./python -m test -v test_codecencodings_kr test_imp test_codeccallbacks

[...]

======================================================================
ERROR: test_xmlcharnamereplace (test.test_codeccallbacks.CodecCallbackTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/antoine/cpython/default/Lib/test/test_codeccallbacks.py", line 112, in test_xmlcharnamereplace
self.assertEqual(sin.encode("ascii", "test.xmlcharnamereplace"), sout)
File "/home/antoine/cpython/default/Lib/test/test_codeccallbacks.py", line 102, in xmlcharnamereplace
l.append("&%s;" % html.entities.codepoint2name[ord(c)])
AttributeError: 'module' object has no attribute 'entities'

----------
assignee: ezio.melotti
components: Tests
messages: 194854
nosy: ezio.melotti, pitrou
priority: high
severity: normal
stage: needs patch
status: open
title: test failure in test_codeccallbacks
type: behavior
versions: Python 3.3, Python 3.4

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 10, 2013, 6:09 PM

Post #2 of 12 (37 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Terry J. Reedy added the comment:

Failure is intermittent. The only related lines in test_codeccallbacks are

002: import html.entities
102: l.append("&%s;" % html.entities.codepoint2name[ord(c)])

----------
nosy: +terry.reedy

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 10, 2013, 11:11 PM

Post #3 of 12 (31 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Changes by Serhiy Storchaka <storchaka [at] gmail>:


----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 10, 2013, 11:49 PM

Post #4 of 12 (31 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Serhiy Storchaka added the comment:

There are two ways to fix this issue -- one change test_imp.py and other change test_codeccallbacks.py. The proposed patch contains both.

----------
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file31228/issue18706.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 12:42 AM

Post #5 of 12 (31 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Serhiy Storchaka added the comment:

However it looks weird:

>>> def f():
... import html.entities
... del sys.modules['html']
...
>>> f()
>>> import html.entities
>>> html.entities
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'entities'

Perhaps import machinery should be fixed instead of tests.

----------
nosy: +brett.cannon, eric.snow, ncoghlan

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 2:46 AM

Post #6 of 12 (28 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Ezio Melotti added the comment:

> ./python -m test -v test_codecencodings_kr test_imp test_codeccallbacks

Thanks, I was trying to reproduce the failure yesterday with test_imp test_codeccallbacks but it wasn't working -- now I can reproduce it.

My idea was to import html and html.parser and then remove them both, and that's what the patch did:

>>> import sys
>>> m = set(sys.modules)
>>> from html import parser
>>> set(sys.modules) - m
{'html.parser', 'linecache', 'tokenize', 'warnings', '_markupbase', 'html', 'token'}
>>> del sys.modules['html']
>>> set(sys.modules) - m
{'html.parser', 'linecache', 'tokenize', '_markupbase', 'warnings', 'token'}
>>> del sys.modules['html.parser']
>>> set(sys.modules) - m
{'token', '_markupbase', 'tokenize', 'warnings', 'linecache'}

I think the problem is that if test_codecencodings_kr is run before test_imp, test_imp deletes html and html.parser but leaves the html.entities imported by test_codecencodings_kr, and when test_codeccallbacks tries to use it again it fails:

>>> import sys
>>> m = set(sys.modules)
>>> import html.entities
>>> set(sys.modules) - m
{'html.entities', 'html'}
>>> from html import parser
>>> set(sys.modules) - m
{'_markupbase', 'html', 'html.entities', 'tokenize', 'html.parser', 'linecache', 'warnings', 'token'}
>>> del sys.modules['html']
>>> set(sys.modules) - m
{'_markupbase', 'tokenize', 'html.entities', 'token', 'html.parser', 'linecache', 'warnings'}
>>> del sys.modules['html.parser']
>>> set(sys.modules) - m
{'_markupbase', 'tokenize', 'html.entities', 'token', 'linecache', 'warnings'}
>>> html.entities
<module 'html.entities' from '/home/wolf/dev/py/py3k/Lib/html/entities.py'>
>>> import html.entities
>>> html.entities
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'entities'

A solution would be to remove all the modules that start with 'html.' (see attached patch):

>>> import sys
>>> m = set(sys.modules)
>>> import html.entities
>>> from html import parser
>>> del sys.modules['html']
>>> del sys.modules['html.parser']
>>> set(sys.modules) - m
{'warnings', 'linecache', 'token', 'html.entities', 'tokenize', '_markupbase'}
>>> del sys.modules['html.entities']
>>> import html.entities
>>> html.entities
<module 'html.entities' from '/home/wolf/dev/py/py3k/Lib/html/entities.py'>

Another (possibly better) solution would be to copy sys.modules before messing with it and restore it afterwards. I was looking in test.support for a context manager to do it but I didn't find it, so I just deleted the modules I imported manually.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 2:52 AM

Post #7 of 12 (28 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Antoine Pitrou added the comment:

> Perhaps import machinery should be fixed instead of tests.

Yes, the import machinery is acting weird here.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 3:23 AM

Post #8 of 12 (28 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Serhiy Storchaka added the comment:

Perhaps `del sys.modules['module']` should remove all 'module.submodule' from sys.modules. Or perhaps `import module.submodule` should ensure that 'module' is in sys.modules and has the 'submodule' attribute.

Ezio, seems you forgot to attach a patch.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 3:24 AM

Post #9 of 12 (28 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Changes by Ezio Melotti <ezio.melotti [at] gmail>:


Added file: http://bugs.python.org/file31230/issue18706.diff

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 4:05 AM

Post #10 of 12 (28 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Antoine Pitrou added the comment:

My preference goes to Serhiy's fix for test_imp.
Note that the import machinery oddity would deserve fixing too (in a separate issue perhaps ?).

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 8:38 AM

Post #11 of 12 (20 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Ezio Melotti added the comment:

> My preference goes to Serhiy's fix for test_imp.

Fair enough. Serhiy do you want to commit your fix?

> Note that the import machinery oddity would deserve fixing too
> (in a separate issue perhaps ?).

This should be a separate issue.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Aug 11, 2013, 10:14 AM

Post #12 of 12 (20 views)
Permalink
[issue18706] test failure in test_codeccallbacks [In reply to]

Roundup Robot added the comment:

New changeset dab790a17c4d by Serhiy Storchaka in branch '3.3':
Issue #18706: Fix a test for issue #18681 so it no longer breaks test_codeccallbacks*.
http://hg.python.org/cpython/rev/dab790a17c4d

New changeset 1f4aed2c914c by Serhiy Storchaka in branch 'default':
Issue #18706: Fix a test for issue #18681 so it no longer breaks test_codeccallbacks*.
http://hg.python.org/cpython/rev/1f4aed2c914c

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue18706>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.