Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Mar 12, 2012, 9:58 PM

Post #1 of 6 (89 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code

New submission from Luke Macken <lmacken [at] redhat>:

Problem:


lib2to3.fixer_util.touch_import('__future__', ...) will insert the import statement below all other imports. This will then trigger the following error:

SyntaxError: from __future__ imports must occur at the beginning of the file


How to reproduce the issue (using modernize, which uses lib2to3):

$ git clone https://github.com/mitsuhiko/python-modernize.git
$ cd python-modernize
$ echo << EOF >> test.py
# test.py
"Test case for lib2to3.fixer_util.touch_import('__future__', ...)"
import os
print "hi"
EOF

$ python modernize.py test.py
--- test.py (original)
+++ test.py (refactored)
@@ -1,4 +1,5 @@
# test.py
"Test case for lib2to3.fixer_util.touch_import('__future__', ...)"
import os
-print "hi"
+from __future__ import print_function
+print("hi")

$ python3 test.py
File "test.py", line 4
from __future__ import print_function
^
SyntaxError: from __future__ imports must occur at the beginning of the file


The following patch to lib2to3 seems to solve the issue:

--- /usr/lib64/python2.7/lib2to3/fixer_util.py.orig 2012-03-09 21:53:10.841083479 -0800
+++ /usr/lib64/python2.7/lib2to3/fixer_util.py 2012-03-09 21:58:18.678946683 -0800
@@ -306,14 +306,15 @@
# figure out where to insert the new import. First try to find
# the first import and then skip to the last one.
insert_pos = offset = 0
- for idx, node in enumerate(root.children):
- if not is_import_stmt(node):
- continue
- for offset, node2 in enumerate(root.children[idx:]):
- if not is_import_stmt(node2):
- break
- insert_pos = idx + offset
- break
+ if package != '__future__':
+ for idx, node in enumerate(root.children):
+ if not is_import_stmt(node):
+ continue
+ for offset, node2 in enumerate(root.children[idx:]):
+ if not is_import_stmt(node2):
+ break
+ insert_pos = idx + offset
+ break

# if there are no imports where we can insert, find the docstring.
# if that also fails, we stick to the beginning of the file


After the patch, all is well:

$ python modernize.py test.py
--- test.py (original)
+++ test.py (refactored)
@@ -1,4 +1,5 @@
# test.py
"Test case for lib2to3.fixer_util.touch_import('__future__', ...)"
+from __future__ import print_function
import os
-print "hi"
+print("hi")

$ python3 test.py
hi

----------
components: 2to3 (2.x to 3.x conversion tool)
messages: 155569
nosy: lmacken
priority: normal
severity: normal
status: open
title: lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code
type: behavior

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 12, 2012, 10:07 PM

Post #2 of 6 (88 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code [In reply to]

Changes by Luke Macken <lmacken [at] redhat>:


----------
keywords: +patch
Added file: http://bugs.python.org/file24814/python-lib2to3-touch-future-import.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 13, 2012, 5:04 AM

Post #3 of 6 (82 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

I think that's actually a bug in python-modernize, not in touch_import - or, rather, touch_import shouldn't be used to add future imports.

Instead, I think there should be a touch_future function which adds a future import, taking into account that future imports need to go before all other imports, but after a possible docstring.

----------
nosy: +loewis

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 13, 2012, 6:17 AM

Post #4 of 6 (80 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code [In reply to]

Martin v. Löwis <martin [at] v> added the comment:

Instead of changing touch_import, I propose to add a function similar to

https://github.com/loewis/python-modernize/commit/0db885e616807d0cc6859b4035d81fd260b06a67

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 13, 2012, 2:06 PM

Post #5 of 6 (84 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code [In reply to]

Luke Macken <lmacken [at] redhat> added the comment:

Yep, that seems like the right solution. Thanks, Martin.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 13, 2012, 3:05 PM

Post #6 of 6 (80 views)
Permalink
[issue14282] lib2to3.fixer_util.touch_import('__future__', ...) can lead to SyntaxError in code [In reply to]

Changes by Benjamin Peterson <benjamin [at] python>:


----------
resolution: -> works for me
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14282>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.