Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

Unexpected python exception

  

 
Python python RSS feed   Index | Next | Previous | View Threaded


rpurdie at rpsys

Nov 11, 2009, 2:59 AM

Post #1 of 12 (353 views)
Permalink
Unexpected python exception
I've been having problems with an unexpected exception from python which
I can summarise with the following testcase:

def A():
import __builtin__
import os

__builtin__.os = os

def B():
os.stat("/")
import os

A()
B()

which results in:

Traceback (most recent call last):
File "./test.py", line 12, in <module>
B()
File "./test.py", line 8, in B
os.stat("/")
UnboundLocalError: local variable 'os' referenced before assignment

If I remove the "import os" from B(), it works as expected.

>From what I've seen, its very unusual to have something operate
"backwards" in scope in python. Can anyone explain why this happens?

Cheers,

Richard

--
http://mail.python.org/mailman/listinfo/python-list


deets at nospam

Nov 11, 2009, 3:21 AM

Post #2 of 12 (341 views)
Permalink
Re: Unexpected python exception [In reply to]
Richard Purdie schrieb:
> I've been having problems with an unexpected exception from python which
> I can summarise with the following testcase:
>
> def A():
> import __builtin__
> import os
>
> __builtin__.os = os
>
> def B():
> os.stat("/")
> import os
>
> A()
> B()
>
> which results in:
>
> Traceback (most recent call last):
> File "./test.py", line 12, in <module>
> B()
> File "./test.py", line 8, in B
> os.stat("/")
> UnboundLocalError: local variable 'os' referenced before assignment
>
> If I remove the "import os" from B(), it works as expected.
>
>>From what I've seen, its very unusual to have something operate
> "backwards" in scope in python. Can anyone explain why this happens?

As the import-statement in a function/method-scope doesn't leak the
imported names into the module scope, python treats them as locals.
Which makes your code equivalent to


x = 1000

def foo():
print x
x = 10

Throws the same error. The remedy is to inform python that a specific
name belongs to global scope, using the "global"-statement.

def foo():
global x
print x
x = 10


Beware though that then of course *assigning* to x is on global level.
This shouldn't be of any difference in your case though, because of the
import-only-once-mechanics of python.

Diez
--
http://mail.python.org/mailman/listinfo/python-list


andreas.loescher at s2005

Nov 11, 2009, 3:25 AM

Post #3 of 12 (261 views)
Permalink
Re: Unexpected python exception [In reply to]
Hi,
unfortunatley I cannot reproduce your error. Which Python Version do you
use?

The expected case in this scenario is that the exception is thrown, as
you import os in A() where it is stored in the local namespace of the
function.

I tested it with Python 2.4, 2.5 and 2.6 and in both cases an exception
is thrown.

Best,
Andreas

--
http://mail.python.org/mailman/listinfo/python-list


lenz at joinville

Nov 11, 2009, 4:20 AM

Post #4 of 12 (339 views)
Permalink
Re: Unexpected python exception [In reply to]
Em Qua 11 Nov 2009, s 03:21:55, Diez B. Roggisch escreveu:
> Richard Purdie schrieb:
> > I've been having problems with an unexpected exception from python which
> > I can summarise with the following testcase:
> >
> > def A():
> > import __builtin__
> > import os
> >
> > __builtin__.os = os
> >
> > def B():
> > os.stat("/")
> > import os
> >
> > A()
> > B()
> >
> > which results in:
> >
> > Traceback (most recent call last):
> > File "./test.py", line 12, in <module>
> > B()
> > File "./test.py", line 8, in B
> > os.stat("/")
> > UnboundLocalError: local variable 'os' referenced before assignment
> >
> > If I remove the "import os" from B(), it works as expected.
> >
> >>From what I've seen, its very unusual to have something operate
> >
> > "backwards" in scope in python. Can anyone explain why this happens?
>
> As the import-statement in a function/method-scope doesn't leak the
> imported names into the module scope, python treats them as locals.
> Which makes your code equivalent to
>
>
> x = 1000
>
> def foo():
> print x
> x = 10
>
> Throws the same error. The remedy is to inform python that a specific
> name belongs to global scope, using the "global"-statement.
>
> def foo():
> global x
> print x
> x = 10
>
>
> Beware though that then of course *assigning* to x is on global level.
> This shouldn't be of any difference in your case though, because of the
> import-only-once-mechanics of python.
>
> Diez
>

So...it should not work

def A():
import __builtin__
import os
__builtin__.os = os

A()
os.stat("/")

but it does. Why ? B() cannot see the import, but the global level can ?

Thanks.

Eduardo.

--
Esta mensagem foi verificada pelo sistema de antivrus e
acredita-se estar livre de perigo.

--
http://mail.python.org/mailman/listinfo/python-list


clp2 at rebertia

Nov 11, 2009, 4:23 AM

Post #5 of 12 (337 views)
Permalink
Re: Unexpected python exception [In reply to]
On Wed, Nov 11, 2009 at 8:49 AM, Eduardo Lenz <lenz [at] joinville> wrote:
> Em Qua 11 Nov 2009, às 03:21:55, Diez B. Roggisch escreveu:
>> Richard Purdie schrieb:
>> > I've been having problems with an unexpected exception from python which
>> > I can summarise with the following testcase:
>> >
>> > def A():
>> >     import __builtin__
>> >     import os
>> >
>> >     __builtin__.os = os
>> >
>> > def B():
>> >     os.stat("/")
>> >     import os
>> >
>> > A()
>> > B()
>> >
>> > which results in:
>> >
>> > Traceback (most recent call last):
>> >   File "./test.py", line 12, in <module>
>> >     B()
>> >   File "./test.py", line 8, in B
>> >     os.stat("/")
>> > UnboundLocalError: local variable 'os' referenced before assignment
>> >
>> > If I remove the "import os" from B(), it works as expected.
>> >
>> >>From what I've seen, its very unusual to have something operate
>> >
>> > "backwards" in scope in python. Can anyone explain why this happens?
>>
>> As the import-statement in a function/method-scope doesn't leak the
>> imported names into the module scope, python treats them as locals.
>> Which makes your code equivalent to
>>
>>
>> x = 1000
>>
>> def foo():
>>      print x
>>      x = 10
>>
>> Throws the same error. The remedy is to inform python that a specific
>> name belongs to global scope, using the "global"-statement.
>>
>> def foo():
>>      global x
>>      print x
>>      x = 10
>>
>>
>> Beware though that then of course *assigning* to x is on global level.
>> This shouldn't be of any difference in your case though, because of the
>> import-only-once-mechanics of python.
>>
>> Diez
>>
>
> So...it should not work
>
> def A():
>     import __builtin__
>     import os
>     __builtin__.os = os
>
> A()
> os.stat("/")
>
> but it does.  Why ? B() cannot see the import, but the global level can ?

The optimization which results in the behavior in question is only
done on functions scopes, not global scope.

Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list


andreas.loescher at s2005

Nov 11, 2009, 4:23 AM

Post #6 of 12 (261 views)
Permalink
Re: Unexpected python exception [In reply to]
Python searches for Variables not only in local or global scoop but also
in __builtins__. If you do something like __builtins__.os = os, than
this variable should be accessible global.

If you then write something like:
def B():
os.stat("/")
import os

Python recognises on compile time, that os is a local variable in B and
allocates memory for it. All reading or writing now goes to the local
variable and not the one in __builtins__. And the local variable has
never been assigned to a value and an Exception is thrown.

Best

--
http://mail.python.org/mailman/listinfo/python-list


rpurdie at rpsys

Nov 11, 2009, 4:37 AM

Post #7 of 12 (333 views)
Permalink
Re: Unexpected python exception [In reply to]
On Wed, 2009-11-11 at 12:21 +0100, Diez B. Roggisch wrote:
> As the import-statement in a function/method-scope doesn't leak the
> imported names into the module scope, python treats them as locals.
> Which makes your code equivalent to
>
>
> x = 1000
>
> def foo():
> print x
> x = 10

Aha, thanks. This makes it clear whats happening.

> Throws the same error. The remedy is to inform python that a specific
> name belongs to global scope, using the "global"-statement.
>
> def foo():
> global x
> print x
> x = 10
>
>
> Beware though that then of course *assigning* to x is on global level.
> This shouldn't be of any difference in your case though, because of the
> import-only-once-mechanics of python.

Is there a way to make the "global x" apply to all functions without
adding it to each one?

I suspect this equates to intentionally "leaking the imported names into
the module scope"? :)

What I'm trying to do is to avoid having "import X" statements
everywhere by changing __builtin__. It seems my approach doesn't have
quite the same effect as a true import though.

Cheers,

Richard

--
http://mail.python.org/mailman/listinfo/python-list


clp2 at rebertia

Nov 11, 2009, 5:04 AM

Post #8 of 12 (334 views)
Permalink
Re: Unexpected python exception [In reply to]
On Wed, Nov 11, 2009 at 4:37 AM, Richard Purdie <rpurdie [at] rpsys> wrote:
<snip>
> Is there a way to make the "global x" apply to all functions without
> adding it to each one?

Thankfully, no.

> What I'm trying to do is to avoid having "import X" statements
> everywhere by changing __builtin__. It seems my approach doesn't have
> quite the same effect as a true import though.

And you can't just put all your imports together at the top of the
file because...?
If you're importing the same stuff across multiple modules, I'd say
you have a code smell on your hands. Injecting stuff into the builtin
namespace purely for convenience's sake is Just Evil (tm). At the
least, you can put all the imports in one module and then use `from
all_imports import *`, which is just slightly less evil.

Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list


ralaxmyself at gmail

Nov 11, 2009, 6:11 AM

Post #9 of 12 (334 views)
Permalink
Re: Unexpected python exception [In reply to]
On Nov 11, 6:59pm, Richard Purdie <rpur...@rpsys.net> wrote:
> I've been having problems with an unexpected exception from python which
> I can summarise with the following testcase:
>
> def A():
> import __builtin__
> import os
>
> __builtin__.os = os
>
> def B():
> os.stat("/")
> import os
>
> A()
> B()
>
> which results in:
>
> Traceback (most recent call last):
> File "./test.py", line 12, in <module>
> B()
> File "./test.py", line 8, in B
> os.stat("/")
> UnboundLocalError: local variable 'os' referenced before assignment
>
> If I remove the "import os" from B(), it works as expected.
>
> >From what I've seen, its very unusual to have something operate
>
> "backwards" in scope in python. Can anyone explain why this happens?
>
> Cheers,
>
> Richard

One word, Python treat objects in a function or method-scope as
locals.
So os is not defined in B(), is it right?
--
http://mail.python.org/mailman/listinfo/python-list


rpurdie at rpsys

Nov 11, 2009, 6:29 AM

Post #10 of 12 (333 views)
Permalink
Re: Unexpected python exception [In reply to]
On Wed, 2009-11-11 at 05:04 -0800, Chris Rebert wrote:
> On Wed, Nov 11, 2009 at 4:37 AM, Richard Purdie <rpurdie [at] rpsys> wrote:
> <snip>
> > Is there a way to make the "global x" apply to all functions without
> > adding it to each one?
>
> Thankfully, no.

Hmm :(.

> > What I'm trying to do is to avoid having "import X" statements
> > everywhere by changing __builtin__. It seems my approach doesn't have
> > quite the same effect as a true import though.
>
> And you can't just put all your imports together at the top of the
> file because...?

The application in question is bitbake, the parsing tool for the
OpenEmbedded and Poky projects.

We're talking about python code fragments spread across about 8,000
files which make up the "metadata" some of which is public and some of
which is not. Rightly or wrongly bitbake does some interesting things
with its execution contexts for these code fragments.

Lets just say its not a traditional use case, we know the way bitbake
does some things is evil but it does other things rather well and we
can't break those.

Cheers,

Richard



--
http://mail.python.org/mailman/listinfo/python-list


gagsl-py2 at yahoo

Nov 13, 2009, 4:07 AM

Post #11 of 12 (305 views)
Permalink
Re: Unexpected python exception [In reply to]
En Wed, 11 Nov 2009 11:11:31 -0300, Ralax <ralaxmyself [at] gmail> escribió:
> On Nov 11, 6:59 pm, Richard Purdie <rpur...@rpsys.net> wrote:

>> def B():
>> os.stat("/")
>> import os
>>
>> Traceback (most recent call last):
>> File "./test.py", line 12, in <module>
>> B()
>> File "./test.py", line 8, in B
>> os.stat("/")
>> UnboundLocalError: local variable 'os' referenced before assignment
>>
>> If I remove the "import os" from B(), it works as expected.
>> From what I've seen, its very unusual to have something operate
>> "backwards" in scope in python. Can anyone explain why this happens?
>
> One word, Python treat objects in a function or method-scope as
> locals.

Not exactly; from the Language Reference [1]:

"The following constructs bind names: formal parameters to functions,
import statements, class and function definitions (these bind the class or
function name in the defining block), and targets that are identifiers if
occurring in an assignment, for loop header, or in the second position of
an except clause header. The import statement of the form “from ...import
*” binds all names defined in the imported module, except those beginning
with an underscore. This form may only be used at the module level. [...]
If a name binding operation occurs anywhere within a code block, all uses
of the name within the block are treated as references to the current
block. This can lead to errors when a name is used within a block before
it is bound. This rule is subtle. Python lacks declarations and allows
name binding operations to occur anywhere within a code block. The local
variables of a code block can be determined by scanning the entire text of
the block for name binding operations."

> So os is not defined in B(), is it right?

"os" is a local name due to import, and starts uninitialized; using it
before it is initialized raises UnboundLocalError

[1] http://docs.python.org/reference/executionmodel.html

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list


tino at wildenhain

Nov 13, 2009, 9:52 AM

Post #12 of 12 (305 views)
Permalink
Re: Unexpected python exception [In reply to]
Am 11.11.2009 15:29, schrieb Richard Purdie:
> On Wed, 2009-11-11 at 05:04 -0800, Chris Rebert wrote:
>> On Wed, Nov 11, 2009 at 4:37 AM, Richard Purdie<rpurdie [at] rpsys> wrote:
>> <snip>
>>> Is there a way to make the "global x" apply to all functions without
>>> adding it to each one?
>>
>> Thankfully, no.
>
> Hmm :(.
>
>>> What I'm trying to do is to avoid having "import X" statements
>>> everywhere by changing __builtin__. It seems my approach doesn't have
>>> quite the same effect as a true import though.
>>
>> And you can't just put all your imports together at the top of the
>> file because...?
>
> The application in question is bitbake, the parsing tool for the
> OpenEmbedded and Poky projects.
>
> We're talking about python code fragments spread across about 8,000
> files which make up the "metadata" some of which is public and some of
> which is not. Rightly or wrongly bitbake does some interesting things
> with its execution contexts for these code fragments.

And you can't just use __import__() to programmatically include some
of the fragments instead of having this local/global weirdness in
funtions in those modules?

> Lets just say its not a traditional use case, we know the way bitbake
> does some things is evil but it does other things rather well and we
> can't break those.

I'd need to see the bigger picture because I don't know of bitbake but I
have a feeling this can be done much nicer in the end.

Regards
Tino
Attachments: smime.p7s (3.18 KB)

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.