Mailing List Archive: Python: Python

Basic importing question

Index | Next | Previous | View Threaded

hubaghdadi at gmail

Aug 20, 2008, 3:08 AM

Post #1 of 11 (101 views)
Permalink

Basic importing question

Hey,
Suppose I have a Python application consists of many modules (lets say
it is a Django application).
If all the modules files are importing sys module, how many times the
sys module will be compiled and executed?
Only once (the first time the PVM locates, compiles and executes the
sys module)? or once for each module importing sys?
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list

bruno.42.desthuilliers at websiteburo

Aug 20, 2008, 3:42 AM

Post #2 of 11 (94 views)
Permalink

Re: Basic importing question [In reply to]

Hussein B a écrit :
> Hey,
> Suppose I have a Python application consists of many modules (lets say
> it is a Django application).
> If all the modules files are importing sys module, how many times the
> sys module will be compiled and executed?
> Only once (the first time the PVM locates, compiles and executes the
> sys module)? or once for each module importing sys?

Hmmm.... Well, could it be possible that Python's developper were smart
enough to implement some cache mechanism here ?-)

import do (approximately) the following:

check if the module already exists in sys.modules
if not:
locate the module's source in sys.path
if not found
raise an import error
look for a compiled .py file
if there's none, or if there's one and it's older than the .py file:
compile the source and save it as .pyc
execute the .pyc file and build a module object from it
add the module object to sys.modules
bind the relevant names into your current namespace
--
http://mail.python.org/mailman/listinfo/python-list

sjmachin at lexicon

Aug 20, 2008, 3:43 AM

Post #3 of 11 (94 views)
Permalink

Re: Basic importing question [In reply to]

On Aug 20, 8:08 pm, Hussein B <hubaghd...@gmail.com> wrote:
> Hey,
> Suppose I have a Python application consists of many modules (lets say
> it is a Django application).
> If all the modules files are importing sys module, how many times the
> sys module will be compiled and executed?
> Only once (the first time the PVM locates, compiles and executes the
> sys module)? or once for each module importing sys?
> Thanks.

sys is a built-in module, so the answer is zero times.

For a non-builtin module foo where there exists:
(1) only a foo.py, it will be compiled into foo.pyc
(2) only a foo.pyc, it will be used
(3) both a foo.py and a foo.pyc, Python compiles the foo.py if the pyc
is out of date or (so I believe [*]) was created by a different
version of Python.

Subsequent imports will use the in-memory copy (in sys.modules, IIRC
[*]) ...

[*] == Please save me the bother of checking this in the manual :-)

HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

hubaghdadi at gmail

Aug 20, 2008, 3:53 AM

Post #4 of 11 (94 views)
Permalink

Re: Basic importing question [In reply to]

On Aug 20, 5:43 am, John Machin <sjmac...@lexicon.net> wrote:
> On Aug 20, 8:08 pm, Hussein B <hubaghd...@gmail.com> wrote:
>
> > Hey,
> > Suppose I have a Python application consists of many modules (lets say
> > it is a Django application).
> > If all the modules files are importing sys module, how many times the
> > sys module will be compiled and executed?
> > Only once (the first time the PVM locates, compiles and executes the
> > sys module)? or once for each module importing sys?
> > Thanks.
>
> sys is a built-in module, so the answer is zero times.
>
> For a non-builtin module foo where there exists:
> (1) only a foo.py, it will be compiled into foo.pyc
> (2) only a foo.pyc, it will be used
> (3) both a foo.py and a foo.pyc, Python compiles the foo.py if the pyc
> is out of date or (so I believe [*]) was created by a different
> version of Python.
>
> Subsequent imports will use the in-memory copy (in sys.modules, IIRC
> [*]) ...
>
> [*] == Please save me the bother of checking this in the manual :-)
>
> HTH,
> John

Thank you both for your kind help and patience :)
Built-in modules are compiled but even if they are so, when importing
them (sys for example), Python will run their code in order to create
bindings and objects, right?
I'm learning Python and I want to learn it well, so that I'm asking a
lot :)
--
http://mail.python.org/mailman/listinfo/python-list

hubaghdadi at gmail

Aug 20, 2008, 4:30 AM

Post #5 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

On Aug 20, 5:43 am, John Machin <sjmac...@lexicon.net> wrote:
> On Aug 20, 8:08 pm, Hussein B <hubaghd...@gmail.com> wrote:
>
> > Hey,
> > Suppose I have a Python application consists of many modules (lets say
> > it is a Django application).
> > If all the modules files are importing sys module, how many times the
> > sys module will be compiled and executed?
> > Only once (the first time the PVM locates, compiles and executes the
> > sys module)? or once for each module importing sys?
> > Thanks.
>
> sys is a built-in module, so the answer is zero times.
>
> For a non-builtin module foo where there exists:
> (1) only a foo.py, it will be compiled into foo.pyc
> (2) only a foo.pyc, it will be used
> (3) both a foo.py and a foo.pyc, Python compiles the foo.py if the pyc
> is out of date or (so I believe [*]) was created by a different
> version of Python.
>
> Subsequent imports will use the in-memory copy (in sys.modules, IIRC
> [*]) ...
>
> [*] == Please save me the bother of checking this in the manual :-)
>
> HTH,
> John

One more question:
If I have this structure:
orig/com/domain/project/Klass1.py
Klass2.py
__init__.py

Why com, domain, project should have __init__.py also? they don't
contain source code files?
Thanks again.
--
http://mail.python.org/mailman/listinfo/python-list

bruno.42.desthuilliers at websiteburo

Aug 20, 2008, 4:39 AM

Post #6 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

Hussein B a écrit :
(snip)
> One more question:
> If I have this structure:
> orig/com/domain/project/Klass1.py
> Klass2.py
> __init__.py
>
> Why com, domain, project should have __init__.py also?

Yes, why should they ? Unless you want to mimic Java's package system so
you do "import com.domain.project.stuff" - which is IMHO a pretty bad
idea -, there's just no reason to turn all your filesystem into a python
package.

Python's philosophy here is that flat is better than nested.
--
http://mail.python.org/mailman/listinfo/python-list

hubaghdadi at gmail

Aug 20, 2008, 4:45 AM

Post #7 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

On Aug 20, 6:39 am, Bruno Desthuilliers <bruno.
42.desthuilli...@websiteburo.invalid> wrote:
> Hussein B a écrit :
> (snip)
>
> > One more question:
> > If I have this structure:
> > orig/com/domain/project/Klass1.py
> > Klass2.py
> > __init__.py
>
> > Why com, domain, project should have __init__.py also?
>
> Yes, why should they ? Unless you want to mimic Java's package system so
> you do "import com.domain.project.stuff" - which is IMHO a pretty bad
> idea -, there's just no reason to turn all your filesystem into a python
> package.
>
> Python's philosophy here is that flat is better than nested.

Sorry I don't follow you :(
Module package is also nested.
--
http://mail.python.org/mailman/listinfo/python-list

lists at cheimes

Aug 20, 2008, 4:46 AM

Post #8 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

Hussein B wrote:
> Thank you both for your kind help and patience :)
> Built-in modules are compiled but even if they are so, when importing
> them (sys for example), Python will run their code in order to create
> bindings and objects, right?
> I'm learning Python and I want to learn it well, so that I'm asking a
> lot :)

Yes, Python has to compile and execute the module before it can be used.
But this happens only *once* for every module during the life time of a
Python process. In Python modules are singletons. The second import just
retrieves the module for an internal cache (sys.modules) and stores it
in the current namespace.

The answer to your question "If all the modules files are importing sys
module, how many times the sys module will be compiled and executed?" is
once.

By the way sys is a very special module, which is *always* loaded and
initialized durings startup. The sys module is not only a builtin module
(embeded into the Python executable / library). It's also part of the
heart of Python and treated more special than any other module.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

bruno.42.desthuilliers at websiteburo

Aug 20, 2008, 4:54 AM

Post #9 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

Hussein B a écrit :
(snip)
> Thank you both for your kind help and patience :)
> Built-in modules are compiled

For which definition of "compiled" ?

> but even if they are so, when importing
> them (sys for example), Python will run their code in order to create
> bindings and objects, right?

As the name imply, built-in modules are built in the interpreter - IOW,
they are part of the interpreter *exposed* as modules[1]. Unless you
have a taste for gory implementation details, just don't worry about this.

Other "ordinary" modules need of course to be executed once when first
loaded - IOW, the first time they are imported. All statements[2] at the
top-level of the module are then sequentially executed, so that any
relevant object (functions, classes, whatever) are created and bound in
the module's namespace.

[1] There's an ambiguity with the term "module", which is used for both
the module object (instance of class 'module') that exists at runtime
and the .py source file a module object is usually - but not necessarily
- built from.

[2] 'def' and 'class' are executable statements that create resp. a
function or class object and bind them to the function/class name in the
containing namespace.
--
http://mail.python.org/mailman/listinfo/python-list

Scott.Daniels at Acm

Aug 20, 2008, 5:48 AM

Post #10 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

Hussein B wrote:
> On Aug 20, 6:39 am, Bruno Desthuilliers <bruno.
> 42.desthuilli...@websiteburo.invalid> wrote:
>> Hussein B a écrit :
>>> One more question:
>>> If I have this structure:
>>> orig/com/domain/project/Klass1.py
>>> Klass2.py
>>> __init__.py
>>> Why com, domain, project should have __init__.py also?
>> Yes, why should they ? Unless you want to mimic Java's package system so
>> you do "import com.domain.project.stuff" - which is IMHO a pretty bad
>> idea -, there's just no reason to turn all your filesystem into a python
>> package.
>>
>> Python's philosophy here is that flat is better than nested.
>
> Sorry I don't follow you :(
> Module package is also nested.

Essentially, "flatter is better than more nested" -- in other words
the things you see after "import this" are principles to trend towards,
not commandments to never be violated. Some structure is better
than no structure, but that doesn't mean more structure is better
ad infinitum. Lists of length 1, modules containing a single class,
dictionaries with a single entry, functions or methods that simply
return a variable or instance are "code smells" for over-exercise of
structuring for its own sake. Each layer should have enough structure
that it is "worth" replacing the more straightforward access with a
name.
Similarly, the name should be associated with few enough things
that you might reasonably expect to hold its scope in your head.
A layer with fifty elements "smells" as bad as one with a single
element, just in a different way.

--Scott David Daniels
Scott.Daniels[at]Acm.Org
--
http://mail.python.org/mailman/listinfo/python-list

lists at cheimes

Aug 20, 2008, 5:51 AM

Post #11 of 11 (92 views)
Permalink

Re: Basic importing question [In reply to]

Bruno Desthuilliers wrote:
> As the name imply, built-in modules are built in the interpreter - IOW,
> they are part of the interpreter *exposed* as modules[1]. Unless you
> have a taste for gory implementation details, just don't worry about this.
>
> Other "ordinary" modules need of course to be executed once when first
> loaded - IOW, the first time they are imported. All statements[2] at the
> top-level of the module are then sequentially executed, so that any
> relevant object (functions, classes, whatever) are created and bound in
> the module's namespace.

Builtin modules like thread, signal etc. as well as C extensions like
socket have an initialization function. The initialization function is
called during the first import of a module. It creates the module object
and assigns objects to its __dict__ attribute.

Builtin modules are statically linked into the Python executable /
library while normal C extensions are shared libraries. Some modules are
builtins because they are required during boot strapping. It's possible
to embed most C modules into the core.

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com