Mailing List Archive: Python: Dev

Excluding the current path from module search path?

Index | Next | Previous | View Threaded

chris at simplistix

Aug 25, 2009, 8:59 AM

Post #1 of 8 (1024 views)
Permalink

Excluding the current path from module search path?

Hi All,

I'm being bitten by this issue:

http://bugs.python.org/issue1734860

I'm not sure I agree with Daniel's closing of it so thought I'd ask here...

Am I right in thinking that the general idea is that "the current
working directory at the time of invoking a script or interpreter ends
up on the python path" or should I be thinking "the directory that a
script exists in should end up on the python path"?

If the latter, then what happens in the case of just starting up an
interpreter?

If neither, then how come when I have two .py files in a directory, I
can import one as a module from the other?

In any case, as a parting comment, http://bugs.python.org/issue1232023
seems to have been committed with no tests and the only documentation
being a one liner in the NEWS.txt file. Was there other discussion of this?

(Incidentally, export PYTHONPATH= or its Windows equivalent circumvents
whatever the patch was trying to achieve, so the change doesn't seem to
make sense anyway...)

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

pje at telecommunity

Aug 25, 2009, 9:30 AM

Post #2 of 8 (974 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

At 04:59 PM 8/25/2009 +0100, Chris Withers wrote:
>Hi All,
>
>I'm being bitten by this issue:
>
>http://bugs.python.org/issue1734860
>
>I'm not sure I agree with Daniel's closing of it so thought I'd ask here...
>
>Am I right in thinking that the general idea is that "the current
>working directory at the time of invoking a script or interpreter
>ends up on the python path" or should I be thinking "the directory
>that a script exists in should end up on the python path"?
>
>If the latter, then what happens in the case of just starting up an
>interpreter?

It's the latter. In the case where there is no script, then the
current directory is considered to be the directory of the script.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

benjamin at python

Aug 25, 2009, 9:31 AM

Post #3 of 8 (985 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

2009/8/25 Chris Withers <chris [at] simplistix>:
> Hi All,
>
> I'm being bitten by this issue:
>
> http://bugs.python.org/issue1734860
>
> I'm not sure I agree with Daniel's closing of it so thought I'd ask here...
>
> Am I right in thinking that the general idea is that "the current working
> directory at the time of invoking a script or interpreter ends up on the
> python path" or should I be thinking "the directory that a script exists in
> should end up on the python path"?

The latter.

>
> If the latter, then what happens in the case of just starting up an
> interpreter?

Because '' is prepended to sys.path then.

--
Regards,
Benjamin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

rdmurray at bitdance

Aug 25, 2009, 9:43 AM

Post #4 of 8 (987 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

On Tue, 25 Aug 2009 at 16:59, Chris Withers wrote:
> In any case, as a parting comment, http://bugs.python.org/issue1232023 seems
> to have been committed with no tests and the only documentation being a one
> liner in the NEWS.txt file. Was there other discussion of this?

It probably should have gone into What's New as well, but it was too
late for that at the time the bug was filed.

> (Incidentally, export PYTHONPATH= or its Windows equivalent circumvents
> whatever the patch was trying to achieve, so the change doesn't seem to make
> sense anyway...)

The change was fixing a clear bug: blank path elements were being
introduced into the path _unintentionally_ and unexpectedly. Setting
PYTHONPATH would be a way to do it intentionally.

--David
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ncoghlan at gmail

Aug 26, 2009, 4:44 AM

Post #5 of 8 (972 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

Chris Withers wrote:
> Hi All,
>
> I'm being bitten by this issue:
>
> http://bugs.python.org/issue1734860
>
> I'm not sure I agree with Daniel's closing of it so thought I'd ask here...
>
> Am I right in thinking that the general idea is that "the current
> working directory at the time of invoking a script or interpreter ends
> up on the python path" or should I be thinking "the directory that a
> script exists in should end up on the python path"?
>
> If the latter, then what happens in the case of just starting up an
> interpreter?
>
> If neither, then how come when I have two .py files in a directory, I
> can import one as a module from the other?

The details of the sys.path manipulation at program startup are
documented here:
http://docs.python.org/using/cmdline.html#command-line

The directory prepended to sys.path is based on the code executed by the
command line.

stdin, -c, -m or nothing specified: current directory
Filesystem path pointing to script (source or compiled): directory
containing script
Filesystem path pointing to directory or zipfile: the named directory or
zipfile

Cheers,
Nick.

--
Nick Coghlan | ncoghlan [at] gmail | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

chris at simplistix

Aug 26, 2009, 4:48 PM

Post #6 of 8 (965 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

Nick Coghlan wrote:
> The details of the sys.path manipulation at program startup are
> documented here:
> http://docs.python.org/using/cmdline.html#command-line
>
> The directory prepended to sys.path is based on the code executed by the
> command line.

It's more subtle than that though...

The OP in http://bugs.python.org/issue1734860 is being bitten by the
same expectation that I am: sitecustomize.py should be found somewhere
on the sys.path present at the start of the script/module/command/etc
being executed. (The bug referenced in that report makes things worse,
because this used to work, at least on Windows ;-) )

The problem is that site.py (and therefore sitecustomize.py) is imported
early in main.c on line 516 as part of Py_Initialize(), but the path of
the current script only gets added later on in RunMainFromImporter
called on line 569.

Strictly speaking, the docs at http://docs.python.org/library/site.html
aren't lying, but it takes an understanding of when site.py is imported
that isn't available to anyone who doesn't read C to know why a path
that is present on sys.path when the user's script starts isn't being
searched for sitecustomize.py

What do people feel about this?

At the very least, I'd like to add a warning box in site.html to explain
why sitecustomize might not be found where people expect.

I'd *like* to have the paths be the same for site.py as they are for the
subsequent code that's executed, but would that make too much of a mess
of main.c and runpython.c?

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ncoghlan at gmail

Aug 27, 2009, 2:37 AM

Post #7 of 8 (961 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

Chris Withers wrote:
> Nick Coghlan wrote:
>> The details of the sys.path manipulation at program startup are
>> documented here:
>> http://docs.python.org/using/cmdline.html#command-line
>>
>> The directory prepended to sys.path is based on the code executed by the
>> command line.
>
> It's more subtle than that though...
>
> The OP in http://bugs.python.org/issue1734860 is being bitten by the
> same expectation that I am: sitecustomize.py should be found somewhere
> on the sys.path present at the start of the script/module/command/etc
> being executed. (The bug referenced in that report makes things worse,
> because this used to work, at least on Windows ;-) )
>
> The problem is that site.py (and therefore sitecustomize.py) is imported
> early in main.c on line 516 as part of Py_Initialize(), but the path of
> the current script only gets added later on in RunMainFromImporter
> called on line 569.
>
> Strictly speaking, the docs at http://docs.python.org/library/site.html
> aren't lying, but it takes an understanding of when site.py is imported
> that isn't available to anyone who doesn't read C to know why a path
> that is present on sys.path when the user's script starts isn't being
> searched for sitecustomize.py
>
> What do people feel about this?
>
> At the very least, I'd like to add a warning box in site.html to explain
> why sitecustomize might not be found where people expect.
>
> I'd *like* to have the paths be the same for site.py as they are for the
> subsequent code that's executed, but would that make too much of a mess
> of main.c and runpython.c?

Ah, OK - I see the problem now. However, I think the current behaviour
is correct, it just needs to be documented better (probably noted in
both the command line doco regarding sys.path manipulation and in the
doco for site.py).

The reason I think the current behaviour is correct is that site.py and
sitecustomize.py are meant to be about customising the *site* (i.e. the
installation of Python that is being executed) rather than about
customizing a particular application. Importing them before the script
specific directories are prepended to sys.path goes a long way towards
achieving that.

Also, as was pointed out on the tracker item, having a script that can
automatically be executed when running an arbitrary Python script
without any request from or notification to the user is not a good idea
from a security standpoint.

When it comes to adding additional paths for specific applications, you
can either bundle the relevant packages into a single directory and use
2.6's directory execution feature or else look into the assorted
application environment customisation tools that are out there like
virtualenv.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan [at] gmail | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

chris at simplistix

Aug 31, 2009, 5:17 AM

Post #8 of 8 (911 views)
Permalink

Re: Excluding the current path from module search path? [In reply to]

Nick Coghlan wrote:
> Ah, OK - I see the problem now. However, I think the current behaviour
> is correct, it just needs to be documented better (probably noted in
> both the command line doco

Not sure what you mean by this?

> regarding sys.path manipulation and in the
> doco for site.py).

Agreed :-)

> The reason I think the current behaviour is correct is that site.py and
> sitecustomize.py are meant to be about customising the *site* (i.e. the
> installation of Python that is being executed) rather than about
> customizing a particular application.

Unless you use virtualenv as Guido suggested in the other thread ;-)

> Importing them before the script
> specific directories are prepended to sys.path goes a long way towards
> achieving that.

If sitecustomize.py had more uses that the setdefaultencoding hack, I'd
argue more about this... If it does have other uses, my argument would
be that "site" wide is a very subjective term tht many people, myself
included, would like to be able to mean "per project, I don't *ever*
want to screw with my actual Python install, it should stay pristine"...

> Also, as was pointed out on the tracker item, having a script that can
> automatically be executed when running an arbitrary Python script
> without any request from or notification to the user is not a good idea
> from a security standpoint.

Agreed, but I think that's only an issue when you're starting up an
interpreter. If you're running a script from a file or module, I'd say
it's more akin to what's specified in PYTHONSTARTUP being executed that
than a random script being silently executed without your permission.

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads