Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

PEP 405 (pyvenv) and system Python upgrades

  

 
Python dev RSS feed   Index | Next | Previous | View Threaded


carl at oddbird

May 4, 2012, 1:49 PM

Post #1 of 24 (354 views)
Permalink
PEP 405 (pyvenv) and system Python upgrades
Hi all,

The recent virtualenv breakage in Python 2.6.8 and 2.7.3 reveals an
issue that deserves to be explicitly addressed in PEP 405: what happens
when the system Python underlying a venv gets an in-place bugfix
upgrade. If the bugfix includes a simultaneous change to the interpreter
and standard library such that the older interpreter will not work with
the newer standard library, all venvs created from that Python
installation will be broken until the new interpreter is copied into them.

Choices for how to address this:

1) Document it and provide a tool for easily upgrading a venv in this
situation. This may be adequate. In practice the situation is quite
rare: 2.6.8/2.7.3 is the only actual example in the history of
virtualenv that I'm aware of. The disadvantage is that if the problem
does occur, the error will probably be quite confusing and seemingly
unrelated to pyvenv.

2) In addition to the above, introduce a versioning marker in the
standard library (is there one already?) and have some code somewhere
(insert hand-waving here) check sys.version_info against the stdlib
version, and fail fast with an unambiguous error if there is a mismatch.
This makes the failure more explicit, but at the significant cost of
making it more common: at every mismatch, not just in the
apparently-rare case of a breaking change.

3) Symlink the interpreter rather than copying. I include this here for
the sake of completeness, but it's already been rejected due to
significant problems on older Windows' and OS X.

4) Adopt a policy of interpreter/stdlib cross-compatibility within a
given X.Y version of Python. I don't expect this to be a popular choice,
given the additional testing requirements it imposes, but it would
certainly be the nicest option from the PEP 405 standpoint (and may also
be complementary to proposals for splitting out the stdlib). In the
2.6.8/2.7.3 case, this would have been technically trivial to do, but
the choice was made not to do it in order to force virtualenv users to
adopt the security-fixed Python interpreter.

Thoughts?

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


tjreedy at udel

May 4, 2012, 8:48 PM

Post #2 of 24 (331 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 5/4/2012 4:49 PM, Carl Meyer wrote:
> Hi all,
>
> The recent virtualenv breakage in Python 2.6.8 and 2.7.3 reveals an
> issue that deserves to be explicitly addressed in PEP 405: what happens
> when the system Python underlying a venv gets an in-place bugfix
> upgrade. If the bugfix includes a simultaneous change to the interpreter
> and standard library such that the older interpreter will not work with
> the newer standard library, all venvs created from that Python
> installation will be broken until the new interpreter is copied into them.

CPython is developed, tested, packaged, distributed, and installed as
one unit. It is intended to be run as one package. If something caches a
copy of python.exe, it seems to me that it should check and update as
needed. Could venv check the file date of the current python.exe versus
that of the one cached, much like is done with .pyc compiled code caches?

> Choices for how to address this:

> 1) Document it and provide a tool for easily upgrading a venv in this
> situation.

Right.

> 4) Adopt a policy of interpreter/stdlib cross-compatibility within a
> given X.Y version of Python. I don't expect this to be a popular choice,

What a droll sense of humor ;=).

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


v+python at g

May 4, 2012, 8:58 PM

Post #3 of 24 (330 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 5/4/2012 8:48 PM, Terry Reedy wrote:
> CPython is developed, tested, packaged, distributed, and installed as
> one unit. It is intended to be run as one package. If something caches
> a copy of python.exe, it seems to me that it should check and update
> as needed. Could venv check the file date of the current python.exe
> versus that of the one cached, much like is done with .pyc compiled
> code caches?

I almost wrote this response (using different words, but the same idea)
but concluded that:

1) Python wouldn't run far without its standard library, so a venv check
would have to be very early, and likely coded in C, and therefore
probably has to be part of Python.exe

2) If it was not part of Python.exe, it would have to work similarly to
the launcher, and there would be yet one more process sitting around
waiting for Python to exit (on Windows, where there is no exec).

So I concluded that probably Python.exe needs to make the check, but if
it is aware it existing in venv, it might be able to put out a better
message than "just" the mismatch between exe and lib; or at least the
message should mention the possibility of an old venv cache.

Glenn


tjreedy at udel

May 4, 2012, 9:39 PM

Post #4 of 24 (331 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 5/4/2012 11:58 PM, Glenn Linderman wrote:
> On 5/4/2012 8:48 PM, Terry Reedy wrote:
>> CPython is developed, tested, packaged, distributed, and installed as
>> one unit. It is intended to be run as one package. If something caches
>> a copy of python.exe, it seems to me that it should check and update
>> as needed. Could venv check the file date of the current python.exe
>> versus that of the one cached, much like is done with .pyc compiled
>> code caches?
>
> I almost wrote this response (using different words, but the same idea)
> but concluded that:
>
> 1) Python wouldn't run far without its standard library, so a venv check
> would have to be very early, and likely coded in C, and therefore
> probably has to be part of Python.exe
>
> 2) If it was not part of Python.exe, it would have to work similarly to
> the launcher, and there would be yet one more process sitting around
> waiting for Python to exit (on Windows, where there is no exec).
>
> So I concluded that probably Python.exe needs to make the check, but if
> it is aware it existing in venv, it might be able to put out a better
> message than "just" the mismatch between exe and lib; or at least the
> message should mention the possibility of an old venv cache.

The gist of my response is that the venv 'tail' should way the python
'dog' as little as possbile.

I also wonder how often such incompatibility occurs. Optionally changing
the de facto semantics of CPython's built-in dict in bug-fix releases
was a rather unique event. I am sure we would all be happy to never have
to do such again.

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


v+python at g

May 4, 2012, 10:58 PM

Post #5 of 24 (326 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 5/4/2012 9:39 PM, Terry Reedy wrote:
> The gist of my response is that the venv 'tail' should way the python
> 'dog' as little as possbile.

Yes, that was exactly my thought too. But I'm not sure the technology
permits, with Windows not having exec. On the other hand, one might
speculate about how venv, instead of copying Python.exe, might instead
install the launcher in the place where python.exe is currently copied.
The launcher does the "next best thing to exec". Plus it would save a
wee bit of space, being smaller than python.exe. On platforms that have
symlinks, they could be used instead.


ncoghlan at gmail

May 4, 2012, 11:41 PM

Post #6 of 24 (327 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On Sat, May 5, 2012 at 6:49 AM, Carl Meyer <carl [at] oddbird> wrote:
> 1) Document it and provide a tool for easily upgrading a venv in this
> situation. This may be adequate. In practice the situation is quite rare:
> 2.6.8/2.7.3 is the only actual example in the history of virtualenv that I'm
> aware of. The disadvantage is that if the problem does occur, the error will
> probably be quite confusing and seemingly unrelated to pyvenv.

I think this is the way to go, for basically the same reasons that we
did it this way this time: there's no good reason to pay an ongoing
cost to further mitigate the risks associated with an already
incredibly rare event.

It would become part of the standard venv debugging toolkit:

Q X.1: Does the problem only occur inside a particular virtual environment?
Q X.2: If yes, did you recently upgrade the system Python to a new
point release?
Q X.3: If yes, did you run <command that updates the Python binary
copy in the virtual environment>?
Q X.4: If no, do so and see if the problem goes away. Even if it still
doesn't work, at least you've eliminated this particular error as a
possible cause.

Personally, I expect that "always update your virtual environment
binaries after updating the system Python to a new point release" will
itself become a recommended practice when using virtual environments.

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan [at] gmail   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


vinay_sajip at yahoo

May 5, 2012, 1:38 AM

Post #7 of 24 (328 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Personally, I expect that "always update your virtual environment
> binaries after updating the system Python to a new point release" will
> itself become a recommended practice when using virtual environments.

Of course, the venv update tool will need to only update environments which were
set up with the particular version of Python which was updated. ISTM pyvenv.cfg
will need to have a version=X.Y.Z line in it, which is added during venv
creation. That information will be used by the tool to only update specific
environments.

Regards,

Vinay Sajip



_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


rosuav at gmail

May 5, 2012, 1:52 AM

Post #8 of 24 (324 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On Sat, May 5, 2012 at 6:49 AM, Carl Meyer <carl [at] oddbird> wrote:
> 2) In addition to the above, introduce a versioning marker in the standard
> library (is there one already?) and have some code somewhere (insert
> hand-waving here) check sys.version_info against the stdlib version, and
> fail fast with an unambiguous error if there is a mismatch. This makes the
> failure more explicit, but at the significant cost of making it more common:
> at every mismatch, not just in the apparently-rare case of a breaking
> change.

Variant: Could the versioning marker give a minimum and/or maximum?
It'd then only cause the explicit failure in the actual case of a
breaking change, and the rest of the time it could happily use any
X.Y.* release.

ChrisA
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


solipsis at pitrou

May 5, 2012, 3:40 AM

Post #9 of 24 (326 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Hi,

On Fri, 04 May 2012 14:49:03 -0600
Carl Meyer <carl [at] oddbird> wrote:
>
> 3) Symlink the interpreter rather than copying. I include this here for
> the sake of completeness, but it's already been rejected due to
> significant problems on older Windows' and OS X.

Perhaps symlinking could be used at least on symlinks-friendly OSes?
I expect older Windows to disappear one day :-) So the only left
outlier would be OS X.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


carl at oddbird

May 6, 2012, 2:56 PM

Post #10 of 24 (312 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 05/05/2012 02:38 AM, Vinay Sajip wrote:
> Nick Coghlan<ncoghlan<at> gmail.com> writes:
>
>> Personally, I expect that "always update your virtual environment
>> binaries after updating the system Python to a new point release" will
>> itself become a recommended practice when using virtual environments.
>
> Of course, the venv update tool will need to only update environments which were
> set up with the particular version of Python which was updated. ISTM pyvenv.cfg
> will need to have a version=X.Y.Z line in it, which is added during venv
> creation. That information will be used by the tool to only update specific
> environments.

I don't think the added "version" key in pyvenv.cfg is needed; the
"home" key provides enough information to know whether the virtualenv
was created by the particular Python that was upgraded.

The "version" key could in theory be useful to know whether a particular
venv created by that Python has or has not yet been upgraded to match,
but since the upgrade is trivial and idempotent I don't think that is
important.

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


carl at oddbird

May 6, 2012, 3:07 PM

Post #11 of 24 (309 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 05/05/2012 04:40 AM, Antoine Pitrou wrote:
> On Fri, 04 May 2012 14:49:03 -0600
> Carl Meyer<carl [at] oddbird> wrote:
>> 3) Symlink the interpreter rather than copying. I include this here for
>> the sake of completeness, but it's already been rejected due to
>> significant problems on older Windows' and OS X.
>
> Perhaps symlinking could be used at least on symlinks-friendly OSes?
> I expect older Windows to disappear one day :-) So the only left
> outlier would be OS X.

It certainly could - at one point the reference implementation did
exactly this. I understand though that even on newer Windows' there are
administrator-privilege issues with symlinks, and I don't know that
there's any prospect of the OS X stub executable going away, so I think
if we did this we should assume that we're accepting a more-or-less
permanent cross-platform difference in the default behavior of venvs.
Maybe that's ok; it would mean that for Linux users there'd be no need
to run any venv-upgrade script at all when Python is updated, which is
certainly a plus.

At one point it was argued that we shouldn't symlink by default because
users expect venvs to be isolated and not upgraded implicitly. I think
this discussion reveals that that's a false argument, since the stdlib
will be upgraded implicitly regardless, and that's just as likely to
break something as an interpreter update (and more likely than upgrading
them in sync). IOW, if you want real full isolation from a system
Python, you build your own Python, you don't use pyvenv.

Carl

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


carl at oddbird

May 6, 2012, 3:08 PM

Post #12 of 24 (310 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 05/05/2012 12:41 AM, Nick Coghlan wrote:
> On Sat, May 5, 2012 at 6:49 AM, Carl Meyer<carl [at] oddbird> wrote:
>> 1) Document it and provide a tool for easily upgrading a venv in this
>> situation. This may be adequate. In practice the situation is quite rare:
>> 2.6.8/2.7.3 is the only actual example in the history of virtualenv that I'm
>> aware of. The disadvantage is that if the problem does occur, the error will
>> probably be quite confusing and seemingly unrelated to pyvenv.
>
> I think this is the way to go, for basically the same reasons that we
> did it this way this time: there's no good reason to pay an ongoing
> cost to further mitigate the risks associated with an already
> incredibly rare event.

This seems to be the rough consensus. I'll update the PEP with a note
about this, and we'll consider switching back to symlink-by-default on
Linux.

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


vinay_sajip at yahoo

May 6, 2012, 5:58 PM

Post #13 of 24 (308 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Carl Meyer <carl <at> oddbird.net> writes:

> them in sync). IOW, if you want real full isolation from a system
> Python, you build your own Python, you don't use pyvenv.

For the interpreter you can use your own Python, but you would still use pyvenv,
as the venv is still useful for you to have an isolated set of library
dependencies for a project.

Regards,

Vinay Sajip



_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

May 7, 2012, 2:52 AM

Post #14 of 24 (296 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
> 3) Symlink the interpreter rather than copying. I include this here for
> the sake of completeness, but it's already been rejected due to
> significant problems on older Windows' and OS X.

That sounds the right solution to me. PEP 405 specifies that bin/python3
exists, but not that it is the actual Python interpreter binary that is
normally used. For each target system, a solution should be defined that
allows in-place updates of Python that also update all venvs automatically.

For example, for Windows, it would be sufficient to just have the
executable in bin/, as the update will only affect pythonXY.dll.
That executable may be different from the regular python.exe, and
it might be necessary that it locates its Python installation first.
For Unix, symlinks sound fine. Not sure what the issue with OS X is.

Regards,
Martin

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ronaldoussoren at mac

May 7, 2012, 3:26 AM

Post #15 of 24 (297 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 7 May, 2012, at 11:52, Martin v. Löwis wrote:

>> 3) Symlink the interpreter rather than copying. I include this here for
>> the sake of completeness, but it's already been rejected due to
>> significant problems on older Windows' and OS X.
>
> That sounds the right solution to me. PEP 405 specifies that bin/python3
> exists, but not that it is the actual Python interpreter binary that is
> normally used. For each target system, a solution should be defined that
> allows in-place updates of Python that also update all venvs automatically.
>
> For example, for Windows, it would be sufficient to just have the executable in bin/, as the update will only affect pythonXY.dll.
> That executable may be different from the regular python.exe, and
> it might be necessary that it locates its Python installation first.
> For Unix, symlinks sound fine. Not sure what the issue with OS X is.

The bin/python3 executable in a framework is a small stub that execv's the real interpreter that is stuffed in a Python.app bundle inside the Python framework. That's done to ensure that GUI code can work from the command-line, Apple's GUI framework refuse to work when the executable is not in an application bundle.

Because of this trick pyvenv won't know which executable the user actually called and hence cannot find the pyvenv configuration file (which is next to the stub executable).

Ronald
Attachments: smime.p7s (4.68 KB)


carl at oddbird

May 7, 2012, 8:25 AM

Post #16 of 24 (296 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 05/07/2012 04:26 AM, Ronald Oussoren wrote:
> On 7 May, 2012, at 11:52, Martin v. Löwis wrote:
>>> 3) Symlink the interpreter rather than copying. I include this
>>> here for the sake of completeness, but it's already been rejected
>>> due to significant problems on older Windows' and OS X.
>>
>> That sounds the right solution to me. PEP 405 specifies that
>> bin/python3 exists, but not that it is the actual Python
>> interpreter binary that is normally used. For each target system, a
>> solution should be defined that allows in-place updates of Python
>> that also update all venvs automatically.
>>
>> For example, for Windows, it would be sufficient to just have the
>> executable in bin/, as the update will only affect pythonXY.dll.
>> That executable may be different from the regular python.exe, and
>> it might be necessary that it locates its Python installation
>> first. For Unix, symlinks sound fine. Not sure what the issue with
>> OS X is.
>
> The bin/python3 executable in a framework is a small stub that
> execv's the real interpreter that is stuffed in a Python.app bundle
> inside the Python framework. That's done to ensure that GUI code can
> work from the command-line, Apple's GUI framework refuse to work when
> the executable is not in an application bundle.
>
> Because of this trick pyvenv won't know which executable the user
> actually called and hence cannot find the pyvenv configuration file
> (which is next to the stub executable).

It occurs to me, belatedly, that this also means that upgrades should be
a non-issue with OS X framework builds (presuming the upgraded
actual-Python-binary gets placed in the same location, and the
previously copied stub will still exec it without trouble), in which
case we can symlink on OS X non-framework builds and copy on OS X
framework builds and be happy.

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


carl at oddbird

May 7, 2012, 9:35 AM

Post #17 of 24 (294 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 05/07/2012 03:52 AM, "Martin v. Löwis" wrote:
>> 3) Symlink the interpreter rather than copying. I include this here for
>> the sake of completeness, but it's already been rejected due to
>> significant problems on older Windows' and OS X.
>
> That sounds the right solution to me. PEP 405 specifies that bin/python3
> exists, but not that it is the actual Python interpreter binary that is
> normally used. For each target system, a solution should be defined that
> allows in-place updates of Python that also update all venvs automatically.

I propose that for Windows, that solution is to have a new enough
version of Windows and the necessary privileges, and use the --symlink
option to the pyvenv script, or else to manually update venvs using
pyvenv --upgrade.

> For example, for Windows, it would be sufficient to just have the
> executable in bin/, as the update will only affect pythonXY.dll.
> That executable may be different from the regular python.exe, and
> it might be necessary that it locates its Python installation first.

This sounds to me like a level of complexity unwarranted by the severity
of the problem, especially when considering the additional burden it
imposes on alternative Python implementations.

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

May 7, 2012, 1:51 PM

Post #18 of 24 (294 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
> The bin/python3 executable in a framework is a small stub that
> execv's the real interpreter that is stuffed in a Python.app bundle
> inside the Python framework. That's done to ensure that GUI code can
> work from the command-line, Apple's GUI framework refuse to work when
> the executable is not in an application bundle.
>
> Because of this trick pyvenv won't know which executable the user
> actually called and hence cannot find the pyvenv configuration file
> (which is next to the stub executable).

I don't understand. The "executable that the user actually called":
does that refer to

a) the stub (which the user *actually* called) or
b) the eventual binary (which is what gets *actually* run).

If a), then I think argv[0] just needs to continue to refer to the
stub, which is easy to achieve in execv.

If b), I wonder why the code needs to know the location to the binary
inside the bundle. But if this is needed to know, I suggest that some
environment variable is passed from the stub to the actual binary
(akin PYTHONHOME). How does the stub normally find out where the
framework is located?

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


martin at v

May 7, 2012, 1:55 PM

Post #19 of 24 (296 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 07.05.2012 18:35, Carl Meyer wrote:
> On 05/07/2012 03:52 AM, "Martin v. Löwis" wrote:
>>> 3) Symlink the interpreter rather than copying. I include this here for
>>> the sake of completeness, but it's already been rejected due to
>>> significant problems on older Windows' and OS X.
>>
>> That sounds the right solution to me. PEP 405 specifies that bin/python3
>> exists, but not that it is the actual Python interpreter binary that is
>> normally used. For each target system, a solution should be defined that
>> allows in-place updates of Python that also update all venvs
>> automatically.
>
> I propose that for Windows, that solution is to have a new enough
> version of Windows and the necessary privileges, and use the --symlink
> option to the pyvenv script, or else to manually update venvs using
> pyvenv --upgrade.

Sounds fine to me as well.

>> For example, for Windows, it would be sufficient to just have the
>> executable in bin/, as the update will only affect pythonXY.dll.
>> That executable may be different from the regular python.exe, and
>> it might be necessary that it locates its Python installation first.
>
> This sounds to me like a level of complexity unwarranted by the severity
> of the problem, especially when considering the additional burden it
> imposes on alternative Python implementations.

OTOH, it *significantly* reduces the burden on Python end users, for
whom creation of a venv under a privileged account is a significant
hassle.

This being free software, anybody needs to scratch her own itches, of
course.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


vinay_sajip at yahoo

May 7, 2012, 2:25 PM

Post #20 of 24 (295 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Ronald Oussoren <ronaldoussoren <at> mac.com> writes:

> Because of this trick pyvenv won't know which executable the user actually
> called and hence cannot find the pyvenv configuration file (which is next to
> the stub executable).

Ah, but the stub has been changed to set an environment variable,
__PYTHONV_LAUNCHER__, which points to itself, before it execs the real Python.
On OS X, Python code checks for this, rather than sys.executable, to determine
the location of the pyvenv.cfg file. This seems to work for me (Ned Deily is
looking into it more closely, I believe).

Regards,

Vinay Sajip

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


p.f.moore at gmail

May 7, 2012, 3:16 PM

Post #21 of 24 (290 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 7 May 2012 21:55, "Martin v. Löwis" <martin [at] v> wrote:
>> This sounds to me like a level of complexity unwarranted by the severity
>> of the problem, especially when considering the additional burden it
>> imposes on alternative Python implementations.
>
>
> OTOH, it *significantly* reduces the burden on Python end users, for
> whom creation of a venv under a privileged account is a significant
> hassle.

Personally, I would find a venv which required being run as an admin
account to be essentially unusable on Windows (particularly Windows 7,
where this means creating venvs in an "elevated" console window).

Allowing for symlinks as an option is fine, I guess, but I'd be -1 on
it being the default.

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


vinay_sajip at yahoo

May 8, 2012, 3:50 AM

Post #22 of 24 (282 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Carl Meyer <carl <at> oddbird.net> writes:

> The "version" key could in theory be useful to know whether a particular
> venv created by that Python has or has not yet been upgraded to match,
> but since the upgrade is trivial and idempotent I don't think that is
> important.

Agreed it's not essential, but it also provides some useful information about
the version (for a user, rather than the update script) without actually having
to invoke the interpreter to check.

Regards,

Vinay Sajip

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


carl at oddbird

May 8, 2012, 9:14 AM

Post #23 of 24 (278 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
Hi Paul,

On 05/07/2012 04:16 PM, Paul Moore wrote:
> On 7 May 2012 21:55, "Martin v. Löwis"<martin [at] v> wrote:
>>> This sounds to me like a level of complexity unwarranted by the severity
>>> of the problem, especially when considering the additional burden it
>>> imposes on alternative Python implementations.
>>
>>
>> OTOH, it *significantly* reduces the burden on Python end users, for
>> whom creation of a venv under a privileged account is a significant
>> hassle.
>
> Personally, I would find a venv which required being run as an admin
> account to be essentially unusable on Windows (particularly Windows 7,
> where this means creating venvs in an "elevated" console window).
>
> Allowing for symlinks as an option is fine, I guess, but I'd be -1 on
> it being the default.

I don't think anyone has proposed making symlinks the default on
Windows. At this point the two options on Windows would be to use the
--symlink option explicitly, or else to need to run "pyvenv --upgrade"
on your envs if you upgrade the underlying Python in-place (and there's
a breaking incompatibility between the new stdlib and the old
interpreter, which there almost never will be if the past is any
indication).

I expect most users will opt for the latter option (equivalent to how
current virtualenv works, except virtualenv doesn't have an --upgrade
flag so you have to upgrade manually), but the former is also available
if some prefer it.

In any case, the situation will be no worse than it is with virtualenv
today.

Carl
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


p.f.moore at gmail

May 8, 2012, 10:34 AM

Post #24 of 24 (278 views)
Permalink
Re: PEP 405 (pyvenv) and system Python upgrades [In reply to]
On 8 May 2012 17:14, Carl Meyer <carl [at] oddbird> wrote:
> I don't think anyone has proposed making symlinks the default on Windows. At
> this point the two options on Windows would be to use the --symlink option
> explicitly, or else to need to run "pyvenv --upgrade" on your envs if you
> upgrade the underlying Python in-place (and there's a breaking
> incompatibility between the new stdlib and the old interpreter, which there
> almost never will be if the past is any indication).
>
> I expect most users will opt for the latter option (equivalent to how
> current virtualenv works, except virtualenv doesn't have an --upgrade flag
> so you have to upgrade manually), but the former is also available if some
> prefer it.
>
> In any case, the situation will be no worse than it is with virtualenv
> today.

That sounds fine. I apologise - I'd got the impression that the
proposal was to make symlinks the default.

Paul.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.