Mailing List Archive: Python: Dev

buildtime vs runtime in Distutils

1 2

View All

Index | Next | Previous | View Threaded

ziade.tarek at gmail

Nov 14, 2009, 5:21 PM

Post #1 of 28 (1568 views)
Permalink

buildtime vs runtime in Distutils

Hello,

http://bugs.python.org/issue4359 reminds me that Distutils reads build
files like Makefile or pyconfig.h to get some environment
variables through the sysconfig module at *runtime*.

This cannot work on all platforms, when our Makefile is not shipped
with python but python-devel. (like Fedora)

Since I am already working on the refactoring of this
distutils/sysconfig module, so it lands in a stdlib module together
with some
elements from site.py (see a previous mail on the topic, and my
tarek_sysconfig branch - work in progress),

I was wondering if we couldn't make this module a template, and inject
at its beginning new values extracted from Makefile and pyconfig.h.

This could happen when "configure" is called,

Regards,
Tarek

--
Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

martin at v

Nov 14, 2009, 5:38 PM

Post #2 of 28 (1515 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
>
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

I don't see a problem with that: you'll need the python-devel package
*anyway* when running distutils, for many packages.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

arfrever.fta at gmail

Nov 14, 2009, 5:41 PM

Post #3 of 28 (1516 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

2009-11-15 02:21:41 Tarek Ziadé napisał(a):
> Hello,
>
> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
>
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)
>
> Since I am already working on the refactoring of this
> distutils/sysconfig module, so it lands in a stdlib module together
> with some
> elements from site.py (see a previous mail on the topic, and my
> tarek_sysconfig branch - work in progress),
>
> I was wondering if we couldn't make this module a template, and inject
> at its beginning new values extracted from Makefile and pyconfig.h.
>
> This could happen when "configure" is called,

It seems to be a good idea. You should create a .py.in file and use
AC_SUBST and AC_CONFIG_FILES macros in configure.in.

--
Arfrever Frehtes Taifersar Arahesis

Attachments:

signature.asc (0.82 KB)

ziade.tarek at gmail

Nov 14, 2009, 5:54 PM

Post #4 of 28 (1516 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

2009/11/15 "Martin v. L�wis" <martin [at] v>:
>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>> files like Makefile or pyconfig.h to get some environment
>> variables through the sysconfig module at *runtime*.
>>
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
>
> I don't see a problem with that: you'll need the python-devel package
> *anyway* when running distutils, for many packages.

The problem is that the main python distribution ("python") is not
working as advertised since
it contains distutils, which requires "python-devel" to work.

This implies that "python" has a dependency on "python-devel", which
does not make sense
anymore for linux distros to have two distinct packages for Python.

Having some of the makefile vars stored in stdlib solve this problem.

Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 14, 2009, 5:56 PM

Post #5 of 28 (1516 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 2:41 AM, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta [at] gmail> wrote:
[..]
>>
>> This could happen when "configure" is called,
>
> It seems to be a good idea. You should create a .py.in file and use
> AC_SUBST and AC_CONFIG_FILES macros in configure.in.

I need to investigate on these, thx

Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

martin at v

Nov 14, 2009, 5:57 PM

Post #6 of 28 (1514 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

> The problem is that the main python distribution ("python") is not
> working as advertised since
> it contains distutils, which requires "python-devel" to work.
>
> This implies that "python" has a dependency on "python-devel", which
> does not make sense
> anymore for linux distros to have two distinct packages for Python.
>
> Having some of the makefile vars stored in stdlib solve this problem.

I don't see how this can solve the problem. Distutils contains the
build_ext command, which definitely will require the devel package.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 14, 2009, 6:13 PM

Post #7 of 28 (1515 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

2009/11/15 "Martin v. L�wis" <martin [at] v>:
>> The problem is that the main python distribution ("python") is not
>> working as advertised since
>> it contains distutils, which requires "python-devel" to work.
>>
>> This implies that "python" has a dependency on "python-devel", which
>> does not make sense
>> anymore for linux distros to have two distinct packages for Python.
>>
>> Having some of the makefile vars stored in stdlib solve this problem.
>
> I don't see how this can solve the problem. Distutils contains the
> build_ext command, which definitely will require the devel package.

Sure that's the edge case where files like Python.h is required for
compiling C extensions.
And compiler errors can make it pretty clear at this level, that the
devel package is required.
Not because of the need to read a Makefile, but to get the header
files to compile.

And it should be possible to compile extensions that don't require
python-devel at all,
when these extensions are not CPython extensions.

Last, commands like "install" also requires using sysconfig, (thus
reading Makefile) to get some values.
I can change this for "install" it's not hard, but we would still
provide some public APIs that are reading Makefile from within the
stlib.

right now, importing "disutils.sysconfig" triggers this behavior.

That makes several use case of requiring python-devel just to read a
few values that could be injected
in the stdlib at build time.

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

Nov 14, 2009, 6:35 PM

Post #8 of 28 (1505 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Tarek Ziadé wrote:
> Hello,
>
> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
>
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

Do we really want to change distutils to solve a problem of a third
party packaging system when the problem was created by the very same
third party in the first place? In other words: Should you spend your
precious development time with fixing a problem that isn't our fault?
The decision to split the header files into a separate package, that
isn't installed by default, has already created tons of bad user
experience in the past. Do you want to endorse the split even further?

-0 from me

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

tjreedy at udel

Nov 14, 2009, 7:02 PM

Post #9 of 28 (1503 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Christian Heimes wrote:
> Tarek Ziadé wrote:
>> Hello,
>>
>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>> files like Makefile or pyconfig.h to get some environment
>> variables through the sysconfig module at *runtime*.
>>
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
>
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?
>
> -0 from me

If the third party disables distutils by removing some of the things it
(sometimes) needs, then it seems to me that they should also remove
disutils into the separate package. If *that* creates a problem, that
should be *their* problem.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Nov 15, 2009, 4:42 AM

Post #10 of 28 (1493 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
>
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

This practice is stupid anyway, because it means you have to install
python-devel even to install pure Python packages with setuptools/distribute.
Just ask Fedora, Mandriva and friends to change their packaging practice
(Mandriva already has a bug open for that by the way).

In Debian/Ubuntu, the Makefile is correctly part of the main Python package:

$ dpkg -S /usr/lib/python2.5/config/Makefile
python2.5: /usr/lib/python2.5/config/Makefile

Regards

Antoine.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 15, 2009, 5:30 AM

Post #11 of 28 (1491 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists [at] cheimes> wrote:
[..]
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?

I didn't know the split story went like this. I took it like the
"natural" split everyone
agreed on, and I saw this distutils <-> Makefile link like something to fix.

So, it sounds like a bad idea now :)

Tarek.
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

g.brandl at gmx

Nov 15, 2009, 5:31 AM

Post #12 of 28 (1493 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Antoine Pitrou schrieb:
> Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
>>
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
>
> This practice is stupid anyway, because it means you have to install
> python-devel even to install pure Python packages with setuptools/distribute.
> Just ask Fedora, Mandriva and friends to change their packaging practice
> (Mandriva already has a bug open for that by the way).

+1. They are the ones splitting what "make install" installs into several
packages, so they are the ones who have to fix the resulting dependency
problems.

Georg

--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 15, 2009, 5:32 AM

Post #13 of 28 (1492 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 4:02 AM, Terry Reedy <tjreedy [at] udel> wrote:
> Christian Heimes wrote:
>>
>> Tarek Ziad� wrote:
>>>
>>> Hello,
>>>
>>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>>> files like Makefile or pyconfig.h to get some environment
>>> variables through the sysconfig module at *runtime*.
>>>
>>> This cannot work on all platforms, when our Makefile is not shipped
>>> with python but python-devel. (like Fedora)
>>
>> Do we really want to change distutils to solve a problem of a third
>> party packaging system when the problem was created by the very same
>> third party in the first place? In other words: Should you spend your
>> precious development time with fixing a problem that isn't our fault?
>> The decision to split the header files into a separate package, that
>> isn't installed by default, has already created tons of bad user
>> experience in the past. Do you want to endorse the split even further?
>>
>> -0 from me
>
> If the third party disables distutils by removing some of the things it
> (sometimes) needs, then it seems to me that they should also remove disutils
> into the separate package. If *that* creates a problem, that should be
> *their* problem.

Ok. Fair enough, I'll work with them this way.

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

cournape at gmail

Nov 15, 2009, 5:38 AM

Post #14 of 28 (1491 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 10:32 PM, Tarek Ziadé <ziade.tarek [at] gmail> wrote:

>
> Ok. Fair enough, I'll work with them this way.

Although packagers should certainly fix the problems they introduce in
the first place, the second suggestion in the bug report would be
useful, independently on how linux distributions package things.

Especially if the data can be obtained for every build (autoconf and
VS-based), this would help packages which use something else than
distutils for their build.

David
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

brett at python

Nov 15, 2009, 12:51 PM

Post #15 of 28 (1478 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 05:31, Georg Brandl <g.brandl [at] gmx> wrote:
> Antoine Pitrou schrieb:
>> Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
>>>
>>> This cannot work on all platforms, when our Makefile is not shipped
>>> with python but python-devel. (like Fedora)
>>
>> This practice is stupid anyway, because it means you have to install
>> python-devel even to install pure Python packages with setuptools/distribute.
>> Just ask Fedora, Mandriva and friends to change their packaging practice
>> (Mandriva already has a bug open for that by the way).
>
> +1. They are the ones splitting what "make install" installs into several
> packages, so they are the ones who have to fix the resulting dependency
> problems.

+1 from me as well. Python is designed to run as a whole. If they
choose to ignore our design decisions they do so at their own peril.

Now if you want to put the time in, Tarek, to make sure distutils can
be removed from the stdlib and have everything still work so the linux
distros can add distutils to python-devel that's fine, but once again,
they are the ones mucking around with things in a way we did not
design for, so only do it if you really want to.

-Brett
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

tseaver at palladion

Nov 15, 2009, 8:27 PM

Post #16 of 28 (1455 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad� wrote:
> On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists [at] cheimes> wrote:
> [..]
>> Do we really want to change distutils to solve a problem of a third
>> party packaging system when the problem was created by the very same
>> third party in the first place? In other words: Should you spend your
>> precious development time with fixing a problem that isn't our fault?
>> The decision to split the header files into a separate package, that
>> isn't installed by default, has already created tons of bad user
>> experience in the past. Do you want to endorse the split even further?
>
> I didn't know the split story went like this. I took it like the
> "natural" split everyone
> agreed on, and I saw this distutils <-> Makefile link like something to fix.
>
> So, it sounds like a bad idea now :)

Parsing the Makefile at runtime seems like an insane choice anyway to
me: +1 for your new module having constants generated at ./configure time.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAksA1LEACgkQ+gerLs4ltQ5gCQCgtYpBewlnocHJf5hp33TfkLjG
72IAnRW1d9n2CO8S2V+9ewcMb81oWNL+
=GG45
-----END PGP SIGNATURE-----

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ncoghlan at gmail

Nov 15, 2009, 8:43 PM

Post #17 of 28 (1454 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Tres Seaver wrote:
> Tarek Ziad� wrote:
>> I didn't know the split story went like this. I took it like the
>> "natural" split everyone
>> agreed on, and I saw this distutils <-> Makefile link like something to fix.
>
>> So, it sounds like a bad idea now :)
>
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me: +1 for your new module having constants generated at ./configure time.

I'm with Tres here - having distutils (aside from build_ext) depend on
non-Python parts of the source tree seem a little strange.

However, given that the recommended packaging includes the needed files,
putting an RFE on the tracker to reduce the runtime dependency on the
source code would be an acceptable response if there is something else
you'd rather be working on.

Cheers,
Nick.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

--
Nick Coghlan | ncoghlan [at] gmail | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

a.badger at gmail

Nov 16, 2009, 11:15 AM

Post #18 of 28 (1416 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 02:31:45PM +0100, Georg Brandl wrote:
> Antoine Pitrou schrieb:
> > Tarek Ziad� <ziade.tarek <at> gmail.com> writes:
> >>
> >> This cannot work on all platforms, when our Makefile is not shipped
> >> with python but python-devel. (like Fedora)
> >
> > This practice is stupid anyway, because it means you have to install
> > python-devel even to install pure Python packages with setuptools/distribute.
> > Just ask Fedora, Mandriva and friends to change their packaging practice
> > (Mandriva already has a bug open for that by the way).
>
> +1. They are the ones splitting what "make install" installs into several
> packages, so they are the ones who have to fix the resulting dependency
> problems.
>
I agree with this, however, my point on the bug was more akin to this:

Tres Seaver wrote:
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me: +1 for your new module having constants generated at ./configure
> time.

Makefiles and C header files are not intended as general purpose data
formats. Using them as such has a variety of disadvantages:

* If someone else wants to get at the data, they have to go through the API
in distutils. Any data that's not exposed by the API is unavailable.
* Since disturils doesn't implement a full parser for the make and C syntax
it is possible to break distutils when making legitimate changes to those
build files.

These are the reasons I opened the bug to get that information into a real
data file rather than parsing the Makefile and header files.

I'll also mention two further things:

The reason that python-devel was split off was to make it more useful for
livecds, olpc, embedded systems, and other places where disk space is at a
premimum. Being able to combine an operating system that is used by people
beyond your immediate community is great for finding and fixing bugs before
your users run into them. Being able to program in a high level language on
these platforms has benefits that I'm sure everyone here can appreciate.

I've brought the issue of Makefile and pyconfig.h being needed for distutils
to the attention of every new Fedora python maintainer since the package
split was made. The current maintainer, David Malcolm, agrees that
distutils.sysconfig needs to be able to use this data and he has moved the
Makefile and header files into the main python package. This doesn't change
the problems with using a Makefile and C header files as a data format for
python.

-Toshio

floris.bruynooghe at gmail

Nov 16, 2009, 11:35 AM

Post #19 of 28 (1417 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 11:27:29PM -0500, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Tarek Ziad� wrote:
> > On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists [at] cheimes> wrote:
> > [..]
> >> Do we really want to change distutils to solve a problem of a third
> >> party packaging system when the problem was created by the very same
> >> third party in the first place? In other words: Should you spend your
> >> precious development time with fixing a problem that isn't our fault?
> >> The decision to split the header files into a separate package, that
> >> isn't installed by default, has already created tons of bad user
> >> experience in the past. Do you want to endorse the split even further?
> >
> > I didn't know the split story went like this. I took it like the
> > "natural" split everyone
> > agreed on, and I saw this distutils <-> Makefile link like something to fix.
> >
> > So, it sounds like a bad idea now :)
>
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me: +1 for your new module having constants generated at ./configure time.

+1

There have been bugs in the past about
distutils.sysconfig.parse_makefile() not knowing the full make syntax
and there will no doubt still be bugs like this. Substituting these
variables at ./configure time into a .py.in file would be much safer.

Regards
Floris

--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

amcnabb at mcnabbs

Nov 16, 2009, 12:10 PM

Post #20 of 28 (1414 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, Nov 15, 2009 at 03:35:00AM +0100, Christian Heimes wrote:
>
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?

The problem being addressed is that distutils is using an artefact of
the build system (a Makefile) as a data file. I think the issue of
whether or not to "endorse the split" is a red herring.

--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

dmalcolm at redhat

Nov 16, 2009, 1:09 PM

Post #21 of 28 (1414 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Sun, 2009-11-15 at 12:42 +0000, Antoine Pitrou wrote:
> Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
> >
> > This cannot work on all platforms, when our Makefile is not shipped
> > with python but python-devel. (like Fedora)
>
> This practice is stupid anyway, because it means you have to install
> python-devel even to install pure Python packages with setuptools/distribute.
> Just ask Fedora, Mandriva and friends to change their packaging practice
> (Mandriva already has a bug open for that by the way).
>
> In Debian/Ubuntu, the Makefile is correctly part of the main Python package:
>
> $ dpkg -S /usr/lib/python2.5/config/Makefile
> python2.5: /usr/lib/python2.5/config/Makefile
For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ;
the bug tracking it was:
https://bugzilla.redhat.com/show_bug.cgi?id=531901

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 19, 2009, 1:30 AM

Post #22 of 28 (1277 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Mon, Nov 16, 2009 at 8:15 PM, Toshio Kuratomi <a.badger [at] gmail> wrote:
[..]
> I've brought the issue of Makefile and pyconfig.h being needed for distutils
> to the attention of every new Fedora python maintainer since the package
> split was made. �The current maintainer, David Malcolm, agrees that
> distutils.sysconfig needs to be able to use this data and he has moved the
> Makefile and header files into the main python package. �This doesn't change
> the problems with using a Makefile and C header files as a data format for
> python.

Great News !

Now for the format problem, I agree that it seems more robust to
pre-process the variables
and inject them in the stdlib when ./configure is run.

I am not sure what is the best strategy here, but I would rather not
add yet another
configuration file (wether its an xml format or an ini-like format).

So what I am proposing is to inject those values in a private dict in
the new sysconfig.py module,
that can be read through the get_config_vars / get_config_var APIs.

This means that sysconfig.py will be added as "sysconfig.py.in"

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

ziade.tarek at gmail

Nov 19, 2009, 1:32 AM

Post #23 of 28 (1277 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

On Mon, Nov 16, 2009 at 10:09 PM, David Malcolm <dmalcolm [at] redhat> wrote:
[..]
>>
>> $ dpkg -S /usr/lib/python2.5/config/Makefile
>> python2.5: /usr/lib/python2.5/config/Makefile
> For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ;
> the bug tracking it was:
> https://bugzilla.redhat.com/show_bug.cgi?id=531901

Thanks David !
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

lists at cheimes

Nov 19, 2009, 1:55 AM

Post #24 of 28 (1278 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Tarek Ziad� wrote:
> Now for the format problem, I agree that it seems more robust to
> pre-process the variables
> and inject them in the stdlib when ./configure is run.
>
> I am not sure what is the best strategy here, but I would rather not
> add yet another
> configuration file (wether its an xml format or an ini-like format).
>
> So what I am proposing is to inject those values in a private dict in
> the new sysconfig.py module,
> that can be read through the get_config_vars / get_config_var APIs.
>
> This means that sysconfig.py will be added as "sysconfig.py.in"

Are you planing to implement the proposal for Windows, too? There may be
no need to change the code because we have full control over the
official installers. Also the VisualStudio Windows build doesn't use
./configure but hand crafted header files. You'd have to fall back to a
hand crafter sysconfig.py.

Christian
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

solipsis at pitrou

Nov 19, 2009, 2:04 AM

Post #25 of 28 (1277 views)
Permalink

Re: buildtime vs runtime in Distutils [In reply to]

Tarek Ziadé <ziade.tarek <at> gmail.com> writes:
>
> So what I am proposing is to inject those values in a private dict in
> the new sysconfig.py module,
> that can be read through the get_config_vars / get_config_var APIs.
>
> This means that sysconfig.py will be added as "sysconfig.py.in"

This means you have to relaunch the whole configure thing each time you make a
change to sysconfig? This doesn't sound like a good idea to me. You could make
the private dict a dedicated module instead, e.g. "_sysconfig_data.py.in".

As for Windows, if people have the ability to choose the installation directory
when installing, I'm not sure how you're gonna handle it.

_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com