Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Refactoring installation schemes

  

 
Python dev RSS feed   Index | Next | Previous | View Threaded


ziade.tarek at gmail

Oct 27, 2009, 2:55 PM

Post #1 of 18 (361 views)
Permalink
Refactoring installation schemes
Hello,

Since the addition of PEP 370, (per-user site packages), site.py and
distutils/command/install.py are *both* providing the various
installation directories for Python,
depending on the system and the Python version.

We have also started to discuss lately in various Mailing Lists the
addition of new schemes for IronPython and Jython, meaning that we
might add some more in both places.

I would like to suggest a simplification by adding a dedicated module
to manage these installation schemes in one single place in the
stdlib.

This new independant module would be used by site.py and distutils and
would also make it easier for third party code to work with these
schemes.
Of course this new module would be rather simple and not add any new
import statement to avoid any overhead when Python starts and loads
site.py

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


mal at egenix

Oct 28, 2009, 3:43 AM

Post #2 of 18 (335 views)
Permalink
Re: Refactoring installation schemes [In reply to]
Tarek Ziadé wrote:
> Hello,
>
> Since the addition of PEP 370, (per-user site packages), site.py and
> distutils/command/install.py are *both* providing the various
> installation directories for Python,
> depending on the system and the Python version.
>
> We have also started to discuss lately in various Mailing Lists the
> addition of new schemes for IronPython and Jython, meaning that we
> might add some more in both places.
>
> I would like to suggest a simplification by adding a dedicated module
> to manage these installation schemes in one single place in the
> stdlib.
>
> This new independant module would be used by site.py and distutils and
> would also make it easier for third party code to work with these
> schemes.
> Of course this new module would be rather simple and not add any new
> import statement to avoid any overhead when Python starts and loads
> site.py

+1

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Oct 28 2009)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fuzzyman at voidspace

Oct 28, 2009, 4:02 AM

Post #3 of 18 (335 views)
Permalink
Re: Refactoring installation schemes [In reply to]
M.-A. Lemburg wrote:
> Tarek Ziadé wrote:
>
>> Hello,
>>
>> Since the addition of PEP 370, (per-user site packages), site.py and
>> distutils/command/install.py are *both* providing the various
>> installation directories for Python,
>> depending on the system and the Python version.
>>
>> We have also started to discuss lately in various Mailing Lists the
>> addition of new schemes for IronPython and Jython, meaning that we
>> might add some more in both places.
>>
>> I would like to suggest a simplification by adding a dedicated module
>> to manage these installation schemes in one single place in the
>> stdlib.
>>
>> This new independant module would be used by site.py and distutils and
>> would also make it easier for third party code to work with these
>> schemes.
>> Of course this new module would be rather simple and not add any new
>> import statement to avoid any overhead when Python starts and loads
>> site.py
>>
>
> +1
>
>
Also +1. It seems like this would make things easier for the alternative
implementations.

Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ssteinerx at gmail

Oct 28, 2009, 7:30 AM

Post #4 of 18 (332 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:

> M.-A. Lemburg wrote:
>> Tarek Ziadé wrote:
>>
>>> Hello,
>>>
>>> Since the addition of PEP 370, (per-user site packages), site.py and
>>> distutils/command/install.py are *both* providing the various
>>> installation directories for Python,
>>> depending on the system and the Python version.
>>>
>>> We have also started to discuss lately in various Mailing Lists the
>>> addition of new schemes for IronPython and Jython, meaning that we
>>> might add some more in both places.
>>>
>>> I would like to suggest a simplification by adding a dedicated
>>> module
>>> to manage these installation schemes in one single place in the
>>> stdlib.
>>>
>>> This new independant module would be used by site.py and distutils
>>> and
>>> would also make it easier for third party code to work with these
>>> schemes.
>>> Of course this new module would be rather simple and not add any new
>>> import statement to avoid any overhead when Python starts and loads
>>> site.py

+1

This would help unpollute ~/ which is getting quite full of 'stuff.'

S

_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ziade.tarek at gmail

Oct 28, 2009, 11:05 AM

Post #5 of 18 (331 views)
Permalink
Re: Refactoring installation schemes [In reply to]
M.-A. Lemburg wrote:
> +1

On Oct 28, 2009, at 7:02 AM, Michael Foord wrote:
> Also +1. It seems like this would make things easier for the alternative implementations.

On Wed, Oct 28, 2009 at 3:30 PM, ssteinerX[at]gmail.com
<ssteinerx[at]gmail.com> wrote:
> +1

Ok then I'll work on a patch for that change and start an issue about it soon.

> This would help unpollute ~/ which is getting quite full of 'stuff.'

Notice that there's a discussion going on about that at
http://bugs.python.org/issue7175

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ziade.tarek at gmail

Nov 4, 2009, 4:29 AM

Post #6 of 18 (240 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Wed, Oct 28, 2009 at 7:05 PM, Tarek Ziadé <ziade.tarek[at]gmail.com> wrote:
> Ok then I'll work on a patch for that change and start an issue about it soon.

As I expected, being able to provide all those paths pulls a lot of
other stuffs out of distutils.

In fact, almost all the APIs that are located in distutils/sysconfig
can be taken out of distutils, and cleaned up for stdlib's benefit.
I've started to refactor the code in a module I have called
"sysconfig", reusing distutils/sysconfig, distutils/command/install
and site code.

This "sysconfig" module should provide at the end very useful APIs to
query the current Python environment.

That's a work in progress but: I've started a branch at
/python/branches/tarek_sysconfig so it's easier to get some feedbacks
if anyone want to help on this.

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


david.lyon at preisshare

Nov 4, 2009, 2:16 PM

Post #7 of 18 (233 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Wed, 4 Nov 2009 13:29:35 +0100, Tarek Ziadé <ziade.tarek[at]gmail.com>
wrote:
> I've started to refactor the code in a module I have called
> "sysconfig", reusing distutils/sysconfig, distutils/command/install
> and site code.
>
> This "sysconfig" module should provide at the end very useful APIs to
> query the current Python environment.
>
> That's a work in progress but: I've started a branch at
> /python/branches/tarek_sysconfig so it's easier to get some feedbacks
> if anyone want to help on this.

Good job so far. Keep going.

imho a refactoring of the installation and build schemes of distutils
is due. It seems like you easily have the skills to do it.

My advice would be to do it gradually, as you are. And focus on
simplification, and filling in the functional gaps. Ask people what
the functional gaps are on their platforms and try to mould it
together in an unplatform specific way.

Myself and others can assist with this, but best to do it on
distutils-sig.

I would even go so far as to use the python 3 as a carrot for
the new work.

imho The biggest and best thing that you could do for python
packaging is to do a refactor of the .EGG format.

What I mean here, is *take* the egg, and run with it. Modernise
it and revamp it into a platform independent thing. People
'get' the egg idea, they just hate the current implementation.

What we all *need* is for the egg to be what the source
distribution now is. Not for it to be some proprietory out
out of python object.

Call it the new Tarek egg...

Then refactor distutils build to focus on good egg creation..

Many people will help you.. including myself..

David








_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


fdrake at acm

Nov 4, 2009, 2:40 PM

Post #8 of 18 (232 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Wed, Nov 4, 2009 at 5:16 PM, David Lyon <david.lyon[at]preisshare.net> wrote:
> I would even go so far as to use the python 3 as a carrot for
> the new work.

The packaging story is in such bad shape that it needs the work
regardless, and keeping it to Python 3 would significantly reduce the
set of potential volunteers.

> imho The biggest and best thing that you could do for python
> packaging is to do a refactor of the .EGG format.
>
> What I mean here, is *take* the egg, and run with it. Modernise
> it and revamp it into a platform independent thing. People
> 'get' the egg idea, they just hate the current implementation.

There's certainly work on that, but... is it that people hate the
format? Or working with setuptools?

The fact that there's more than one egg format doesn't help, so you
may be right.

> Call it the new Tarek egg...

The tegg? ;-)


-Fred

--
Fred L. Drake, Jr. <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


david.lyon at preisshare

Nov 4, 2009, 4:13 PM

Post #9 of 18 (232 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Wed, 4 Nov 2009 17:40:35 -0500, Fred Drake <fdrake[at]acm.org> wrote:

> The packaging story is in such bad shape that it needs the work
> regardless, and keeping it to Python 3 would significantly reduce the
> set of potential volunteers.

Well I guess that is a 'marketing decision'. Not a coding issue.

Actually, I don't honestly think that there is a lack of volounteers
or any lack of resources.

The main challenge imho is to get peoples bikesheds (including my own
bikeshed) to line up in such a way that people can ride together
sharing some sort of parts.

Tarek has been doing just fine at facilitating this.

> There's certainly work on that, but... is it that people hate the
> format? Or working with setuptools?

In my experience, working with the setuptools implementation is
the difficult part. There's a barrage of impossible to remember
command line options and the documentation is imho more convoluted
than it needs to be.

Let me put the distutils documentation forward as some sort of
reference as to where the setuptools documentation should be.

Then, there are some relatively minor issues that just annoy
users to no end. The simple one is that they live in your
site-packages directory unpacked. People wouldn't have so
many issues if they lived like every other 'normal' package.

As for the format itself, there's nothing to hate that I
can see.

> The fact that there's more than one egg format doesn't help, so you
> may be right.

Yes. The implementation can be confusing. That's the only
problem imo.

>> Call it the new Tarek egg...
>
> The tegg? ;-)

l'oeuf ?

In summmary, it doesn't matter so much what an 'egg' is.

All that is important is that it works on lots of python
platforms, distutils can easily pop them out, they are well
documented and relatively trouble free.

Most importantly, credit for the original idea goes back
to PJE and everything gets refactored to make it 'nice'.

I don't think it's a massive refactoring operation myself,
and I'm very sure well within Tareks skillset to take on.

David



_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ziade.tarek at gmail

Nov 5, 2009, 2:35 AM

Post #10 of 18 (225 views)
Permalink
Re: Refactoring installation schemes [In reply to]
2009/11/5 David Lyon <david.lyon[at]preisshare.net>:
>>> Call it the new Tarek egg...
>>
>> The tegg?  ;-)
>
> l'oeuf ?

While I like 'tegg' a lot ;) , I would like to make it clear that
"eggs" are not part of Distutils. That's the Setuptools format.

Although there's a distinction to make between the *installation*
format, and the *distribution* format (sdist, bdist, etc)

Right now we have several installation formats which leads to problems
I won't mention here.

PEP 376 is working on a default, unified, *installation* format, that
tries to gather the good ideas of Pip, Setuptools etc.. and propose a
unified format for our site-packages. This new standard will come with
APIs in pkgutil to be able to query installed distribution etc. This
work is also linked to PEP 345 work where we are modifying the
Metadata, and to PEP 386 that proposes a standard version comparison
scheme.

But there's no plan to include a new *distribution* format in Distutils.

In any case those PEPs are not finished yet, so everyone can help at
distutils-SIG

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


david.lyon at preisshare

Nov 5, 2009, 2:29 PM

Post #11 of 18 (214 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé <ziade.tarek[at]gmail.com>
wrote:
> PEP 376 is working on a default, unified, *installation* format, that
> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
> unified format for our site-packages. This new standard will come with
> APIs in pkgutil to be able to query installed distribution etc. This
> work is also linked to PEP 345 work where we are modifying the
> Metadata, and to PEP 386 that proposes a standard version comparison
> scheme.

Perphaps..

But if you put all these PEPs together, implementing all the new features
can't come to more than 300 lines of code...

Since we hardly got anywhere on them in 2009, it will be interesting to
see how much of it gets done in 2010.

> But there's no plan to include a new *distribution* format in Distutils.

I wasn't suggesting that - at all.

And saying that 'eggs' are a *new* python package format isn't really
really helpful because to my understanding they've been around for
some number of years.

No, i won't raise why we have EGG_INFO directories and a whole lot
of half working egg stuff in standard python... I'm just asking
why it can't be more consistant? while we're on the refactoring
topic.

Be fair...

I'm saying finish what is already there.. or take out the crap ..

It isn't fair to suggest that I am somehow asking for some big
change when I am simply pointing out all the junk that's in
there that's already half built.

> In any case those PEPs are not finished yet, so everyone can help at
> distutils-SIG

True - and False.

But I've been on the list for some twelve months asking for work
to help out with, and haven't been assigned a single task to do
yet.

Seriously, if you won't allocate work out to people then how can
it get done?

Whilst I personally think a lot of the stuff in those PEPs is not
high on quality, why don't we just get them implemented anyway?

I'm a fairly proficient develper, but I can't get assigned a single
work item..

And to me, it doesn't seem any harder than just selecting 'djlyon'
on the python tracker for some work items...

Surely those PEPs all amount to 300+ lines of code. With two people
working on it, that's only surely 150+ lines of code each... That
shouldn't be such a big challenge for 2010..

David

_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


robert.kern at gmail

Nov 5, 2009, 2:57 PM

Post #12 of 18 (214 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On 2009-11-05 16:29 PM, David Lyon wrote:

> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?

With rare exceptions, open source projects don't really "assign" work. If you
want to help, help. Don't wait for someone to tell you exactly what to do. No
one will. Go to the tracker, find something interesting, and do it.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ziade.tarek at gmail

Nov 5, 2009, 3:03 PM

Post #13 of 18 (214 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Thu, Nov 5, 2009 at 11:29 PM, David Lyon <david.lyon[at]preisshare.net> wrote:
[...]
>
> But if you put all these PEPs together, implementing all the new features
> can't come to more than 300 lines of code...
>
> Since we hardly got anywhere on them in 2009, it will be interesting to
> see how much of it gets done in 2010.

The work that was done during the last year was brainstorming on PEPs.
You can't count that work
in SLOCs, but in trying to find a consensus on packaging problems.

And I can fairly say that we are very close to something that can get
accepted by the community,
then turned into code in Distutils/pkgutil.

>
>> But there's no plan to include a new *distribution* format in Distutils.
>
> I wasn't suggesting that - at all.
>
> And saying that 'eggs' are a *new* python package format isn't really
> really helpful because to my understanding they've been around for
> some number of years.

They are new to Distutils. They are not new to the Python packaging
eco-system of course,
and PEP 376 rely on many ideas created in Setuptools.

> No, i won't raise why we have EGG_INFO directories and a whole lot
> of half working egg stuff in standard python... I'm just asking
> why it can't be more consistant? while we're on the refactoring
> topic.

The consistency will come through the standard we are building in PEP 376.

[..]
> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?

I am sorry that you feel that way. We don't allocate coding work to
people in this process.
That's not the way it works.

Rather, people help in building the PEPs by providing their own
feedback/experience.

At the end, we are trying to have PEPs that adresses the wider range of cases.


>
> Whilst I personally think a lot of the stuff in those PEPs is not
> high on quality, why don't we just get them implemented anyway?
>
> I'm a fairly proficient develper, but I can't get assigned a single
> work item..

We have already prototypes for each PEP so people can try them out,
enhance them while the PEPs
are being built. If you want to help in their coding, you are more
than welcome. They are on a DVCS
at bitbucket.

>
> And to me, it doesn't seem any harder than just selecting 'djlyon'
> on the python tracker for some work items...
>
> Surely those PEPs all amount to 300+ lines of code. With two people
> working on it, that's only surely 150+ lines of code each... That
> shouldn't be such a big challenge for 2010..

Again, the big challenge is not on the coding part. If it was,
Distutils would have them already.
The challenge is on the PEPs, and on making sure we collect all PoVs
and feedbacks, before we change
Distutils in Python 2.7/3.2

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


guido at python

Nov 5, 2009, 3:18 PM

Post #14 of 18 (214 views)
Permalink
Re: Refactoring installation schemes [In reply to]
David, you have an attitude problem. Your contributions (the post
below is just an example) don't sound healthy to me -- you just
complain and whine and denigrate Tarek's work. In a previous post you
claimed to have had a particular idea first (it doesn't matter which
idea) and you managed to make it sound bad that your idea was
eventually accepted. This is not a productive attitude.

Surely the problem isn't writing 300 lines of code. The problem is
getting everyone to agree on which 300 lines of code should be
written. That is the problem at hand, and claiming that nothing
happened because no code was written and all that was agreed on
amounts to 300 lines of code is outright demeaning.

Stop it. You are wearing out your welcome.

--Guido

PS. Submitting a counter-PEP to the peps editors that hasn't been
discussed on the SIG list at all is also a bad move. You really need
to change the way you try to interact with the SIG.

On Thu, Nov 5, 2009 at 2:29 PM, David Lyon <david.lyon[at]preisshare.net> wrote:
> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé <ziade.tarek[at]gmail.com>
> wrote:
>> PEP 376 is working on a default, unified, *installation* format, that
>> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
>> unified format for our site-packages. This new standard will come with
>> APIs in pkgutil to be able to query installed distribution etc. This
>> work is also linked to PEP 345 work where we are modifying the
>> Metadata, and to PEP 386 that proposes a standard version comparison
>> scheme.
>
> Perphaps..
>
> But if you put all these PEPs together, implementing all the new features
> can't come to more than 300 lines of code...
>
> Since we hardly got anywhere on them in 2009, it will be interesting to
> see how much of it gets done in 2010.
>
>> But there's no plan to include a new *distribution* format in Distutils.
>
> I wasn't suggesting that - at all.
>
> And saying that 'eggs' are a *new* python package format isn't really
> really helpful because to my understanding they've been around for
> some number of years.
>
> No, i won't raise why we have EGG_INFO directories and a whole lot
> of half working egg stuff in standard python... I'm just asking
> why it can't be more consistant? while we're on the refactoring
> topic.
>
> Be fair...
>
> I'm saying finish what is already there.. or take out the crap ..
>
> It isn't fair to suggest that I am somehow asking for some big
> change when I am simply pointing out all the junk that's in
> there that's already half built.
>
>> In any case those PEPs are not finished yet, so everyone can help at
>> distutils-SIG
>
> True - and False.
>
> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?
>
> Whilst I personally think a lot of the stuff in those PEPs is not
> high on quality, why don't we just get them implemented anyway?
>
> I'm a fairly proficient develper, but I can't get assigned a single
> work item..
>
> And to me, it doesn't seem any harder than just selecting 'djlyon'
> on the python tracker for some work items...
>
> Surely those PEPs all amount to 300+ lines of code. With two people
> working on it, that's only surely 150+ lines of code each... That
> shouldn't be such a big challenge for 2010..
>
> David
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev[at]python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


david.lyon at preisshare

Nov 5, 2009, 3:55 PM

Post #15 of 18 (215 views)
Permalink
Re: Refactoring installation schemes [In reply to]
Tarek, Guido,

Forgive my grumpy tone..

Looking positive, if we now have a rough consensus that the PEPS might
represent some 300+ lines of code... then good - lets get started, that's
all that I meant. I'm glad above all, that you noticed that part foremost.

If it's a simple case that alternative's to the solutions cannot
be considered - because I am grumpy - whatever, then that's something
I can live with. I didn't know that proposing a different solution
wasn't a good idea.

> PS. Submitting a counter-PEP to the peps editors that hasn't been
> discussed on the SIG list at all is also a bad move. You really need
> to change the way you try to interact with the SIG.

Here are the references to the discussions from distutils-sig. In these
references you'll see that my interaction with the list was not grumpy
but was entirely positive. I feel that I got a fair hearing on distutils
list and I had a number of supporters to my proposal.

The proposal is in the final stages of being wrapped up.

Here are the discussion References that have occurred over the year:

http://mail.python.org/pipermail/distutils-sig/2009-November/thread.html

November 2009 Archives by thread

* Messages sorted by: [ subject ] [ author ] [ date ]
* More info on this list...

Starting: Sun Nov 1 09:31:46 CET 2009
Ending: Fri Nov 6 00:09:22 CET 2009
Messages: 65

* [Distutils] Alternate static metadata PEP submission... Georg
Brandl
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission...
Floris Bruynooghe
+ [Distutils] Alternate static metadata PEP submission...
Georg Brandl
+ [Distutils] Alternate static metadata PEP submission...
David Lyon
# [Distutils] Alternate static metadata PEP
submission... Georg Brandl
# [Distutils] Alternate static metadata PEP
submission... David Lyon


http://mail.python.org/pipermail/distutils-sig/2009-October/thread.html

# [Distutils] Alternate static metadata PEP submission... David Lyon

* [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Eric
Smith
o [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
* [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
* [Distutils] Alternate static metadata PEP submission... Floris
Bruynooghe
o [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission...
Floris Bruynooghe
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Tarek
Ziadé
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Ian
Bicking
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Fred
Drake
o [Distutils] Alternate static metadata PEP submission... Kevin
Teague
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... Fred
Drake
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... Piotr
Ozarowski
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Piotr
Ozarowski
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... Fred
Drake
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... Fred
Drake
o [Distutils] Alternate static metadata PEP submission... Chris
Withers
o [Distutils] Alternate static metadata PEP submission... Eric
Smith
o [Distutils] Alternate static metadata PEP submission... Fred
Drake
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon
o [Distutils] Alternate static metadata PEP submission... David
Cournapeau
o [Distutils] Alternate static metadata PEP submission... David
Lyon

http://mail.python.org/pipermail/distutils-sig/2009-August/thread.html
[Distutils] Static metadata using setup.cfg Tarek Ziadé

* [Distutils] Static metadata using setup.cfg Hanno Schlichting
* [Distutils] Static metadata using setup.cfg P.J. Eby
o [Distutils] Static metadata using setup.cfg Tarek Ziadé
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
* [Distutils] Static metadata using setup.cfg David Lyon
o [Distutils] Static metadata using setup.cfg Jeff Rush
+ [Distutils] Static metadata using setup.cfg Ben Finney
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg David Lyon
+ [Distutils] Static metadata using setup.cfg Eric Smith
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg Chris
Withers
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg Chris
Withers
* [Distutils] Static metadata using setup.cfg David Lyon
o [Distutils] Static metadata using setup.cfg Sridhar
Ratnakumar
+ [Distutils] Static metadata using setup.cfg David Lyon
o [Distutils] Static metadata using setup.cfg Alex Grönholm
+ [Distutils] Static metadata using setup.cfg David Lyon
+ [Distutils] Static metadata using setup.cfg Floris
Bruynooghe
+ [Distutils] Static metadata using setup.cfg Ronald
Oussoren
+ [Distutils] Static metadata using setup.cfg P.J. Eby
+ [Distutils] Static metadata using setup.cfg Chris
Withers
o [Distutils] Static metadata using setup.cfg Chris Withers
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg Chris
Withers
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé
+ [Distutils] Static metadata using setup.cfg David Lyon
+ [Distutils] Static metadata using setup.cfg Tarek
Ziadé

http://mail.python.org/pipermail/distutils-sig/2009-July/thread.html

# [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions
on python-dev) David Lyon
# [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions
on python-dev) P.J. Eby

* [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Eric Smith
* [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Cournapeau
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Cournapeau
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Cournapeau
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Tres Seaver
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Cournapeau
o [Distutils] Questions about virtual-python on ubuntu David
Lyon
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Paul Moore
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Lyon
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Lyon
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) Jim Fulton
o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev) David Lyon

# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure Jeff Rush
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure P.J. Eby
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure Jean-Paul Calderone
# [Distutils] Setup.py as a Class, not a Procedure David Lyon
# [Distutils] Setup.py as a Class, not a Procedure Jean-Paul Calderone
# [Distutils] Setup.py as a Class, not a Procedure David Lyon
[Distutils] introspecting package data Chris Withers

http://mail.python.org/pipermail/distutils-sig/2009-June/thread.html

* [Distutils] introspecting package data Tarek Ziadé
o [Distutils] introspecting package data Chris Withers
+ [Distutils] introspecting package data Jim Fulton
+ [Distutils] introspecting package data Chris Withers
+ [Distutils] introspecting package data Tarek Ziadé
+ [Distutils] introspecting package data David Lyon


On Thu, 5 Nov 2009 15:18:12 -0800, Guido van Rossum <guido[at]python.org>
wrote:
> David, you have an attitude problem. Your contributions (the post
> below is just an example) don't sound healthy to me -- you just
> complain and whine and denigrate Tarek's work. In a previous post you
> claimed to have had a particular idea first (it doesn't matter which
> idea) and you managed to make it sound bad that your idea was
> eventually accepted. This is not a productive attitude.
>
> Surely the problem isn't writing 300 lines of code. The problem is
> getting everyone to agree on which 300 lines of code should be
> written. That is the problem at hand, and claiming that nothing
> happened because no code was written and all that was agreed on
> amounts to 300 lines of code is outright demeaning.
>
> Stop it. You are wearing out your welcome.
>
> --Guido
>
> PS. Submitting a counter-PEP to the peps editors that hasn't been
> discussed on the SIG list at all is also a bad move. You really need
> to change the way you try to interact with the SIG.
>
> On Thu, Nov 5, 2009 at 2:29 PM, David Lyon <david.lyon[at]preisshare.net>
> wrote:
>> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé <ziade.tarek[at]gmail.com>
>> wrote:
>>> PEP 376 is working on a default, unified, *installation* format, that
>>> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
>>> unified format for our site-packages. This new standard will come with
>>> APIs in pkgutil to be able to query installed distribution etc. This
>>> work is also linked to PEP 345 work where we are modifying the
>>> Metadata, and to PEP 386 that proposes a standard version comparison
>>> scheme.
>>
>> Perphaps..
>>
>> But if you put all these PEPs together, implementing all the new
features
>> can't come to more than 300 lines of code...
>>
>> Since we hardly got anywhere on them in 2009, it will be interesting to
>> see how much of it gets done in 2010.
>>
>>> But there's no plan to include a new *distribution* format in
Distutils.
>>
>> I wasn't suggesting that - at all.
>>
>> And saying that 'eggs' are a *new* python package format isn't really
>> really helpful because to my understanding they've been around for
>> some number of years.
>>
>> No, i won't raise why we have EGG_INFO directories and a whole lot
>> of half working egg stuff in standard python... I'm just asking
>> why it can't be more consistant? while we're on the refactoring
>> topic.
>>
>> Be fair...
>>
>> I'm saying finish what is already there.. or take out the crap ..
>>
>> It isn't fair to suggest that I am somehow asking for some big
>> change when I am simply pointing out all the junk that's in
>> there that's already half built.
>>
>>> In any case those PEPs are not finished yet, so everyone can help at
>>> distutils-SIG
>>
>> True - and False.
>>
>> But I've been on the list for some twelve months asking for work
>> to help out with, and haven't been assigned a single task to do
>> yet.
>>
>> Seriously, if you won't allocate work out to people then how can
>> it get done?
>>
>> Whilst I personally think a lot of the stuff in those PEPs is not
>> high on quality, why don't we just get them implemented anyway?
>>
>> I'm a fairly proficient develper, but I can't get assigned a single
>> work item..
>>
>> And to me, it doesn't seem any harder than just selecting 'djlyon'
>> on the python tracker for some work items...
>>
>> Surely those PEPs all amount to 300+ lines of code. With two people
>> working on it, that's only surely 150+ lines of code each... That
>> shouldn't be such a big challenge for 2010..
>>
>> David
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev[at]python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


ziade.tarek at gmail

Nov 5, 2009, 5:09 PM

Post #16 of 18 (211 views)
Permalink
Re: Refactoring installation schemes [In reply to]
2009/11/6 David Lyon <david.lyon[at]preisshare.net>:
[..]
> Here are the references to the discussions from distutils-sig. In these
> references you'll see that my interaction with the list was not grumpy
> but was entirely positive. I feel that I got a fair hearing on distutils
> list and I had a number of supporters to my proposal.
>
> The proposal is in the final stages of being wrapped up.

There are three different topics :

1. a standard for the installation format: PEP 376
2. changes in the metadata format : PEP 345 (-- work in progress here,
some parts are still missing)
3. ways to built the metadata that are shipped into a distribution
statically, using Distutils : PEP 390 was started for that

And as a matter of fact, PEP 390 is not so important, because at the
end, the changes that are made in PEP 345 are
the most important ones. They allow describing metadata fields that
can vary depending on the target platform.

PEP 390 is just the simplest way to make it possible to describe
static metadata using a setup.cfg ConfigParser section,
rather than some code in setup.py. And when we built it, it helped us
understand what we really wanted on PEP 345 side (with a big insights
from people like Marc-André on this)

But PEP 390 limits its scope to the Metadata fields and now waits for
PEP 345 changes.

Your proposal is a partial alternative to PEP 390, and is a quite
interesting work to dig in, and the SIG is discussing it,
but a very important point about it is that it does a lot more. It's a
new *building* system on its own, and does not limit itself to
describing metadata fields.

That's fine, and that's quite interesting, but I doubt we will be able
to push it in Distutils itself any sooner, because
it a whole different system ala scons (which is a great tool don't get
me wrong). I think (as I told you before IIRC) that you need to create
a project to demonstrate it and I think it can be great to continue
getting some help from Distutils-SIG for this.

Furthermore, building such a tool on the top of Distutils can improve
Distutils itself, because it would be yet another project that would
provide valuable feedback on the APIs and standards we provide. That's
for example what we are getting from projects like PyPM or Enthought
Enstaller.

As a matter of fact, there's already great feedback from Active State
on PEP 376 because they want to be early adopters of the upcoming
standard and make sure we do the right choices in there. Enthought
also started to use the prototype we have written for PEP 386 (version
comparisons) IIRC.

But in the existing PEPs discussions, we need to focus on their
respective scopes. And they are all targeted to improve the existing
stdlib tool : Distutils. (and a wanted side effect is to make
Distutils smaller if we can, and a robust basis for all third party
tools)


That's why you can't just drop a counter-PEP to any existing PEP we
have started for Distutils. Imho help us build those PEPs, and / or
create your own build system, wether its based or not on Distutils.

Regards
Tarek
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


brett at python

Nov 5, 2009, 5:25 PM

Post #17 of 18 (211 views)
Permalink
Re: Refactoring installation schemes [In reply to]
On Thu, Nov 5, 2009 at 15:55, David Lyon <david.lyon[at]preisshare.net> wrote:
>
> Tarek, Guido,
>
> Forgive my grumpy tone..
>
> Looking positive, if we now have a rough consensus that the PEPS might
> represent some 300+ lines of code... then good - lets get started, that's
> all that I meant. I'm glad above all, that you noticed that part foremost.
>
> If it's a simple case that alternative's to the solutions cannot
> be considered - because I am grumpy - whatever, then that's something
> I can live with. I didn't know that proposing a different solution
> wasn't a good idea.

That is passive-aggressive and uncalled for. Proposing an opposing
view is fine when done respectfully which you have not done here on
python-dev.

>
>> PS. Submitting a counter-PEP to the peps editors that hasn't been
>> discussed on the SIG list at all is also a bad move. You really need
>> to change the way you try to interact with the SIG.
>
> Here are the references to the discussions from distutils-sig. In these
> references you'll see that my interaction with the list was not grumpy
> but was entirely positive. I feel that I got a fair hearing on distutils
> list and I had a number of supporters to my proposal.
>

That's fine, but it's Tarek's call and as a group we reached the
conclusion that Tarek is doing a fine job and is being more than fair.
It has been stated as such and yet you keep pushing us like we have
not heard you. We did hear you the first time, but you just seem to
not get the message. Adding to the fact that your tone has been an
issue and that we don't want to listen to it anymore has pushed Guido
to the brink and me over the edge. I honestly don't know how Tarek
keeps his patience over on the distutils SIG if he has to deal with
kind of crap constantly.

Personally, I am now sending your emails to the trash since you can't
seem to even reply to Guido's email w/o some nasty undertone.
Hopefully someday you can learn to communicate in a better, friendlier
manner.

-Brett
_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com


david.lyon at preisshare

Nov 5, 2009, 6:51 PM

Post #18 of 18 (206 views)
Permalink
Re: Refactoring installation schemes [In reply to]
Tarek,

> Your proposal is a partial alternative to PEP 390, and is a quite
> interesting work to dig in, and the SIG is discussing it,
> but a very important point about it is that it does a lot more. It's a
> new *building* system on its own, and does not limit itself to
> describing metadata fields.

Actually no, the proposal is only for a metadata installation of packages
that addresses the security problem of requiring users to run untrusted and

unverified setup.py's.

That was where this discussion originated and that is what I am addressing.

PEP-390 doesn't go there at all...

My proposal uses the static metadata contained within existing package
formats (PKG_INFO + sources.txt) and therefore doesn't require any
changes to the build system.

There isn't a new build system. I never proposed that.

However PEP-390 does imply changing the build system.

> That's why you can't just drop a counter-PEP to any existing PEP we
> have started for Distutils. Imho help us build those PEPs, and / or
> create your own build system, wether its based or not on Distutils.

Once again, I can't see why all the fuss.

All I did was have a discussion on distutils-ml about a different way
of specifying package dependencies that might work across different
python platforms.

It was suggested to me that I do a PEP..

So I took the documentation on face value that developers could submit
a PEP-proposal.

> Imho help us build those PEPs,

This could best apply to PEP-345:

(PKG_INFO)
Requires: sys
Requires(python <= 2.4): lxml
Requires(windows): win32com
Requires(linux): pyodbc
Requires(linux ubuntu gnome python <= 3.4): gix
Requires(windows xp python < 2.2): win32prn

Thank's for reading my alternate-metadata installation proposal
and I accept your feedback. What I can do is clarify that my
proposal is not for a build system but for a metadata installation
scheme based on a static setup.py, existing metadata and use of
"python -m setup install" for invocation.

I'm just trying to make this stuff no more complicated than it
needs to be...

David











_______________________________________________
Python-Dev mailing list
Python-Dev[at]python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.