Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Introduction

 

 

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded


g9robjef at cdf

May 21, 2003, 7:04 PM

Post #1 of 26 (835 views)
Permalink
Introduction

Hello all !

I'm new to the list and thought I would quickly introduce myself. My name
is Jeff and I am a university student [4th year] living in Toronto.

I would love to be able to help with Python-dev in some way. I'm
especially interested in issues directly related to the interpreter
itself. I have gained some compiler development experience while at the
university and would love to continue working in this area.

If anyone has any thoughts or suggestions on how best I could proceed in
this direction, I would love to hear them.

Thanks !

Jeff Roberts


tismer at tismer

May 21, 2003, 7:25 PM

Post #2 of 26 (821 views)
Permalink
Re: Introduction [In reply to]

Jeffery Roberts wrote:

> Hello all !
>
> I'm new to the list and thought I would quickly introduce myself. My name
> is Jeff and I am a university student [4th year] living in Toronto.
>
> I would love to be able to help with Python-dev in some way. I'm
> especially interested in issues directly related to the interpreter
> itself. I have gained some compiler development experience while at the
> university and would love to continue working in this area.
>
> If anyone has any thoughts or suggestions on how best I could proceed in
> this direction, I would love to hear them.

All I can say is: Get involved with PyPy!
There is nothing harder, Python related stuff that I know of.
It can of course do some damage to your brain. I know what
I'm talking about.
Google for pypy and you got it.

cheers - chris

--
Christian Tismer :^) <mailto:tismer [at] tismer>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/


tim.one at comcast

May 21, 2003, 8:11 PM

Post #3 of 26 (819 views)
Permalink
RE: Introduction [In reply to]

[Jeffery Roberts]
> I'm new to the list and thought I would quickly introduce myself. My
> name is Jeff and I am a university student [4th year] living in
> Toronto.
>
> I would love to be able to help with Python-dev in some way. I'm
> especially interested in issues directly related to the interpreter
> itself. I have gained some compiler development experience while at
> the university and would love to continue working in this area.
>
> If anyone has any thoughts or suggestions on how best I could proceed
> in this direction, I would love to hear them.

As Christian said, you should enjoy pypy (an ambitious new project). Less
ambitious is a rewrite of the front end, currently in progress on the
ast-branch branch of the Python CVS repository. If you'd like to get your
feet wet first, there's always a backlog of Python bug and patch reports on
SourceForge begging for attention. Check out

http://www.python.org/dev/

for orientation, and leave your spare time at the door <wink>.


martin at v

May 22, 2003, 12:10 AM

Post #4 of 26 (822 views)
Permalink
Re: Introduction [In reply to]

Jeffery Roberts <g9robjef [at] cdf> writes:

> I would love to be able to help with Python-dev in some way. I'm
> especially interested in issues directly related to the interpreter
> itself. I have gained some compiler development experience while at the
> university and would love to continue working in this area.

In addition to what Christian suggested, the most valuable short-term
contribution would be to look into open bug reports, and propose fixes
for them. In particular, the Parser/Compiler, and "Python Interpreter
Core" bug categories might attract you (there are 4 bugs in the
former, and about 40 in the latter category). Many of these issues are
still open because they are really tricky, so expect some of these to
be middle-sized projects on their own.

Regards,
Martin


bac at OCF

May 22, 2003, 1:29 PM

Post #5 of 26 (836 views)
Permalink
Re: Introduction [In reply to]

Tim Peters wrote:
>>I moved over to Mozilla Mail and I keep hitting "Reply" when I mean to
>>hit "Reply All". Sorry about that.
>
>
> Oh, it doesn't bother me a bit, Brett! I'm more concerned that your
> response would have been helpful to the OP, and he didn't get to see it.
>

Well, lets find out! Here is my email that was meant to go to the list
pasted below.

Tim Peters wrote:

> [Jeffery Roberts]
>
<snip>

>> I would love to be able to help with Python-dev in some way. I'm
>> especially interested in issues directly related to the interpreter
>> itself. I have gained some compiler development experience while at
>> the university and would love to continue working in this area.
>>
>> If anyone has any thoughts or suggestions on how best I could proceed
>> in this direction, I would love to hear them.
>
>
> If you'd like to get your
> feet wet first, there's always a backlog of Python bug and patch
reports on
> SourceForge begging for attention.


I know I learned a lot from working on patches and bugs. It especially
helps if you jump in on a patch that is being actively worked on and can
ask how something works. Otherwise just read the source until your eyes
bleed and curse anyone who doesn't write extensive documentation for
code. =)

There also has been mention of the AST branch. I know I plan on working
on that after I finish going through the bug and patch backlog. Only
trouble is that the guys who actually fully understand it (Jeremy, Tim,
and Neal) are rather busy so it is going to be a "jump in the pool and
drown and hope your flailing manages to at least generate something
useful but you die and come back in another life wiser and able to
attempt again until you stop drowning and manage to only get sick from
gulping down so much chlorinated water". =)

> Check out
>
> http://www.python.org/dev/
>
> for orientation, and leave your spare time at the door <wink>.
>

I will vouch for the loss of spare time. This has become a job. Best
job ever, though. =)

The only big piece of advice I can offer is to just make sure you are
nice and cordial on the list; there is a low tolerance for jerks here.
Don't take this as meaning to not take a stand on an issue! All I am
saying is realize that email does not transcribe humor perfectly and
until the list gets used to your personal writing style you might have
to just make sure what you write does not come off as insulting.

-Brett


g9robjef at cdf

May 22, 2003, 1:56 PM

Post #6 of 26 (816 views)
Permalink
RE: Introduction [In reply to]

Thanks for all of your replies. The front-end rewrite sounds especially
interesting. I'm going to look into that. Is the entire front end
changing (ie scan/parse/ast) or just the AST structure ?

If you have any more information or directions please let me know.

Jeff

On Wed, 21 May 2003, Tim Peters wrote:

> [Jeffery Roberts]
> > I'm new to the list and thought I would quickly introduce myself. My
> > name is Jeff and I am a university student [4th year] living in
> > Toronto.
> >
> > I would love to be able to help with Python-dev in some way. I'm
> > especially interested in issues directly related to the interpreter
> > itself. I have gained some compiler development experience while at
> > the university and would love to continue working in this area.
> >
> > If anyone has any thoughts or suggestions on how best I could proceed
> > in this direction, I would love to hear them.
>
> As Christian said, you should enjoy pypy (an ambitious new project). Less
> ambitious is a rewrite of the front end, currently in progress on the
> ast-branch branch of the Python CVS repository. If you'd like to get your
> feet wet first, there's always a backlog of Python bug and patch reports on
> SourceForge begging for attention. Check out
>
> http://www.python.org/dev/
>
> for orientation, and leave your spare time at the door <wink>.
>
>


bac at OCF

May 22, 2003, 8:21 PM

Post #7 of 26 (822 views)
Permalink
Re: Introduction [In reply to]

Jeffery Roberts wrote:
> Thanks for all of your replies. The front-end rewrite sounds especially
> interesting. I'm going to look into that. Is the entire front end
> changing (ie scan/parse/ast) or just the AST structure ?
>
> If you have any more information or directions please let me know.
>

It is just a new AST. Redoing/replacing pgen is something else
entirely. =)

The branch that this is being developed under in CVS is ast-branch.
There is a incomplete README in Python/compile.txt that explains the
basic idea and direction.

-Brett


jeremy at ZOPE

May 23, 2003, 9:23 AM

Post #8 of 26 (818 views)
Permalink
RE: Introduction [In reply to]

On Thu, 2003-05-22 at 16:56, Jeffery Roberts wrote:
> Thanks for all of your replies. The front-end rewrite sounds especially
> interesting. I'm going to look into that. Is the entire front end
> changing (ie scan/parse/ast) or just the AST structure ?
>
> If you have any more information or directions please let me know.

The current plan is to create an AST and replace the bytecode compiler.
We're leaving a rewrite of the parser for a later project. It's a
fairly big project; large parts of it are done, but there is work
remaining to do in nearly every part -- the concrete-to-abstract
translator, error checking, compilation to byte-code.

At the moment, it's possible to start an interactive interpreter session
and see what works. But it isn't possible to compile and run all of
site.py and everything it imports.

Jeremy


jeremy at zope

May 23, 2003, 12:44 PM

Post #9 of 26 (839 views)
Permalink
RE: Introduction [In reply to]

On Fri, 2003-05-23 at 15:44, logistix wrote:
> Should patches just go to sourceforge's "parser/compiler" category, or
> will that create too much confusion?

I think that would be fine. We don't have a lot of parser/compiler
patches.

Jeremy


logistix at cathoderaymission

May 23, 2003, 12:44 PM

Post #10 of 26 (823 views)
Permalink
RE: Introduction [In reply to]

On 23 May 2003, Jeremy Hylton wrote:

> On Thu, 2003-05-22 at 16:56, Jeffery Roberts wrote:
> > Thanks for all of your replies. The front-end rewrite sounds especially
> > interesting. I'm going to look into that. Is the entire front end
> > changing (ie scan/parse/ast) or just the AST structure ?
> >
> > If you have any more information or directions please let me know.
>
> The current plan is to create an AST and replace the bytecode compiler.
> We're leaving a rewrite of the parser for a later project. It's a
> fairly big project; large parts of it are done, but there is work
> remaining to do in nearly every part -- the concrete-to-abstract
> translator, error checking, compilation to byte-code.
>
> At the moment, it's possible to start an interactive interpreter session
> and see what works. But it isn't possible to compile and run all of
> site.py and everything it imports.
>
> Jeremy
>

Should patches just go to sourceforge's "parser/compiler" category, or
will that create too much confusion?


g9robjef at cdf

May 26, 2003, 9:51 AM

Post #11 of 26 (820 views)
Permalink
Re: Introduction [In reply to]

Thanks for that reply Brett. It is really helpful.

I'm currently in Ottawa at the GCC summit trying to sponge some knowledge
but I will begin following your advice when I get back home later this
week.

Thanks again !

Jeff

> I know I learned a lot from working on patches and bugs. It especially
> helps if you jump in on a patch that is being actively worked on and can
> ask how something works. Otherwise just read the source until your eyes
> bleed and curse anyone who doesn't write extensive documentation for
> code. =)
>
> There also has been mention of the AST branch. I know I plan on working
> on that after I finish going through the bug and patch backlog. Only
> trouble is that the guys who actually fully understand it (Jeremy, Tim,
> and Neal) are rather busy so it is going to be a "jump in the pool and
> drown and hope your flailing manages to at least generate something
> useful but you die and come back in another life wiser and able to
> attempt again until you stop drowning and manage to only get sick from
> gulping down so much chlorinated water". =)
>
> > Check out
> >
> > http://www.python.org/dev/
> >
> > for orientation, and leave your spare time at the door <wink>.
> >
>
> I will vouch for the loss of spare time. This has become a job. Best
> job ever, though. =)
>
> The only big piece of advice I can offer is to just make sure you are
> nice and cordial on the list; there is a low tolerance for jerks here.
> Don't take this as meaning to not take a stand on an issue! All I am
> saying is realize that email does not transcribe humor perfectly and
> until the list gets used to your personal writing style you might have
> to just make sure what you write does not come off as insulting.
>
> -Brett
>
>
>
> --__--__--
>
> Message: 9
> Date: Thu, 22 May 2003 20:21:20 -0700
> From: "Brett C." <bac [at] OCF>
> Reply-To: drifty [at] alum
> To: Jeffery Roberts <g9robjef [at] cdf>
> CC: Tim Peters <tim.one [at] comcast>, python-dev [at] python
> Subject: Re: [Python-Dev] Introduction
>
> Jeffery Roberts wrote:
> > Thanks for all of your replies. The front-end rewrite sounds especially
> > interesting. I'm going to look into that. Is the entire front end
> > changing (ie scan/parse/ast) or just the AST structure ?
> >
> > If you have any more information or directions please let me know.
> >
>
> It is just a new AST. Redoing/replacing pgen is something else
> entirely. =)
>
> The branch that this is being developed under in CVS is ast-branch.
> There is a incomplete README in Python/compile.txt that explains the
> basic idea and direction.
>
> -Brett
>
>
>
> --__--__--
>
> Message: 10
> Date: Thu, 22 May 2003 23:30:45 -0400
> Cc: python-list [at] python, python-dev [at] python
> To: python-announce [at] python
> From: Barry Warsaw <barry [at] python>
> Subject: [Python-Dev] RELEASED Python 2.2.3c1
>
> I'm happy to announce the release of Python 2.2.3c1 (release candidate
> 1). This is a bug fix release for the stable Python 2.2 code line.
> Barring any critical issues, we expect to release Python 2.2.3 final by
> this time next week. We encourage those with an interest in a solid
> 2.2.3 release to download this candidate and test it on their code.
>
> The new release is available here:
>
> http://www.python.org/2.2.3/
>
> Python 2.2.3 has a large number of bug fixes and memory leak patches.
> For full details, see the release notes at
>
> http://www.python.org/2.2.3/NEWS.txt
>
> There are a small number of minor incompatibilities with Python 2.2.2;
> for details see:
>
> http://www.python.org/2.2.3/bugs.html
>
> Perhaps the most important is that the Bastion.py and rexec.py modules
> have been disabled, since we do not deem them to be safe.
>
> As usual, a Windows installer and a Unix/Linux source tarball are made
> available, as well as tarballs of the documentation in various forms.
> At the moment, no Mac version or Linux RPMs are available, although I
> expect them to appear soon after 2.2.3 final is released.
>
> On behalf of Guido, I'd like to thank everyone who contributed to this
> release, and who continue to ensure Python's success.
>
> Enjoy,
> -Barry
>
>
>
> --__--__--
>
> Message: 11
> Date: Fri, 23 May 2003 13:42:20 +0200
> Subject: Re: [Python-Dev] RELEASED Python 2.2.3c1
> Cc: python-dev [at] python
> To: Barry Warsaw <barry [at] python>
> From: Jack Jansen <Jack.Jansen [at] cwi>
>
>
> On Friday, May 23, 2003, at 05:30 Europe/Amsterdam, Barry Warsaw wrote:
>
> > I'm happy to announce the release of Python 2.2.3c1 (release candidate
> > 1).
>
> Oops, that suddenly went *very* fast, I though I had until the
> weekend...
>
> Is there a chance I could get #723495 still in before 2.2.3 final? I
> was also hoping to find a fix for #571343, but I don't have a patch yet
> (although I'll try to get one up in the next few hours).
> --
> Jack Jansen, <Jack.Jansen [at] cwi>, http://www.cwi.nl/~jack
> If I can't dance I don't want to be part of your revolution -- Emma
> Goldman
>
>
>
> --__--__--
>
> Message: 12
> Date: Fri, 23 May 2003 09:11:35 -0400
> From: Guido van Rossum <guido [at] python>
> Subject: Re: [Python-Dev] Of what use is commands.getstatus()
> To: skip [at] pobox
> Cc: python-dev [at] python
>
> > I was reading the docs for the commands module and noticed getstatus() seems
> > to be completely unrelated to getstatusoutput() and getoutput(). I thought,
> > "I'll correct the docs. They must be wrong." Then I looked at commands.py
> > and saw the docs are correct. It's the function definition which is weird.
> > Of what use is it to return 'ls -ld file'? Based on its name I would have
> > guessed its function was
> >
> > def getoutput(cmd):
> > """Return status of executing cmd in a shell."""
> > return getstatusoutput(cmd)[0]
> >
> > This particular function dates from 1990, so it clearly can't just be
> > deleted, but it seems completely superfluous to me, especially given the
> > existence of os.stat, os.listdir, etc. Should it be deprecated or modified
> > to do (what I think is) the obvious thing?
>
> That whole module wasn't thought out very well. I recently tried to
> use it and found that the strip of the trailing \n on getoutput() is
> also a counterproductive feature. I suggest that someone should
> design a replacement, perhaps to live in shutil, and then we can
> deprecate it. Until then I would leave it alone. Certainly don't
> "fix" it by doing something incompatible.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
>
> --__--__--
>
> Message: 13
> Date: Fri, 23 May 2003 10:06:05 -0400
> From: Guido van Rossum <guido [at] python>
> Subject: Re: [Python-Dev] Descriptor API
> To: =?iso-8859-1?Q?Gon=E7alo_Rodrigues?= <op73418 [at] mail>
> Cc: python-dev [at] python
>
> > I was doing some tricks with metaclasses and descriptors in Python 2.2 and
> > stumbled on the following:
> >
> > >>> class test(object):
> > ... a = property(lambda: 1)
> > ...
> > >>> print test.a
> > <property object at 0x01504D20>
> > >>> print test.a.__set__
> > <method-wrapper object at 0x01517220>
> > >>> print test.a.fset
> > None
> >
> > What this means in practice, is that if I want to test if a
> > descriptor is read-only I have to have two tests: One for custom
> > descriptors, checking that getting __set__ does not barf and another
> > for property, checking that fset returns None.
>
> Why are you interested in knowing whether a descriptor is read-only?
>
> > So, why doesn't getting __set__ raise AttributeError in the above case?
>
> This is a feature. The presence of __set__ (even if it always raises
> AttributeError when *called*) signals this as a "data descriptor".
> The difference between data descriptors and others is that a data
> descriptor can not be overridden by putting something in the instance
> dict; a non-data descriptor can be overridden by assignment to an
> instance attribute, which will store a value in the instance dict.
>
> For example, a method is a non-data descriptor (and the prevailing
> example of such). This means that the following example works:
>
> class C(object):
> def meth(self): return 42
>
> x = C()
> x.meth() # prints 42
> x.meth = lambda: 24
> x.meth() # prints 24
>
> > Is this a bug? If it's not, it sure is a (minor) feature request
> > from my part :-)
>
> Because of the above explanation, the request cannot be granted.
>
> You can test the property's fset attribute however to tell whether a
> 'set' argument was passed to the constructor.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
> --__--__--
>
> Message: 14
> From: =?iso-8859-1?Q?Gon=E7alo_Rodrigues?= <op73418 [at] mail>
> To: <python-dev [at] python>
> Subject: Re: [Python-Dev] Descriptor API
> Date: Fri, 23 May 2003 15:34:14 +0100
>
>
> ----- Original Message -----
> From: "Guido van Rossum" <guido [at] python>
> To: "Gonçalo Rodrigues" <op73418 [at] mail>
> Cc: <python-dev [at] python>
> Sent: Friday, May 23, 2003 3:06 PM
> Subject: Re: [Python-Dev] Descriptor API
>
>
> > > I was doing some tricks with metaclasses and descriptors in Python 2.2
> and
> > > stumbled on the following:
> > >
> > > >>> class test(object):
> > > ... a = property(lambda: 1)
> > > ...
> > > >>> print test.a
> > > <property object at 0x01504D20>
> > > >>> print test.a.__set__
> > > <method-wrapper object at 0x01517220>
> > > >>> print test.a.fset
> > > None
> > >
> > > What this means in practice, is that if I want to test if a
> > > descriptor is read-only I have to have two tests: One for custom
> > > descriptors, checking that getting __set__ does not barf and another
> > > for property, checking that fset returns None.
> >
> > Why are you interested in knowing whether a descriptor is read-only?
> >
>
> Introspection dealing with a metaclass that injected methods in its
> instances depending on a descriptor. In other words, having fun with
> Python's wacky tricks.
>
> > > So, why doesn't getting __set__ raise AttributeError in the above case?
> >
> > This is a feature. The presence of __set__ (even if it always raises
> > AttributeError when *called*) signals this as a "data descriptor".
> > The difference between data descriptors and others is that a data
> > descriptor can not be overridden by putting something in the instance
> > dict; a non-data descriptor can be overridden by assignment to an
> > instance attribute, which will store a value in the instance dict.
> >
> > For example, a method is a non-data descriptor (and the prevailing
> > example of such). This means that the following example works:
> >
> > class C(object):
> > def meth(self): return 42
> >
> > x = C()
> > x.meth() # prints 42
> > x.meth = lambda: 24
> > x.meth() # prints 24
> >
> > > Is this a bug? If it's not, it sure is a (minor) feature request
> > > from my part :-)
> >
> > Because of the above explanation, the request cannot be granted.
> >
>
> Thanks for the reply (and also to P. Eby, btw). I was way off track when I
> sent the email, because it did not occured to me that property was a type
> implementing __get__ and __set__. With this piece of info connecting the
> dots the idea is just plain foolish.
>
> > You can test the property's fset attribute however to tell whether a
> > 'set' argument was passed to the constructor.
> >
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> With my best regards,
> G. Rodrigues
>
>
>
> --__--__--
>
> Message: 15
> Date: Fri, 23 May 2003 10:39:10 -0400
> From: Guido van Rossum <guido [at] python>
> Subject: Re: [Python-Dev] Need advice, maybe support
> To: Christian Tismer <tismer [at] tismer>
> Cc: python-dev [at] python
>
> > > The other is the new style where the PyMethodDef
> > > array is in tp_methods, and is scanned once by PyType_Ready.
> >
> > Right, again. Now, under the hopeful assumption that every
> > sensible extension module that has some types to publish also
> > does this through its module dictionary, I would have the
> > opportunity to cause PyType_Ready being called early enough
> > to modify the method table, before any of its methods is used
> > at all.
>
> Dangerous assumption! It's not inconceivable that a class would
> instantiate some of its own classes as part of its module
> initialization.
>
> > > 3rd party modules that have been around for a while are likely to use
> > > Py_FindMethod. With Py_FindMethod you don't have a convenient way to
> > > store the pointer to the converted table, so it may be better to
> > > simply check your bit in the first array element and then cast to a
> > > PyMethodDef or a PyMethodDefEx array based on what the bit says (you
> > > can safely assume that all elements of an array are the same size :-).
> >
> > Hee hee, yeah. Of course, if there isn't a reliable way to
> > intercept method table access before the first Py_FindMethod
> > call, I could of course modify Py_FindMethod. For instance,
> > a modified, new-style method table might be required to always
> > start with a dummy entry, where the flags word is completely
> > -1, to signal having been converted to new-style.
>
> Why so drastic? You could just set a reserved bit.f
>
> > ...
> >
> > >>If that approach is trustworthy, I also could drop
> > >>the request for these 8 bits.
> > >
> > > Sure. Ah, a bit in the type would work just as well, and
> > > Py_FindMethod *does* have access to the type.
> >
> > You think of the definition in methodobject.c, as it is
> >
> > """
> > /* Find a method in a single method list */
> >
> > PyObject *
> > Py_FindMethod(PyMethodDef *methods, PyObject *self, char *name)
> > """
> >
> > , assuming that self always is not NULL, but representing a valid
> > object with a type, and this type is already referring to the
> > methods table?
>
> Right. There is already code that uses self->ob_type in
> Py_FindMethodInChain(), which is called by Py_FindMethod().
>
> > Except for module objects, this seems to be right. I've run
> > Python against a lot of Python modules, but none seems
> > to call Py_FindMethod with a self parameter of NULL.
>
> I don't think it would be safe to do so.
>
> > If that is true, then I can patch a small couple of
> > C functions to check for the new bit, and if it's not
> > there, re-create the method table in place.
> > This is music to me ears. But...
> >
> > Well, there is a drawback:
> > I *do* need two bits, and I hope you will allow me to add this
> > second bit, as well.
> >
> > The one, first bit, tells me if the source has been compiled
> > with Stackless and its extension stuff. Nullo problemo.
> > I can then in-place modify the method table in a compatible
> > way, or leave it as it is, bny default.
> > But then, this isn't sufficient to set this bit then, like an
> > "everything is fine, now" relief. This is so, since this is *still*
> > an old module, and while its type's method tables have been
> > patched, the type is still not augmented by new slots, like
> > the new tp_call_nr slots (and maybe a bazillion to come, soon).
> > The drawback is, that I cannot simply replace the whole type
> > object, since type objects are not represented as object
> > pointers (like they are now, most of the time, in the dynamic
> > heaptype case), but they are constant struct addresses, where
> > the old C module might be referring to.
> >
> > So, what I think to need is no longer 9 bits, but two of them:
> > One that says "everything great from the beginning", and another
> > one that says "well, ok so far, but this is still an old object".
> >
> > I do think this is the complete story, now.
> > Instead of requiring nine bits, I'm asking for two.
> > But this is just *your options; I also can live with one bit,
> > but then I have to add a special, invalid method table entry
> > that just serves for this purpose.
> > In order to keep my souce code hack to the minimum, I'd really
> > like to ask for the two bits in the typeobject flags.
>
> OK, two bits you shall have. Don't spend them all at once!
>
> > Thanks so much for being so supportive -- chris
>
> Anything to keep ctual stackless support out of the core. :-)
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
>
> --__--__--
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev [at] python
> http://mail.python.org/mailman/listinfo/python-dev
>
>
> End of Python-Dev Digest
>


terry at wayforward

May 28, 2003, 10:32 AM

Post #12 of 26 (818 views)
Permalink
Introduction [In reply to]

I've been lurking for a bit, and now seems like a good time to
introduce myself.

* I build messaging systems for banks, earlier I was CTO of a dot-com.
* I started programming on the TRS-80 and the RCA COSMAC VIP, later
on the Apple ][.
* I am a Java refugee (well, I might still code in Java for pay).
* I'm into formal methods. Translation: I like *talking* about
formal methods, but I never use them myself :-)

I read somewhere that the best way to build big Python callouses was
to write a PEP. Here goes:
http://www.wayforward.net/pycontract/pep-0999.html

Programming by Contract for Python... pre-conditions, post-conditions,
invariants, with all the Eiffel goodness like weakening pre-conditions
and strengthening invariants and post-conditions on inheritance, and
access to old values. All from docstrings, like doctest.

I'm also into handling insane numbers of incoming connections on cheap
boxes: compare Jef Poskanzer's thttpd to Apache. 10000 simultaneous
HTTP connections on a $400 computer just gets me giggling. Stackless
Python intrigues me greatly for the same reason.

I guess that's it for now... Cheers!


pje at telecommunity

May 28, 2003, 11:23 AM

Post #13 of 26 (838 views)
Permalink
Re: Introduction [In reply to]

At 01:32 PM 5/28/03 -0400, Terence Way wrote:

>I read somewhere that the best way to build big Python callouses was
>to write a PEP.

Guess I'll start helping you work on the callouses, then. :)


> Here goes:
> http://www.wayforward.net/pycontract/pep-0999.html

Please don't number a pre-PEP; I believe PEP 1 recomends using 'XXX' until
a PEP number has been assigned by the PEP editors.


>Programming by Contract for Python... pre-conditions, post-conditions,
>invariants, with all the Eiffel goodness like weakening pre-conditions
>and strengthening invariants and post-conditions on inheritance, and
>access to old values. All from docstrings, like doctest.

A number of things aren't clear from your PEP. For example, how would
syntax errors in assertions be handled? How is backward compatibility with
existing docstrings that may use 'inv:' or 'pre:' to specify conditions
informally?

Are you proposing that this be part of Python's core syntax? If so, then
why do it as docstrings? Are you proposing instead that your
implementation be part of the standard library? If so, then where is the
documentation for how a developer enables the behavior?

Also, I didn't find the motivation section convincing. Your answer to "Why
not have several different implementations, or let programmers implement
their own assertions?" isn't actually a justification. If Alice uses some
package to wrap her methods with checks, I can weaken the preconditions in
a subclass, by simply overriding the methods. If I can't do that, then it
is a weakness of the DBC package Alice used, or of Alice's package, not a
weakness of Python.


terry at wayforward

May 28, 2003, 12:37 PM

Post #14 of 26 (820 views)
Permalink
Re: Introduction [In reply to]

On Wednesday, May 28, 2003, at 02:23 PM, Phillip J. Eby wrote:

> Please don't number a pre-PEP; I believe PEP 1 recomends using 'XXX'
> until a PEP number has been assigned by the PEP editors.
>
Ack. Oops. I've sent it off to peps [at] python with the XXX, but
posted
here with the 999.

> A number of things aren't clear from your PEP. For example, how would
> syntax errors in assertions be handled? How is backward compatibility
> with existing docstrings that may use 'inv:' or 'pre:' to specify
> conditions informally?
Um. No thought given to that. My first guess is: syntax errors printed
to standard error, optionally silently ignored, no safety checks
installed
either way. Run-time errors trapped and re-raised as some kind of
ContractViolation::

def read_stuff(input)
"""pre: input.readline"""

would be valid, and the AttributeError would be wrapped inside a
PreconditionViolationError if the ``input`` parameter isn't some
type of input stream.

> Are you proposing that this be part of Python's core syntax? If so,
> then why do it as docstrings? Are you proposing instead that your
> implementation be part of the standard library? If so, then where is
> the documentation for how a developer enables the behavior?
Proposing that some implementation, hopefully mine, be put in the
standard library. I *really* don't think contracts should be part of
the
core syntax: contracts belong in the documentation, and changing all the
doc tools to parse code looking for contract assertions is harder than
building one or two docstring implementations.

self.note(): where *is* the documentation on how to enable the
behavior.

> Also, I didn't find the motivation section convincing. Your answer to
> "Why not have several different implementations, or let programmers
> implement their own assertions?" isn't actually a justification. If
> Alice uses some package to wrap her methods with checks, I can weaken
> the preconditions in a subclass, by simply overriding the methods. If
> I can't do that, then it is a weakness of the DBC package Alice used,
> or of Alice's package, not a weakness of Python.
Consider when Alice's preconditions work, but Bob's do not. Code that
thinks it's calling Alice's code *must not* break when calling Bob's.
Weakening pre-conditions means that Alice's pre-conditions must be
tested as well: and Bob's code is run even if his pre-conditions fail.
The converse is also true: code that understands Bob's pre-conditions
must not fail even if Alice's pre-conditions fail. This is tough to
do with asserts, or with incompatible contract packages.

I haven't made that clear in the PEP or the samples, and it needs to
be clear, because it is the /only/ reason why contracts need to be in
the language/standard runtime.

Excellent points, thanks for taking an interest.


pjones at redhat

May 28, 2003, 1:55 PM

Post #15 of 26 (816 views)
Permalink
Re: Introduction [In reply to]

Hi, I'm Peter. Long time listener, first time caller.

On Wed, 28 May 2003, Terence Way wrote:

> > A number of things aren't clear from your PEP. For example, how would
> > syntax errors in assertions be handled? How is backward compatibility
> > with existing docstrings that may use 'inv:' or 'pre:' to specify
> > conditions informally?
>
> Um. No thought given to that. My first guess is: syntax errors printed
> to standard error, optionally silently ignored, no safety checks
> installed either way.

It seems like either of these methods of coping with legacy docstrings
thwart your basic premises. Unless the well-formed nature of the
contracts are enforced, it seems to be fairly difficult to e.g. randomly
test a function.

What if the docstring fails to parse? I have to be listening to stderr to
know that it didn't work. I then have to parse the message from stderr to
figure out which function didn't work, and finally I have to somehow mark
this function as not compliant, and ignore whatever results I get.

It really seems like you want them either "on" or "off", not "on, but it
might fail in some silent or hard to trap way".

> > Are you proposing that this be part of Python's core syntax? If so,
> > then why do it as docstrings? Are you proposing instead that your
> > implementation be part of the standard library? If so, then where is
> > the documentation for how a developer enables the behavior?
>
> Proposing that some implementation, hopefully mine, be put in the
> standard library. I *really* don't think contracts should be part of
> the core syntax: contracts belong in the documentation, and changing all
> the doc tools to parse code looking for contract assertions is harder
> than building one or two docstring implementations.

The assertion that contracts don't belong in the core seems entirely
seperate from the discussion of their place in docstrings or in real code.

That being said, you still haven't explained *why* contracts belong in
docstrings (or in documentation in general). They are executable code;
why not treat them as such?

> self.note(): where *is* the documentation on how to enable the
> behavior.

I suspect we have to know this before we can know which way is easier.

That being said, I really don't see how these contracts can be meaningful
as part of a docstring without some better mechanism for handling old
docstrings that have been ruled malformed. What's your reasoning against
making them their own kind of block, like "try:"?

--
Peter


patmiller at llnl

May 28, 2003, 2:01 PM

Post #16 of 26 (821 views)
Permalink
Re: Introduction [In reply to]

> http://www.wayforward.net/pycontract/pep-0999.html

I think another issue with using doc strings in this way
is that you are overloading a feature visible to end users.

If I look at the doc string then I would expect to be confused the result:

>>> help(circbuf)
Help on class circbuf in module __main__:

class circbuf
| Methods defined here:
|
| get(self)
| Pull an entry from a non-empty circular buffer.
|
| pre: not self.is_empty()
| post[self.g, self.len]:
| __return__ == self.buf[__old__.self.g]
| self.len == __old__.self.len - 1
| ...

Way to cryptic even for me :-)

I think you could get the same effect by overloading property
so you could make methods "smart" about pre and post conditions
The following is a first quick hack at it...:

class eiffel(property):
"""eiffel(method,precondition,postcondition)

Implement a Eiffel style method that enforces pre-
and post- conditions. I guess you could turn this
on and off if you wanted...

class foo:
def pre(self): assert self.x > 0
def post(self): assert self.x > 0
def increment(self):
self.x += 1
return

increment = eiffel(method,precondition,postcondition)
"""

def __init__(self,method,precondition,postcondition,doc=None):
self.method = method
self.precondition = precondition
self.postcondition = postcondition
super(eiffel,self).__init__(self.__get,None,None,doc)
return


def __get(self,this):
class funny_method:
def __init__(self,this,method,precondition,postcondition):
self.this = this
self.method = method
self.precondition = precondition
self.postcondition = postcondition
return
def __call__(self,*args,**kw):
self.precondition(self.this)
value = self.method(self.this,*args,**kw)
self.postcondition(self.this)
return value

return funny_method(this,self.method,self.precondition,self.postcondition)



class circbuf:
def __init__(self):
self.stack = []
return

def _get_pre(self):
assert not self.is_empty()
return
def _get_post(self):
assert not self.is_empty()
return
def _get(self):
"""Pull an entry from a non-empty circular buffer."""
val = self.stack[-1]
del self.stack[-1]
return val

get = eiffel(_get,_get_pre,_get_post)

def put(self,val):
self.stack.append(val)
return

def is_empty(self):
return len(self.stack) == 0


B = circbuf()
B.put('hello')
print B.get()

# Will bomb...
print B.get()


--
Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller

If you think you can do a thing or think you can't do a thing, you're
right. -- Henry Ford


terry at wayforward

May 29, 2003, 7:53 AM

Post #17 of 26 (824 views)
Permalink
Re: Introduction [In reply to]

On Wednesday, May 28, 2003, at 04:55 PM, Peter Jones wrote:

> What if the docstring fails to parse? I have to be listening to
> stderr to
> know that it didn't work. I then have to parse the message from
> stderr to
> figure out which function didn't work, and finally I have to somehow
> mark
> this function as not compliant, and ignore whatever results I get.
>
I probably would have figured that out too, eventually... :-) More on
this further down, when I talk about how to enable docstring testing.

> That being said, you still haven't explained *why* contracts belong in
> docstrings (or in documentation in general). They are executable code;
> why not treat them as such?
>
Okay, the whole docstring vs syntax thing, and I'm going to quote
liberally from Bertrand Meyer's Object Oriented Software Construction,
1st edition, 7.9 Using Assertions.

There are four main reasons for adding contracts to code:
"""
* Help in writing correct software.
* Documentation aid.
* Debugging tool.
* Support for software fault tolerance.

[...]

The second use is essential in the production of reusable software
elements and, more generally, in organizing the interfaces of
modules in large software systems. Preconditions, postconditions,
and class invariants provide potential clients of a module with
crucial information about the services offered by the module, expressed
in a concise and precise form. No amount of verbose documentation can
replace a set of carefully expressed assertions.
"""

I really like Extreme Programming's cut-to-the-bone approach: there
are only two things worth knowing about the code: *what* it does and
*how* it does it. In XP, what the code does can be inferred from
test cases; how it does it from the source code. And if you can't
read the code, you have no business talking about how the software
does what it does anyway.

With contracts, I want to move the knowledge of *what* the code does
from the test cases back into the programming documentation. It
is merely a bonus feature that this documentation can be executed.

When I was learning Python (um, not too long ago) the epiphany of
what this language was all about hit me when I saw the 'doctest'
module. We're *always* using examples as clear, concise ways to
describe what our code does, but we're all guilty of letting those
examples get out-of-date. Doctest can crawl into our software
deep enough to keep us honest about our documentation. Contracts
extend this so it's not just about the basic sample cases, but
about the entire state space that a function supports... "Here be
dragons" but over there be heap-based priority queues.

>> self.note(): where *is* the documentation on how to enable the
>> behavior.
>
> I suspect we have to know this before we can know which way is easier.
>
Now that I've come out as a doctest fanboy, it should be no surprise
that contracts are enabled like this:
import contracts, mymodule
contracts.checkmod(mymodule)

The checkmod side effect is that all functions within mymodule are
replaced by auto-generated checking functions.

And now I think I'm clear in my own mind about backwards-
compatibility with informal 'pre:' docstrings... a programmer
doesn't run checkmod unless she's sure that all docstring
contracts are valid. Syntax error exceptions will be passed
through to the checkmod caller.

Cheers!


gustav at morpheus

Jun 1, 2003, 9:17 AM

Post #18 of 26 (821 views)
Permalink
Re: Introduction [In reply to]

Terence Way <terry [at] wayforward> writes:

> On Wednesday, May 28, 2003, at 04:55 PM, Peter Jones wrote:
>
>> That being said, you still haven't explained *why* contracts belong
>> in docstrings (or in documentation in general). They are executable
>> code; why not treat them as such?
>>
> Okay, the whole docstring vs syntax thing, and I'm going to quote
> liberally from Bertrand Meyer's Object Oriented Software
> Construction, 1st edition, 7.9 Using Assertions.

Note: I'm *not* a fan of Eiffel-style formal pre and post conditions.
I probably wouldn't use them, personally. But as I could end up
dealing with others' code which uses such a feature if it was added, I
do have an interest. Also, I haven't read the PEP yet (I'm offline
right now).

But I disagree fairly strongly with the idea of having pre/post
conditions in docstrings. That is *not* what docstrings are for.
Docstrings are documentation, not arbitrary metadata.

How about this as a proposal? Attach pre and post conditions to a
functions as function attributes. A silly example:


def f():
pass

def pre():
assert True
f.pre = pre

def post():
assert True
f.post = post

This is a bit wordy, but entirely doable with Python as it stands and
the conditions are syntax checked at definition time, just like they
should be. If pre/post conditions turn out to be so useful that they
deserve a more concise syntax, *that's* the time to propose better
syntax. (One obvious possibility would be to allow more flexibility in
def statements, so that

def f.post():
whatever

becomes possible.)

Paul.
--
This signature intentionally left blank


barry at python

Jun 1, 2003, 3:38 PM

Post #19 of 26 (822 views)
Permalink
Re: Re: Introduction [In reply to]

On Sunday, June 1, 2003, at 12:17 PM, Paul Moore wrote:
> Note: I'm *not* a fan of Eiffel-style formal pre and post conditions.
> I probably wouldn't use them, personally. But as I could end up
> dealing with others' code which uses such a feature if it was added, I
> do have an interest. Also, I haven't read the PEP yet (I'm offline
> right now).
>
> But I disagree fairly strongly with the idea of having pre/post
> conditions in docstrings. That is *not* what docstrings are for.
> Docstrings are documentation, not arbitrary metadata.
>
> How about this as a proposal? Attach pre and post conditions to a
> functions as function attributes. A silly example:
>
>
> def f():
> pass
>
> def pre():
> assert True
> f.pre = pre
>
> def post():
> assert True
> f.post = post
>
> This is a bit wordy, but entirely doable with Python as it stands and
> the conditions are syntax checked at definition time, just like they
> should be. If pre/post conditions turn out to be so useful that they
> deserve a more concise syntax, *that's* the time to propose better
> syntax. (One obvious possibility would be to allow more flexibility in
> def statements, so that
>
> def f.post():
> whatever
>
> becomes possible.)

Although I haven't played with it much, in the back of my mind, I think
descriptors might be the right implementation technique for this. And
the syntax that seems to keep coming up is something like:

def f(x, y) [pre, post]:
foo()

For other reasons, I'm finding that I'd really like to see this syntax
for function decorators embodied in its own PEP.

-Barry


agthorr at barsoom

Jun 1, 2003, 3:59 PM

Post #20 of 26 (823 views)
Permalink
Re: Re: Introduction [In reply to]

On Sun, Jun 01, 2003 at 05:17:10PM +0100, Paul Moore wrote:
> But I disagree fairly strongly with the idea of having pre/post
> conditions in docstrings. That is *not* what docstrings are for.
> Docstrings are documentation, not arbitrary metadata.
>
> How about this as a proposal? Attach pre and post conditions to a
> functions as function attributes. A silly example:

FWIW, I stumbled across an implementation of contracts for Python that
works more or less in the manner you describe:

http://www.nongnu.org/pydbc/

I agree that contracts shouldn't be in docstrings. I confess though,
that is because I know editor indenting and syntax highlighting won't
handle it smoothly ;)

-- Agthorr


python at rcn

Jun 1, 2003, 5:44 PM

Post #21 of 26 (819 views)
Permalink
Re: Re: Introduction [In reply to]

> FWIW, I stumbled across an implementation of contracts for Python that
> works more or less in the manner you describe:

With the new surge of interest in metaclasses, I expect many
competing DBC implementations to come into existence. An early
commitment to one of them may preclude better ideas from
getting a chance.

Another thought is that all of the machinery is in place to enable
a library of cooperating metaclass tools as envisioned by
Forman and Danforth's metaclass book. Any metaclass work
included in the distribution ought to make allowances for
being included side-by-side with threadsafety metaclasses,
logging metaclasses, auto property creators, auto delegators,
and such.

In summary, I prefer that DBC not be PEPed. Instead, let things
grow on SF, the cookbook, the vaults, and private offerings.

'nuff said,


Raymond Hettinger


guido at python

Jun 1, 2003, 6:01 PM

Post #22 of 26 (815 views)
Permalink
Re: Re: Introduction [In reply to]

> With the new surge of interest in metaclasses, I expect many
> competing DBC implementations to come into existence. An early
> commitment to one of them may preclude better ideas from
> getting a chance.
>
> Another thought is that all of the machinery is in place to enable
> a library of cooperating metaclass tools as envisioned by
> Forman and Danforth's metaclass book. Any metaclass work
> included in the distribution ought to make allowances for
> being included side-by-side with threadsafety metaclasses,
> logging metaclasses, auto property creators, auto delegators,
> and such.
>
> In summary, I prefer that DBC not be PEPed. Instead, let things
> grow on SF, the cookbook, the vaults, and private offerings.

Good suggestions! This can be done many ways. Let's find ot which
works best.

--Guido van Rossum (home page: http://www.python.org/~guido/)


terry at wayforward

Jun 1, 2003, 7:19 PM

Post #23 of 26 (815 views)
Permalink
Re: Re: Introduction [In reply to]

On Sunday, June 1, 2003, at 09:01 PM, Guido van Rossum wrote:

>> In summary, I prefer that DBC not be PEPed. Instead, let things
>> grow on SF, the cookbook, the vaults, and private offerings.
>
> Good suggestions! This can be done many ways. Let's find ot which
> works best.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

Sigh...

PEP withdrawn.

I would just like to point out, however, that the challenge with DBC
is *not* how to install contracts, whether that's done with meta-
classes, attributes, whatever. The challenges are 1) dual use as code
and documentation, and 2) correct behavior under inheritance.

The problem with multiple implementations is that they may not work
together under inheritance, ie all super-class invariants might
not be checked, all overridden post-conditions not checked, if the
superclass and the derived class use different implementations.

What I'll do now is:
1. clean up the loose ends on my implementation; and
2. continue to write contracts for the Python standard library. I've
already found 1 bug :-) a full report due soon(ish).

Cheers.


mwh at python

Jun 2, 2003, 2:08 AM

Post #24 of 26 (813 views)
Permalink
Re: Re: Introduction [In reply to]

Barry Warsaw <barry [at] python> writes:

> Although I haven't played with it much, in the back of my mind, I
> think descriptors might be the right implementation technique for
> this. And the syntax that seems to keep coming up is something like:
>
> def f(x, y) [pre, post]:
> foo()
>
> For other reasons, I'm finding that I'd really like to see this syntax
> for function decorators embodied in its own PEP.

Well, I implemented it, so I've been sort of assuming that I'll be
writing the PEP. However if someone else beats me to it, I'm not
going to be offended! I'm not going to get to it in a hurry, TBH.

Cheers,
M.


--
Richard Gabriel was wrong: worse is not better, lying is better.
Languages and systems succeed in the marketplace to the extent that
their proponents lie about what they can do.
-- Tim Bradshaw, comp.lang.lisp


python at rcn

Mar 9, 2007, 12:04 PM

Post #25 of 26 (837 views)
Permalink
Re: Introduction [In reply to]

[Žiga Seilnacht]
>I have just accepted an invitation to become a Python
>developer, so I feel obliged to introduce myself.

Welcome aboard.


Raymond
_______________________________________________
Python-Dev mailing list
Python-Dev [at] python
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

First page Previous page 1 2 Next page Last page  View All Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.