Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Dev

Re: breaking list.append() (was: Python-checkins digest, Vol 1 #370 -8 msgs)

 

 

Python dev RSS feed   Index | Next | Previous | View Threaded


tismer at tismer

Feb 29, 2000, 12:52 PM

Post #1 of 31 (901 views)
Permalink
Re: breaking list.append() (was: Python-checkins digest, Vol 1 #370 -8 msgs)

Greg Stein wrote:
>
> +1 on breaking it now, rather than deferring it Yet Again.
>
> IMO, there has been plenty of warning, and there is plenty of time to
> correct the software.
>
> I'm +0 on adding a warning architecture to Python to support issuing a
> warning/error when .append is called with multiple arguments.

Well, the (bad) effect of this patch is that you cannot run
PythonWin any longer unless Mark either supplies an updated
distribution, or one corrects the two barfing Scintilla
support scripts by hand.
Bad for me, since I'm building Stackless Python against 1.5.2+,
and that means the users will see PythonWin barf when installing SLP.

Adding a warning instead of raising an exception would be nice IMHO,
since the warning could probably contain the file name and line
number to change, and I would leave my users with this easy task.

cheers - chris

--
Christian Tismer :^) <mailto:tismer [at] appliedbiometrics>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaunstr. 26 : *Starship* http://starship.python.net
14163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home


skip at mojam

Feb 29, 2000, 2:46 PM

Post #2 of 31 (877 views)
Permalink
Re: breaking list.append() [In reply to]

Christian> Well, the (bad) effect of this patch is that you cannot run
Christian> PythonWin any longer unless Mark either supplies an updated
Christian> distribution, or one corrects the two barfing Scintilla
Christian> support scripts by hand.

Zope has the multi-arg append problem as well.

It wasn't clear to me where all the problems resided. I've been thinking
mostly about the problem in Python code. Thankfully Tim released
checkappend last night! Even after getting rid of all the cases it found I
was still getting some flaky tracebacks which suggest to me that some C code
makes the same sort of assumptions.

I've never labored under the assumption that append would take multiple
arguments. I was mildly surprised to find an instance of this crop up in my
own code...

Skip Montanaro | http://www.mojam.com/
skip [at] mojam | http://www.musi-cal.com/


gstein at lyra

Feb 29, 2000, 4:47 PM

Post #3 of 31 (879 views)
Permalink
Re: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Christian Tismer wrote:
> Greg Stein wrote:
> > +1 on breaking it now, rather than deferring it Yet Again.
> >
> > IMO, there has been plenty of warning, and there is plenty of time to
> > correct the software.
> >
> > I'm +0 on adding a warning architecture to Python to support issuing a
> > warning/error when .append is called with multiple arguments.
>
> Well, the (bad) effect of this patch is that you cannot run
> PythonWin any longer unless Mark either supplies an updated
> distribution, or one corrects the two barfing Scintilla
> support scripts by hand.

Yes, but there is no reason to assume this won't happen.

Why don't we simply move forward with the assumption that PythonWin and
Scintilla will be updated? If we stand around pointing at all the uses of
append that are incorrect and claim that is why we can't move forward,
then we won't get anywhere. Instead, let's just *MOVE* and see that
software authors update accordingly. It isn't like it is a difficult
change to make. Heck, PythonWin and Scintilla could be updated within the
week and re-released. *WAY* ahead of the 1.6 release.

> Bad for me, since I'm building Stackless Python against 1.5.2+,
> and that means the users will see PythonWin barf when installing SLP.

If you're building a system using an interim release of Python, then I
think you need to take responsibility for that. If you don't want those
people to have problems, then you can back out the list.append change. Or
you can release patches to PythonWin. I don't think the Python world at
large should be hampered because somebody is using an unstable/interim
version of Python. Again: we couldn't move forward.

> Adding a warning instead of raising an exception would be nice IMHO,
> since the warning could probably contain the file name and line
> number to change, and I would leave my users with this easy task.

Yes, this would be nice. But somebody has to take the time to code it up.
The warning won't appear out of nowhere...

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


mhammond at skippinet

Feb 29, 2000, 4:57 PM

Post #4 of 31 (877 views)
Permalink
RE: breaking list.append() [In reply to]

> Why don't we simply move forward with the assumption that PythonWin and
> Scintilla will be updated?

Done :-)

However, I think dropping it now _is_ a little heavy handed. I decided to
do a wider search and found a few in, eg, Sam Rushings calldll based ODBC
package.

Personally, I would much prefer a warning now, and drop it later. _Then_ we
can say we have made enough noise about it. It would only be 2 years ago
that I became aware that this "feature" of append was not a feature at all -
up until then I used it purposely, and habits are sometimes hard to change
:-)

MArk.


gstein at lyra

Feb 29, 2000, 5:12 PM

Post #5 of 31 (875 views)
Permalink
RE: breaking list.append() [In reply to]

On Wed, 1 Mar 2000, Mark Hammond wrote:
> > Why don't we simply move forward with the assumption that PythonWin and
> > Scintilla will be updated?
>
> Done :-)

hehe...

> However, I think dropping it now _is_ a little heavy handed. I decided to
> do a wider search and found a few in, eg, Sam Rushings calldll based ODBC
> package.
>
> Personally, I would much prefer a warning now, and drop it later. _Then_ we
> can say we have made enough noise about it. It would only be 2 years ago
> that I became aware that this "feature" of append was not a feature at all -
> up until then I used it purposely, and habits are sometimes hard to change
> :-)

What's the difference between a warning and an error? If you're running a
program and it suddenly spits out a warning about a misuse of list.append,
I'd certainly see that as "the program did something unexpected; that is
an error."

But this is all moot. Guido has already said that we would be amenable to
a warning/error infrastructure which list.append could use. His
description used some awkward sentences, so I'm not sure (without spending
some brain cycles to parse the email) exactly what his desired defaults
and behavior are. But hey... the possibility is there, and is just waiting
for somebody to code it.

IMO, Guido has left an out for people that are upset with the current
hard-line approach. One of those people just needs to spend a bit of time
coming up with a patch :-)

And yes, Guido is also the Benevolent Dictator and can certainly have his
mind changed, so people can definitely continue pestering him to back away
from the hard-line approach...

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


ping at lfw

Feb 29, 2000, 5:20 PM

Post #6 of 31 (877 views)
Permalink
RE: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Greg Stein wrote:
>
> What's the difference between a warning and an error? If you're running a
> program and it suddenly spits out a warning about a misuse of list.append,
> I'd certainly see that as "the program did something unexpected; that is
> an error."

A big, big difference. Perhaps to one of us, it's the minor inconvenience
of reading the error message and inserting a couple of parentheses in the
appropriate file -- but to the end user, it's the difference between the
program working (albeit noisily) and *not* working. When the program throws
an exception and stops, it is safe to say most users will declare it broken
and give up.

We can't assume that they're going to be able to figure out what to edit
(or be brave enough to try) just by reading the error message... or even
what interpreter flag to give, if errors (rather than warnings) are the
default behaviour.


-- ?!ng


klm at digicool

Feb 29, 2000, 5:37 PM

Post #7 of 31 (877 views)
Permalink
RE: breaking list.append() [In reply to]

On Wed, 1 Mar 2000, Mark Hammond wrote:

> > Why don't we simply move forward with the assumption that PythonWin and
> > Scintilla will be updated?
>
> Done :-)
>
> However, I think dropping it now _is_ a little heavy handed. I decided to
> do a wider search and found a few in, eg, Sam Rushings calldll based ODBC
> package.
>
> Personally, I would much prefer a warning now, and drop it later. _Then_ we
> can say we have made enough noise about it. It would only be 2 years ago
> that I became aware that this "feature" of append was not a feature at all -
> up until then I used it purposely, and habits are sometimes hard to change
> :-)

I agree with mark. Why the sudden rush?? It seems to me to be unfair to
make such a change - one that will break peoples code - without advanced
warning, which typically is handled by a deprecation period. There *are*
going to be people who won't be informed of the change in the short span
of less than a single release. Just because it won't cause you pain isn't
a good reason to disregard the pain of those that will suffer,
particularly when you can do something relatively low-cost to avoid it.

Ken
klm [at] digicool


gstein at lyra

Feb 29, 2000, 5:57 PM

Post #8 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Ken Manheimer wrote:
>...
> I agree with mark. Why the sudden rush?? It seems to me to be unfair to
> make such a change - one that will break peoples code - without advanced
> warning, which typically is handled by a deprecation period. There *are*
> going to be people who won't be informed of the change in the short span
> of less than a single release. Just because it won't cause you pain isn't
> a good reason to disregard the pain of those that will suffer,
> particularly when you can do something relatively low-cost to avoid it.

Sudden rush?!?

Mark said he knew about it for a couple years. Same here. It was a long
while ago that .append()'s semantics were specified to "no longer" accept
multiple arguments.

I see in the HISTORY file, that changes were made to Python 1.4 (October,
1996) to avoid calling append() with multiple arguments.

So, that is over three years that append() has had multiple-args
deprecated. There was probably discussion even before that, but I can't
seem to find something to quote. Seems like plenty of time -- far from
rushed.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


klm at digicool

Feb 29, 2000, 6:02 PM

Post #9 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Greg Stein wrote:

> On Tue, 29 Feb 2000, Ken Manheimer wrote:
> >...
> > I agree with mark. Why the sudden rush?? It seems to me to be unfair to
> > make such a change - one that will break peoples code - without advanced
> > warning, which typically is handled by a deprecation period. There *are*
> > going to be people who won't be informed of the change in the short span
> > of less than a single release. Just because it won't cause you pain isn't
> > a good reason to disregard the pain of those that will suffer,
> > particularly when you can do something relatively low-cost to avoid it.
>
> Sudden rush?!?
>
> Mark said he knew about it for a couple years. Same here. It was a long
> while ago that .append()'s semantics were specified to "no longer" accept
> multiple arguments.
>
> I see in the HISTORY file, that changes were made to Python 1.4 (October,
> 1996) to avoid calling append() with multiple arguments.
>
> So, that is over three years that append() has had multiple-args
> deprecated. There was probably discussion even before that, but I can't
> seem to find something to quote. Seems like plenty of time -- far from
> rushed.

None the less, for those practicing it, the incorrectness of it will be
fresh news. I would be less sympathetic with them if there was recent
warning, eg, the schedule for changing it in the next release was part of
the current release. But if you tell somebody you're going to change
something, and then don't for a few years, you probably need to renew the
warning before you make the change. Don't you think so? Why not?

Ken
klm [at] digicool


paul at prescod

Feb 29, 2000, 7:56 PM

Post #10 of 31 (877 views)
Permalink
Re: breaking list.append() [In reply to]

Software configuration management is HARD. Every sudden backwards
incompatible change (warranted or not) makes it harder. Mutli-arg append
is not hurting anyone as much as a sudden change to it would. It would
be better to leave append() alone and publicize its near-term removal
rather than cause random, part-time supported modules to stop working
because their programmers may be too busy to update them right now.

So no, I'm not stepping up to do it. But I'm also saying that the better
"lazy" option is to put something in a prominent place in the
documentation and otherwise leave it alone.

<aside>
As far as I am concerned, a formal warning-based deprecation mechanism
is necessary for Python's continued evolution. Perhaps we can even
expose the deprecation flag to the programmer so we can say:

if deprecation:
print "This module isn't supported anymore."

if deprecation:
print "Use method FooEx instead."

If we had a deprecation mechanism, maybe introducing new keywords would
not be quite so painful. Version x deprecates, version y adds the
keyword. Mayhap we should also deprecate implicit truncating integral
division while we are at it...
</aside>

--
Paul Prescod - ISOGEN Consulting Engineer speaking for himself
"The calculus and the rich body of mathematical analysis to which it
gave rise made modern science possible, but it was the algorithm that
made possible the modern world."
- from "Advent of the Algorithm" David Berlinski
http://www.opengroup.com/mabooks/015/0151003386.shtml


guido at python

Feb 29, 2000, 9:11 PM

Post #11 of 31 (876 views)
Permalink
Re: breaking list.append() [In reply to]

> Software configuration management is HARD. Every sudden backwards
> incompatible change (warranted or not) makes it harder. Mutli-arg append
> is not hurting anyone as much as a sudden change to it would. It would
> be better to leave append() alone and publicize its near-term removal
> rather than cause random, part-time supported modules to stop working
> because their programmers may be too busy to update them right now.

I'm tired of this rhetoric. It's not like I'm changing existing
Python installations retroactively. I'm planning to release a new
version of Python which no longer supports certain long-obsolete and
undocumented behavior. If you maintain a non-core Python module, you
should test it against the new release and fix anything that comes up.
This is why we have an alpha and beta test cycle and even before that
the CVS version. If you are a Python user who depends on a 3rd party
module, you need to find out whether the new version is compatible
with the 3rd party code you are using, or whether there's a newer
version available that solves the incompatibility.

There are people who still run Python 1.4 (really!) because they
haven't upgraded. I don't have a problem with that -- they don't get
much support, but it's their choice, and they may not need the new
features introduced since then. I expect that lots of people won't
upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the
other modules/packages they need are compatible with 1.6. Multi-arg
append probably won't be the only reason why e.g. Digital Creations
may need to release an update to Zope for Python 1.6. Zope comes with
its own version of Python anyway, so they have control over when they
make the switch.

--Guido van Rossum (home page: http://www.python.org/~guido/)


gstein at lyra

Mar 1, 2000, 12:51 AM

Post #12 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Ken Manheimer wrote:
>...
> None the less, for those practicing it, the incorrectness of it will be
> fresh news. I would be less sympathetic with them if there was recent
> warning, eg, the schedule for changing it in the next release was part of
> the current release. But if you tell somebody you're going to change
> something, and then don't for a few years, you probably need to renew the
> warning before you make the change. Don't you think so? Why not?

I agree.

Note that Guido posted a note to c.l.py on Monday. I believe that meets
your notification criteria.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


gstein at lyra

Mar 1, 2000, 1:10 AM

Post #13 of 31 (875 views)
Permalink
Re: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Guido van Rossum wrote:
> I'm tired of this rhetoric. It's not like I'm changing existing
> Python installations retroactively. I'm planning to release a new
> version of Python which no longer supports certain long-obsolete and
> undocumented behavior. If you maintain a non-core Python module, you
> should test it against the new release and fix anything that comes up.
> This is why we have an alpha and beta test cycle and even before that
> the CVS version. If you are a Python user who depends on a 3rd party
> module, you need to find out whether the new version is compatible
> with the 3rd party code you are using, or whether there's a newer
> version available that solves the incompatibility.
>
> There are people who still run Python 1.4 (really!) because they
> haven't upgraded. I don't have a problem with that -- they don't get
> much support, but it's their choice, and they may not need the new
> features introduced since then. I expect that lots of people won't
> upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the
> other modules/packages they need are compatible with 1.6. Multi-arg
> append probably won't be the only reason why e.g. Digital Creations
> may need to release an update to Zope for Python 1.6. Zope comes with
> its own version of Python anyway, so they have control over when they
> make the switch.

I wholeheartedly support his approach. Just ask Mark Hammond :-) how many
times I've said "let's change the code to make it Right; people aren't
required to upgrade [and break their code]."

Of course, his counter is that people need to upgrade to fix other,
unrelated problems. So I relax and try again later :-). But I still
maintain that they can independently grab the specific fixes and leave the
other changes we make.

Maybe it is grey, but I think this change is quite fine. Especially given
Tim's tool.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


tim_one at email

Mar 1, 2000, 1:22 AM

Post #14 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

[Greg Stein]
> ...
> Maybe it is grey, but I think this change is quite fine. Especially given
> Tim's tool.

What the heck does Tim's one-eyed trouser snake have to do with this? I
know *it* likes to think it's the measure of all things, but, frankly, my
tool barely affects the world at all a mere two feet beyond its base <wink>.

tim-and-his-tool-think-the-change-is-a-mixed-thing-but-on-balance-
the-best-thing-ly y'rs - tim


effbot at telia

Mar 1, 2000, 1:40 AM

Post #15 of 31 (876 views)
Permalink
Re: breaking list.append() [In reply to]

Greg Stein wrote:
> Note that Guido posted a note to c.l.py on Monday. I believe that meets
> your notification criteria.

ahem. do you seriously believe that everyone in the
Python universe reads comp.lang.python?

afaik, most Python programmers don't.

...

so as far as I'm concerned, this was officially deprecated
with Guido's post. afaik, no official python documentation
has explicitly mentioned this (and the fact that it doesn't
explicitly allow it doesn't really matter, since the docs don't
explicitly allow the x[a, b, c] syntax either. both work in
1.5.2).

has anyone checked the recent crop of Python books,
btw? the eff-bot guide uses old syntax in two examples
out of 320. how about the others?

...

sigh. running checkappend over a 50k LOC application, I
just realized that it doesn't catch a very common append
pydiom.

how fun. even though 99% of all append calls are "legal",
this "minor" change will break every single application and
library we have :-(

oh, wait. xmlrpclib isn't affected. always something!

</F>


gstein at lyra

Mar 1, 2000, 1:43 AM

Post #16 of 31 (875 views)
Permalink
Re: breaking list.append() [In reply to]

On Wed, 1 Mar 2000, Fredrik Lundh wrote:
> Greg Stein wrote:
> > Note that Guido posted a note to c.l.py on Monday. I believe that meets
> > your notification criteria.
>
> ahem. do you seriously believe that everyone in the
> Python universe reads comp.lang.python?
>
> afaik, most Python programmers don't.

Now you're simply taking my comments out of context. Not a proper thing to
do. Ken said that he wanted notification along certain guidelines. I said
that I believed Guido's post did just that. Period.

Personally, I think it is fine. I also think that a CHANGES file that
arrives with 1.6 that points out the incompatibility is also fine.

>...
> sigh. running checkappend over a 50k LOC application, I
> just realized that it doesn't catch a very common append
> pydiom.

And which is that? Care to help out? Maybe just a little bit? Or do you
just want to talk about how bad this change is? :-(

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


gstein at lyra

Mar 1, 2000, 2:01 AM

Post #17 of 31 (877 views)
Permalink
RE: breaking list.append() [In reply to]

On Wed, 1 Mar 2000, Tim Peters wrote:
> [Greg Stein]
> > ...
> > Maybe it is grey, but I think this change is quite fine. Especially given
> > Tim's tool.
>
> What the heck does Tim's one-eyed trouser snake have to do with this? I
> know *it* likes to think it's the measure of all things, but, frankly, my
> tool barely affects the world at all a mere two feet beyond its base <wink>.
>
> tim-and-his-tool-think-the-change-is-a-mixed-thing-but-on-balance-
> the-best-thing-ly y'rs - tim

Heh. Now how is one supposed to respond to *that* ??!

All right. Fine. +3 cool points go to Tim.

:-)

--
Greg Stein, http://www.lyra.org/


effbot at telia

Mar 1, 2000, 2:13 AM

Post #18 of 31 (876 views)
Permalink
Re: breaking list.append() [In reply to]

Greg Stein <gstein [at] lyra> wrote:
> On Wed, 1 Mar 2000, Fredrik Lundh wrote:
> > Greg Stein wrote:
> > > Note that Guido posted a note to c.l.py on Monday. I believe that meets
> > > your notification criteria.
> >
> > ahem. do you seriously believe that everyone in the
> > Python universe reads comp.lang.python?
> >
> > afaik, most Python programmers don't.
>
> Now you're simply taking my comments out of context. Not a proper thing to
> do. Ken said that he wanted notification along certain guidelines. I said
> that I believed Guido's post did just that. Period.

my point was that most Python programmers won't
see that notification. when these people download
1.6 final and find that all theirs apps just broke, they
probably won't be happy with a pointer to dejanews.

> And which is that? Care to help out? Maybe just a little bit?

this rather common pydiom:

append = list.append
for x in something:
append(...)

it's used a lot where performance matters.

> Or do you just want to talk about how bad this change is? :-(

yes, I think it's bad. I've been using Python since 1.2,
and no other change has had the same consequences
(wrt. time/money required to fix it)

call me a crappy programmer if you want, but I'm sure
there are others out there who are nearly as bad. and
lots of them won't be aware of this change until some-
one upgrades the python interpreter on their server.

</F>


tim_one at email

Mar 1, 2000, 3:06 AM

Post #19 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

[/F]
> ...
> so as far as I'm concerned, this was officially deprecated
> with Guido's post. afaik, no official python documentation
> has explicitly mentioned this (and the fact that it doesn't
> explicitly allow it doesn't really matter, since the docs don't
> explicitly allow the x[a, b, c] syntax either. both work in
> 1.5.2).

The "Subscriptions" section of the Reference Manual explicitly allows for

dict[a, b, c]

and explicitly does not allow for

sequence[a, b, c]

The "Mapping Types" section of the Library Ref does not explicitly allow for
it, though, and if you read it as implicitly allowing for it (based on the
Reference Manual's clarification of "key" syntax), you would also have to
read the Library Ref as allowing for

dict.has_key(a, b, c)

Which 1.5.2 does allow, but which Guido very recently patched to treat as a
syntax error.

> ...
> sigh. running checkappend over a 50k LOC application, I
> just realized that it doesn't catch a very common append
> pydiom.

[And, later, after prodding by GregS]

> this rather common pydiom:
>
> append = list.append
> for x in something:
> append(...)

This limitation was pointed out in checkappend's module docstring. Doesn't
make it any easier for you to swallow, but I needed to point out that you
didn't *have* to stumble into this the hard way <wink>.

> how fun. even though 99% of all append calls are "legal",
> this "minor" change will break every single application and
> library we have :-(
>
> oh, wait. xmlrpclib isn't affected. always something!

What would you like to do, then? The code will be at least as broken a year
from now, and probably more so -- unless you fix it. So this sounds like an
indirect argument for never changing Python's behavior here. Frankly, I
expect you could fix the 50K LOC in less time than it took me to write this
naggy response <0.50K wink>.

embrace-change-ly y'rs - tim


tim_one at email

Mar 1, 2000, 3:31 AM

Post #20 of 31 (877 views)
Permalink
RE: breaking list.append() [In reply to]

[Tim. needing sleep]
> dict.has_key(a, b, c)
>
> Which 1.5.2 does allow, but which Guido very recently patched to
> treat as a syntax error.

No, a runtime error. haskeynanny.py, anyone?

not-me-ly y'rs - tim


mal at lemburg

Mar 1, 2000, 3:40 AM

Post #21 of 31 (876 views)
Permalink
Re: breaking list.append() [In reply to]

Fredrik Lundh wrote:
>
> Greg Stein <gstein [at] lyra> wrote:
> > On Wed, 1 Mar 2000, Fredrik Lundh wrote:
> > > Greg Stein wrote:
> > > > Note that Guido posted a note to c.l.py on Monday. I believe that meets
> > > > your notification criteria.
> > >
> > > ahem. do you seriously believe that everyone in the
> > > Python universe reads comp.lang.python?
> > >
> > > afaik, most Python programmers don't.
> >
> > Now you're simply taking my comments out of context. Not a proper thing to
> > do. Ken said that he wanted notification along certain guidelines. I said
> > that I believed Guido's post did just that. Period.
>
> my point was that most Python programmers won't
> see that notification. when these people download
> 1.6 final and find that all theirs apps just broke, they
> probably won't be happy with a pointer to dejanews.

Dito. Anyone remember the str(2L) == '2' change, BTW ?
That one will cost lots of money in case someone implemented
an eShop using the common str(2L)[:-1] idiom...

There will need to be a big warning sign somewhere that
people see *before* finding the download link. (IMHO, anyways.)

> > And which is that? Care to help out? Maybe just a little bit?
>
> this rather common pydiom:
>
> append = list.append
> for x in something:
> append(...)
>
> it's used a lot where performance matters.

Same here. checkappend.py doesn't find these (a great tool BTW,
thanks Tim; I noticed that it leaks memory badly though).

> > Or do you just want to talk about how bad this change is? :-(
>
> yes, I think it's bad. I've been using Python since 1.2,
> and no other change has had the same consequences
> (wrt. time/money required to fix it)
>
> call me a crappy programmer if you want, but I'm sure
> there are others out there who are nearly as bad. and
> lots of them won't be aware of this change until some-
> one upgrades the python interpreter on their server.

--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/


fredrik at pythonware

Mar 1, 2000, 4:14 AM

Post #22 of 31 (877 views)
Permalink
Re: breaking list.append() [In reply to]

Tim Peters wrote:
> The "Subscriptions" section of the Reference Manual explicitly allows for
>
> dict[a, b, c]
>
> and explicitly does not allow for
>
> sequence[a, b, c]

I'd thought we'd agreed that nobody reads the
reference manual ;-)

> What would you like to do, then?

more time to fix it, perhaps? it's surely a minor
code change, but fixing it can be harder than
you think (just witness Gerrit's bogus patches)

after all, python might be free, but more and more
people are investing lots of money in using it [1].

> The code will be at least as broken a year
> from now, and probably more so -- unless you fix it.

sure. we've already started. but it's a lot of work,
and it's quite likely that it will take a while until we
can be 100% confident that all the changes are pro-
perly done.

(not all software have a 100% complete test suite that
simply says "yes, this works" or "no, it doesn't")

</F>

1) fwiw, some poor soul over here posted a short note
to the pythonworks mailing, mentioning that we've now
fixed the price. a major flamewar erupted, and my mail-
box is now full of mail from unknowns telling me that I
must be a complete moron that doesn't understand that
Python is just a toy system, which everyone uses just be-
cause they cannot afford anything better...


akuchlin at mems-exchange

Mar 1, 2000, 7:59 AM

Post #23 of 31 (876 views)
Permalink
Re: breaking list.append() [In reply to]

Fredrik Lundh writes:
>yes, I think it's bad. I've been using Python since 1.2,
>and no other change has had the same consequences
>(wrt. time/money required to fix it)

There are more things in 1.6 that might require fixing existing code:
str(2L) returning '2', the int/long changes, the Unicode changes, and
if it gets added, garbage collection -- and bugs caused by those
changes might not be catchable by a nanny. IMHO it's too early to
point at the .append() change as breaking too much existing code;
there may be changes that break a lot more. I'd wait and see what
happens once the 1.6 alphas become available; if c.l.p is filled with
shrieks and groans, GvR might decide to back the offending change out.
(Or he might not...)

--
A.M. Kuchling http://starship.python.net/crew/amk/
I have no skills with machines. I fear them, and because I cannot help
attributing human qualities to them, I suspect that they hate me and will kill
me if they can.
-- Robertson Davies, "Reading"


klm at digicool

Mar 1, 2000, 8:37 AM

Post #24 of 31 (876 views)
Permalink
RE: breaking list.append() [In reply to]

On Tue, 29 Feb 2000, Greg Stein wrote:

> On Tue, 29 Feb 2000, Ken Manheimer wrote:
> >...
> > None the less, for those practicing it, the incorrectness of it will be
> > fresh news. I would be less sympathetic with them if there was recent
> > warning, eg, the schedule for changing it in the next release was part of
> > the current release. But if you tell somebody you're going to change
> > something, and then don't for a few years, you probably need to renew the
> > warning before you make the change. Don't you think so? Why not?
>
> I agree.
>
> Note that Guido posted a note to c.l.py on Monday. I believe that meets
> your notification criteria.

Actually, by "part of the current release", i meant having the
deprecation/impending-deletion warning in the release notes for the
release before the one where the deletion happens - saying it's being
deprecated now, will be deleted next time around.

Ken
klm [at] digicool

I mean, you tell one guy it's blue. He tells his guy it's brown, and it
lands on the page sorta purple. Wavy Gravy/Hugh Romney


effbot at telia

Mar 1, 2000, 12:20 PM

Post #25 of 31 (880 views)
Permalink
Re: breaking list.append() [In reply to]

Andrew M. Kuchling wrote:
> There are more things in 1.6 that might require fixing existing code:
> str(2L) returning '2', the int/long changes, the Unicode changes, and
> if it gets added, garbage collection -- and bugs caused by those
> changes might not be catchable by a nanny.

hey, you make it sound like "1.6" should really be "2.0" ;-)

</F>


tim_one at email

Mar 1, 2000, 10:08 PM

Post #26 of 31 (184 views)
Permalink
RE: breaking list.append() [In reply to]

[/F]
> append = list.append
> for x in something:
> append(...)

[M.-A. Lemburg]
> Same here. checkappend.py doesn't find these

As detailed in a c.l.py posting, I have yet to find a single instance of
this actually called with multiple arguments. Pointing out that it's
*possible* isn't the same as demonstrating it's an actual problem. I'm
quite willing to believe that it is, but haven't yet seen evidence of it.
For whatever reason, people seem much (and, in my experience so far,
infinitely <wink>) more prone to make the

list.append(1, 2, 3)

error than the

maybethisisanappend(1, 2, 3)

error.

> (a great tool BTW, thanks Tim; I noticed that it leaks memory badly
> though).

Which Python? Which OS? How do you know? What were you running it over?

Using 1.5.2 under Win95, according to wintop, & over the whole CVS tree, the
total (code + data) virtual memory allocated to it peaked at about 2Mb a few
seconds into the run, and actually decreased as time went on. So, akin to
the bound method multi-argument append problem, the "checkappend leak
problem" is something I simply have no reason to believe <wink>. Check your
claim again? checkappend.py itself obviously creates no cycles or holds on
to any state across files, so if you're seeing a leak it must be a bug in
some other part of the version of Python + std libraries you're using.
Maybe a new 1.6 bug? Something you did while adding Unicode? Etc. Tell us
what you were running.

Has anyone else seen a leak?


mal at lemburg

Mar 2, 2000, 12:42 AM

Post #27 of 31 (183 views)
Permalink
Re: breaking list.append() [In reply to]

Tim Peters wrote:
>
> [/F]
> > append = list.append
> > for x in something:
> > append(...)
>
> [M.-A. Lemburg]
> > Same here. checkappend.py doesn't find these
>
> As detailed in a c.l.py posting, I have yet to find a single instance of
> this actually called with multiple arguments. Pointing out that it's
> *possible* isn't the same as demonstrating it's an actual problem. I'm
> quite willing to believe that it is, but haven't yet seen evidence of it.

Haven't had time to check this yet, but I'm pretty sure
there are some instances of this idiom in my code. Note that
I did in fact code like this on purpose: it saves a tuple
construction for every append, which can make a difference
in tight loops...

> For whatever reason, people seem much (and, in my experience so far,
> infinitely <wink>) more prone to make the
>
> list.append(1, 2, 3)
>
> error than the
>
> maybethisisanappend(1, 2, 3)
>
> error.

Of course... still there are hidden instances of the problem
which are yet to be revealed. For my own code the siutation
is even worse, since I sometimes did:

add = list.append
for x in y:
add(x,1,2)

> > (a great tool BTW, thanks Tim; I noticed that it leaks memory badly
> > though).
>
> Which Python? Which OS? How do you know? What were you running it over?

That's Python 1.5 on Linux2. I let the script run over
a large lib directory and my projects directory. In the
projects directory the script consumed as much as 240MB
of process size.

> Using 1.5.2 under Win95, according to wintop, & over the whole CVS tree, the
> total (code + data) virtual memory allocated to it peaked at about 2Mb a few
> seconds into the run, and actually decreased as time went on. So, akin to
> the bound method multi-argument append problem, the "checkappend leak
> problem" is something I simply have no reason to believe <wink>. Check your
> claim again? checkappend.py itself obviously creates no cycles or holds on
> to any state across files, so if you're seeing a leak it must be a bug in
> some other part of the version of Python + std libraries you're using.
> Maybe a new 1.6 bug? Something you did while adding Unicode? Etc. Tell us
> what you were running.

I'll try the same thing again using Python1.5.2 and the CVS version.

--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/


mal at lemburg

Mar 2, 2000, 12:46 AM

Post #28 of 31 (185 views)
Permalink
Re: breaking list.append() [In reply to]

"M.-A. Lemburg" wrote:
>
> > > (a great tool BTW, thanks Tim; I noticed that it leaks memory badly
> > > though).
> >
> > Which Python? Which OS? How do you know? What were you running it over?
>
> That's Python 1.5 on Linux2. I let the script run over
> a large lib directory and my projects directory. In the
> projects directory the script consumed as much as 240MB
> of process size.
>
> > Using 1.5.2 under Win95, according to wintop, & over the whole CVS tree, the
> > total (code + data) virtual memory allocated to it peaked at about 2Mb a few
> > seconds into the run, and actually decreased as time went on. So, akin to
> > the bound method multi-argument append problem, the "checkappend leak
> > problem" is something I simply have no reason to believe <wink>. Check your
> > claim again? checkappend.py itself obviously creates no cycles or holds on
> > to any state across files, so if you're seeing a leak it must be a bug in
> > some other part of the version of Python + std libraries you're using.
> > Maybe a new 1.6 bug? Something you did while adding Unicode? Etc. Tell us
> > what you were running.
>
> I'll try the same thing again using Python1.5.2 and the CVS version.

Using the Unicode patched CVS version there's no leak anymore.
Couldn't find a 1.5.2 version on my machine... I'll build one
later.

--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/


paul at prescod

Mar 3, 2000, 6:04 PM

Post #29 of 31 (183 views)
Permalink
Re: breaking list.append() [In reply to]

Guido van Rossum wrote:
>
> ..
> Multi-arg
> append probably won't be the only reason why e.g. Digital Creations
> may need to release an update to Zope for Python 1.6. Zope comes with
> its own version of Python anyway, so they have control over when they
> make the switch.

My concernc is when I want to build an application with a module that
only works with Python 1.5.2 and another one that only works with Python
1.6. If we can avoid that situation by making 1.6 compatible with 1.5.2.
we should. By the time 1.7 comes around I will accept that everyone has
had enough time to update their modules. Remember that many module
authors are just part time volunteers. They may only use Python every
few months when they get a spare weekend!

I really hope that Andrew is wrong when he predicts that there may be
lots of different places where Python 1.6 breaks code! I'm in favor of
being a total jerk when it comes to Py3K but Python has been pretty
conservative thus far.

Could someone remind in one sentence what the downside is for treating
this as a warning condition as Java does with its deprecated features?
Then the CP4E people don't get into bad habits and those same CP4E
people trying to use older modules don't run into frustrating runtime
errors. Do it for the CP4E people! (how's that for rhetoric)
--
Paul Prescod - ISOGEN Consulting Engineer speaking for himself
"We still do not know why mathematics is true and whether it is
certain. But we know what we do not know in an immeasurably richer way
than we did. And learning this has been a remarkable achievement,
among the greatest and least known of the modern era."
- from "Advent of the Algorithm" David Berlinski
http://www.opengroup.com/mabooks/015/0151003386.shtml


tim_one at email

Mar 4, 2000, 8:24 PM

Post #30 of 31 (187 views)
Permalink
RE: breaking list.append() [In reply to]

Just noting that two instances of this were found in Zope.

[/F]
> append = list.append
> for x in something:
> append(...)

[Tim]
> As detailed in a c.l.py posting, I have yet to find a single instance of
> this actually called with multiple arguments. Pointing out that it's
> *possible* isn't the same as demonstrating it's an actual problem. I'm
> quite willing to believe that it is, but haven't yet seen evidence of it.


tim_one at email

Mar 6, 2000, 12:13 AM

Post #31 of 31 (184 views)
Permalink
RE: breaking list.append() [In reply to]

[Paul Prescod]
> ...
> Could someone remind in one sentence what the downside is for treating
> this as a warning condition as Java does with its deprecated features?

Simply the lack of anything to build on: Python has no sort of runtime
warning system now, and nobody has volunteered to create one. If you do
<wink>, remember that stdout & stderr may go to the bit bucket in a GUI app.

The bit about dropping the "L" suffix on longs seems unwarnable-about in any
case (short of warning every time anyone uses long()).

remember-that-you-asked-for-the-problems-not-for-solutions<wink>-ly y'rs
- tim

Python dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.