Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Perl: porters

[perl #114020] given should localize topic, not lexicalize topic

 

 

Perl porters RSS feed   Index | Next | Previous | View Threaded


perlbug-followup at perl

Jul 6, 2012, 4:07 PM

Post #1 of 14 (198 views)
Permalink
[perl #114020] given should localize topic, not lexicalize topic

# New Ticket Created by Ricardo SIGNES
# Please include the string: [perl #114020]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=114020 >



This is a bug report for perl from rjbs [at] cpan,
generated with the help of perlbug 1.39 running under perl 5.16.0.


-----------------------------------------------------------------
[Please describe your issue here]

perl -E 'given (10) { say $::_; say $_ }'

Prints "\n10\n"

It should print "10\n10\n"

Lexical topicalization by given is the cause of nearly endless grief and
very little, if any, benefit.


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=core
severity=medium
---
Site configuration information for perl 5.16.0:

Configured by rjbs at Tue May 22 08:24:33 EDT 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:

Platform:
osname=linux, osvers=2.6.39-linode33, archname=i686-linux
uname='linux cancer 2.6.39-linode33 #5 smp sun may 29 02:01:17 utc 2011 i686 gnulinux '
config_args='-de -Dprefix=/home/rjbs/perl5/perlbrew/perls/16.0'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.6.3', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib -L/opt/local/lib'
libpth=/usr/local/lib /opt/local/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib /usr/lib
libs=-lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.13'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -L/opt/local/lib -fstack-protector'

Locally applied patches:


---
@INC for perl 5.16.0:
/home/rjbs/.perlbrew/libs/16.0 [at] st/lib/perl5/i686-linux
/home/rjbs/.perlbrew/libs/16.0 [at] st/lib/perl5/i686-linux
/home/rjbs/.perlbrew/libs/16.0 [at] st/lib/perl5
/home/rjbs/perl5/perlbrew/perls/16.0/lib/site_perl/5.16.0/i686-linux
/home/rjbs/perl5/perlbrew/perls/16.0/lib/site_perl/5.16.0
/home/rjbs/perl5/perlbrew/perls/16.0/lib/5.16.0/i686-linux
/home/rjbs/perl5/perlbrew/perls/16.0/lib/5.16.0
.

---
Environment for perl 5.16.0:
HOME=/home/rjbs
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/rjbs/.perlbrew/libs/16.0 [at] st/bin:/home/rjbs/perl5/perlbrew/bin:/home/rjbs/perl5/perlbrew/perls/16.0/bin:/home/rjbs/bin:/usr/local/bin:/home/rjbs/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/opt/local/bin:/opt/local/bin
PERL5LIB=/home/rjbs/.perlbrew/libs/16.0 [at] st/lib/perl5/i686-linux:/home/rjbs/.perlbrew/libs/16.0 [at] st/lib/perl5
PERLBREW_BASHRC_VERSION=0.43
PERLBREW_HOME=/home/rjbs/.perlbrew
PERLBREW_LIB=std
PERLBREW_MANPATH=/home/rjbs/.perlbrew/libs/16.0 [at] st/man:/home/rjbs/perl5/perlbrew/perls/16.0/man
PERLBREW_PATH=/home/rjbs/.perlbrew/libs/16.0 [at] st/bin:/home/rjbs/perl5/perlbrew/bin:/home/rjbs/perl5/perlbrew/perls/16.0/bin
PERLBREW_PERL=16.0
PERLBREW_ROOT=/home/rjbs/perl5/perlbrew
PERLBREW_VERSION=0.43
PERL_AUTOINSTALL=--skipdeps
PERL_BADLANG (unset)
PERL_LOCAL_LIB_ROOT=/home/rjbs/.perlbrew/libs/16.0 [at] st
PERL_MB_OPT=--install_base /home/rjbs/.perlbrew/libs/16.0 [at] st
PERL_MM_OPT=INSTALL_BASE=/home/rjbs/.perlbrew/libs/16.0 [at] st
SHELL=zsh


doy at tozt

Jul 6, 2012, 4:09 PM

Post #2 of 14 (192 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Fri, Jul 06, 2012 at 04:07:30PM -0700, Ricardo SIGNES wrote:
> perl -E 'given (10) { say $::_; say $_ }'
>
> Prints "\n10\n"
>
> It should print "10\n10\n"
>
> Lexical topicalization by given is the cause of nearly endless grief and
> very little, if any, benefit.

Definite +1.

-doy


perlbug-followup at perl

Jul 6, 2012, 4:14 PM

Post #3 of 14 (195 views)
Permalink
[perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Fri Jul 06 16:07:30 2012, rjbs wrote:
>
> This is a bug report for perl from rjbs [at] cpan,
> generated with the help of perlbug 1.39 running under perl 5.16.0.
>
>
> -----------------------------------------------------------------
> [Please describe your issue here]
>
> perl -E 'given (10) { say $::_; say $_ }'
>
> Prints "\n10\n"
>
> It should print "10\n10\n"
>
> Lexical topicalization by given is the cause of nearly endless grief
> and
> very little, if any, benefit.

Does this mean I can merge my patch now (which breaks none of CPAN)?

BTW, this is a duplicate of #90018 and #53186.

--

Father Chrysostomos


---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=114020


aaron at priven

Jul 6, 2012, 11:11 PM

Post #4 of 14 (188 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Jul 6, 2012, at 4:07 PM, Ricardo SIGNES (via RT) wrote:
> Lexical topicalization by given is the cause of nearly endless grief and
> very little, if any, benefit.


Once upon a time I was innocently reading perldelta and came across this new lexical $_. It seemed like a great idea, for the same reason that any other lexical variable is a good idea. I started using "my $_" all over the place, because it just seemed like the right thing to do, having read "Coping with Scoping" and similar things.

And then I ran into problems with List::Util functions, and the problem I described here: http://perlmonks.org/?node_id=958820 , and I realized lexical $_ had more or less insoluble problems and stopped using it.

But that's just me. I don't mean this rhetorically -- If lexical $_ is problematic enough that it's worth changing given, is it problematic enough that it should be discouraged in general -- at least by indicating some of its problems in perlvar?


doy at tozt

Jul 7, 2012, 6:13 AM

Post #5 of 14 (186 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Fri, Jul 06, 2012 at 11:11:09PM -0700, Aaron Priven wrote:
> On Jul 6, 2012, at 4:07 PM, Ricardo SIGNES (via RT) wrote:
> > Lexical topicalization by given is the cause of nearly endless grief and
> > very little, if any, benefit.
>
>
> Once upon a time I was innocently reading perldelta and came across this new lexical $_. It seemed like a great idea, for the same reason that any other lexical variable is a good idea. I started using "my $_" all over the place, because it just seemed like the right thing to do, having read "Coping with Scoping" and similar things.
>
> And then I ran into problems with List::Util functions, and the problem I described here: http://perlmonks.org/?node_id=958820 , and I realized lexical $_ had more or less insoluble problems and stopped using it.
>
> But that's just me. I don't mean this rhetorically -- If lexical $_ is problematic enough that it's worth changing given, is it problematic enough that it should be discouraged in general -- at least by indicating some of its problems in perlvar?

I would also not be opposed to this. The problems with given are
definitely more important, because given lexicalizes $_ implicitly, but
the underlying issues are really with lexical $_ as a whole. Saying "you
asked for it by saying my $_" is a valid response in some respects, but
do we really want to let people just ask for something that might break
their code that subtly?

-doy


chris at prather

Jul 8, 2012, 5:30 PM

Post #6 of 14 (186 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Sat, Jul 7, 2012 at 9:13 AM, Jesse Luehrs <doy [at] tozt> wrote:

> do we really want to let people just ask for something that might break
> their code that subtly?

Er well this *is* Perl, enough rope, shoot foot ... yada yada yada.
The last time I remember this coming up it was shown pretty much that
no matter *what* happened we would be letting people subtly break
their code. See also Ric's epiphany:
http://www.nntp.perl.org/group/perl.perl5.porters/2011/11/msg178998.html

-Chris


aaron at priven

Jul 9, 2012, 12:32 PM

Post #7 of 14 (194 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

Thanks for pointing out that thread.

The problem with the "we're handing them their own rope and they can decide
how much to use" position is that most people haven't been following this
issue for years and aren't going to understand why $_ is different from
every other lexical variable. And it is.

Only $_ (and sometimes $a and $b, but we don't use "my" on those either) is
used as the placeholder variable in called builtins and subroutines. This
causes problems with lots of called routines -- ones like Text::Trim, which
use $_ as a default, and like List::Util::first, which use $_ as the
placeholder variable in a block. Of course it works fine on core builtins,
because they've been hacked to handle it.

And it's possible for some (but not all -- see below) called routines to
handle lexical $_ via the _ prototype, but of course, most don't, and it's
not at all obvious to someone not familiar with the issue why some calling
routines will handle lexical $_ and some won't. And, of course, the fact
that it *is* possible makes lexical $_ not entirely lexical...

Most people won't understand this unless they are told, and there's no
mention of this problem anywhere in the docs that I know of, except for
entries in old perldeltas, which just point people to rt #67694. That is
insufficient. If people haven't been following this issue for years end up
using my $_ for the same reason they use my $whatever -- because after all,
lexical is better, otherwise why would perl 5 have introduced my in the
first place? -- they're not being handed rope. They're being handed a
loaded gun, without a class in gun safety. And people like Abigail have
"done almost two decades of preaching 'use my, not local'".

I do not know enough about what might happen if perl's behavior were
changed to suggest eliminating lexical $_. Maybe it should; maybe not; I
don't know. I do know that the documentation in perlvar is not enough to
help most people understand whether to use it or not, or why using it might
not work the way they expect.

--

I will point out again the other issue I have with lexical $_, which is
that calling routines can use it as a default only sometimes, because the _
prototype imposes scalar context. You can't create a function that works
like chop or chomp, where it takes any number of arguments but defaults to
$_. Here's what I wrote on that -- I think I've linked to it before here
but not posted it on perl5-porters. (It's originally from
http://perlmonks.org/?node_id=958820 )

So, one of my favorite pieces of perl is Text::Trim, which uses contexts to
do Just The Right Thing. Here is the trim() function, with some comments
that I wrote:

sub trim {
@_ = @_ ? @_ : $_ if defined wantarray;
# When the caller is looking for a return value
# (not void context), make a copy of the arguments,
# so the original value is not changed.
# If there are no arguments passed,
# copy the value of $_ instead.

for (@_ ? @_ : $_) { next unless defined; s/\A\s+//; s/\s+\z// }
# If @_ contains any values, perform the
# substitution on all defined values. If @_ contains
# no values, perform the substitution on $_ instead.

return @_ if wantarray || !defined wantarray;
# return the changed values in array context
# or (for a reason which escapes me) void context.

if (my @def = grep defined, @_) {
return "@def"
} else { return }
# In scalar context, return the results of
# the substitution on all
# defined values, if any. Otherwise just return.

}

This code is pretty hard to read, mostly because of the punctuation
variables. But it allows the caller to do

@trimmed = trim(@not_trimmed);

or

foreach (@untrimmed) {
say trim;
}

or

trim(@data);


And the results make sense.

But, as with so many things, it doesn't work with lexical $_. I thought
maybe I could get this to work with the new underscore _ prototype, but
unfortunately that imposes scalar context on the argument. In other words,

sub trim (_;@) {
# ... as above
}

my @data = (' a', ' b ' , 'c ');
say trim (@data);

displays not "abc" but "3", since that's the value of @data in scalar
context (the number of elements).

I can't think of any way around this that allows the flexibility of the
Text::Trim calling conventions and also the use of lexical $_.

Any ideas?



On Jul 8, 2012, at 5:30 PM, Chris Prather wrote:

Er well this *is* Perl, enough rope, shoot foot ... yada yada yada.
The last time I remember this coming up it was shown pretty much that
no matter *what* happened we would be letting people subtly break
their code. See also Ric's epiphany:
http://www.nntp.perl.org/group/perl.perl5.porters/2011/11/msg178998.html

-Chris


perlbug-comment at perl

Jul 9, 2012, 12:41 PM

Post #8 of 14 (187 views)
Permalink
[perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Mon Jul 09 12:34:17 2012, aaron [at] priven wrote:
> This code is pretty hard to read, mostly because of the punctuation
> variables. But it allows the caller to do
>
> @trimmed = trim(@not_trimmed);
>
> or
>
> foreach (@untrimmed) {
> say trim;
> }
>
> or
>
> trim(@data);
>
>
> And the results make sense.
>
> But, as with so many things, it doesn't work with lexical $_. I thought
> maybe I could get this to work with the new underscore _ prototype, but
> unfortunately that imposes scalar context on the argument. In other words,
>
> sub trim (_;@) {
> # ... as above
> }
>
> my @data = (' a', ' b ' , 'c ');
> say trim (@data);
>
> displays not "abc" but "3", since that's the value of @data in scalar
> context (the number of elements).
>
> I can't think of any way around this that allows the flexibility of the
> Text::Trim calling conventions and also the use of lexical $_.
>
> Any ideas?

Well, if you rewrite the routine in XS, you can use the UNDERBAR macro
to access it.

But a better solution would be to deprecate my $_ or at least put big
warnings up in the documentation for it.

It’s unreasonable for every module that wants to handle $_ to go through
that. Also, if someone is relying on it using the global $_ when there
is a lexical $_ in scope, that code will break. That’s something that
always bothered me about it. Code that doesn’t use the caller’s lexical
$_ cannot be changed to use it, because that’s not a backward-compatible
change. It has to use lexical $_ to begin with.

--

Father Chrysostomos


fawaka at gmail

Jul 10, 2012, 2:20 AM

Post #9 of 14 (181 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Mon, Jul 9, 2012 at 10:41 PM, Father Chrysostomos via RT
<perlbug-comment [at] perl> wrote:
> Well, if you rewrite the routine in XS, you can use the UNDERBAR macro
> to access it.

Yeah, but what you usually want is a localized or lexical $_, not just
a dynamic or lexical one. It's better than nothing, but still not
quite what you want most of the time.

Leon


perl.p5p at rjbs

Aug 1, 2012, 8:57 PM

Post #10 of 14 (141 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

* Father Chrysostomos via RT <perlbug-followup [at] perl> [2012-08-01T23:41:24]
> > Does this mean I can merge my patch now (which breaks none of CPAN)?
>
> Since you said
>
> > It should print "10\n10\n"
>
> I am taking that as a ‘yes’.

Oops, I'm sure I replied with a "yes" immediately at the time, but maybe I sent
it to the wrong place or used my until-recently-broken MTA or something.

At any rate, it was an emphatic "yes." I was just telling someone yesterday
how strongly I support this change and how much better it is for the language.

--
rjbs
Attachments: signature.asc (0.48 KB)


doy at tozt

Aug 1, 2012, 9:03 PM

Post #11 of 14 (141 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Wed, Aug 01, 2012 at 08:46:55PM -0700, Father Chrysostomos via RT wrote:
> On Mon Jul 09 12:34:17 2012, aaron [at] priven wrote:
> > Thanks for pointing out that thread.
> >
> > The problem with the "we're handing them their own rope and they can decide
> > how much to use" position is that most people haven't been following this
> > issue for years and aren't going to understand why $_ is different from
> > every other lexical variable. And it is.
> >
> > Only $_ (and sometimes $a and $b, but we don't use "my" on those either) is
> > used as the placeholder variable in called builtins and subroutines. This
> > causes problems with lots of called routines -- ones like Text::Trim, which
> > use $_ as a default, and like List::Util::first, which use $_ as the
> > placeholder variable in a block. Of course it works fine on core builtins,
> > because they've been hacked to handle it.
> >
> > And it's possible for some (but not all -- see below) called routines to
> > handle lexical $_ via the _ prototype, but of course, most don't, and it's
> > not at all obvious to someone not familiar with the issue why some calling
> > routines will handle lexical $_ and some won't. And, of course, the fact
> > that it *is* possible makes lexical $_ not entirely lexical...
> >
> > Most people won't understand this unless they are told, and there's no
> > mention of this problem anywhere in the docs that I know of, except for
> > entries in old perldeltas, which just point people to rt #67694. That is
> > insufficient. If people haven't been following this issue for years end up
> > using my $_ for the same reason they use my $whatever -- because after all,
> > lexical is better, otherwise why would perl 5 have introduced my in the
> > first place? -- they're not being handed rope. They're being handed a
> > loaded gun, without a class in gun safety.
>
> With commit fc33dad, I’ve downgraded it to rope.

This is a pretty strongly worded section in the documentation for a
feature that isn't deprecated.

-doy


doy at tozt

Aug 1, 2012, 10:11 PM

Post #12 of 14 (142 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Wed, Aug 01, 2012 at 10:00:13PM -0700, Father Chrysostomos via RT wrote:
> On Wed Aug 01 21:04:23 2012, doy [at] tozt wrote:
> > On Wed, Aug 01, 2012 at 08:46:55PM -0700, Father Chrysostomos via RT
> > wrote:
> > > With commit fc33dad, I’ve downgraded it to rope.
> >
> > This is a pretty strongly worded section in the documentation for a
> > feature that isn't deprecated.
>
> Do you have any suggestions? Are you referring to the last sentence?
> If I change it to ‘Hence, lexical C<$_> is discouraged’ does that
> address your concern?

Well, my personal suggestion would be to follow though and deprecate
it(: If the general opinion is that there really isn't any reason that a
user should be using this feature (which is basically what that addition
to the documentation is saying, and which I agree with), then why would
we not tell people who are actually using it?

In general, hiding this sort of thing in some random place in our
massive documentation seems like it's only useful to provide more
ammunition to code style critics ("everybody knows that you shouldn't
use lexical $_!") rather than to the people who might actually directly
benefit from it. If it's bad enough to merit this strong of a warning
against using it in the documentation, it's worth being just a bit more
helpful and making sure people actually notice it.

-doy


doy at tozt

Aug 1, 2012, 10:27 PM

Post #13 of 14 (142 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

On Wed, Aug 01, 2012 at 10:20:42PM -0700, Father Chrysostomos via RT wrote:
> On Wed Aug 01 22:11:32 2012, doy [at] tozt wrote:
> > Well, my personal suggestion would be to follow though and deprecate
> > it(: If the general opinion is that there really isn't any reason that a
> > user should be using this feature (which is basically what that addition
> > to the documentation is saying, and which I agree with), then why would
> > we not tell people who are actually using it?
>
> Abigail complained earlier when someone suggested deprecation. I still
> support deprecating it, but am trying to accommodate others’ views.

Well, surely if he is opposed to deprecating it, he would also be
opposed to the official perl documentation containing something like
this(: I just think that this sort of compromise ends up making
everybody involved worse off.

> But deprecating and removing it would be the easiest way to fix this:-)
>
> $ ./perl -Ilib -le 'my $_ = 345; sub r {reverse} r'
> Bizarre copy of ARRAY in reverse at -e line 1.
>
> >
> > In general, hiding this sort of thing in some random place in our
> > massive documentation seems like it's only useful to provide more
> > ammunition to code style critics ("everybody knows that you shouldn't
> > use lexical $_!") rather than to the people who might actually directly
> > benefit from it. If it's bad enough to merit this strong of a warning
> > against using it in the documentation, it's worth being just a bit more
> > helpful and making sure people actually notice it.
>
> Hmm, I wonder whether we could warn for $_ being closed over by an inner
> subroutine, or perhaps only when the ‘sub’ keyword is implicit. That
> would help with the List::Util problem.

This may actually be a much better place to compromise. It would address
the majority of my problems with it, anyway.

> I don’t know the best way to proceed.

Maybe try a CPAN smoke where only closing over lexical $_ warns?

-doy


pagaltzis at gmx

Aug 11, 2012, 6:32 PM

Post #14 of 14 (134 views)
Permalink
Re: [perl #114020] given should localize topic, not lexicalize topic [In reply to]

* Father Chrysostomos via RT <perlbug-followup [at] perl> [2012-08-02 09:15]:
> In the end, I don’t think it is worth trying to salvage $_.

Dead by 5.20 then?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>

Perl porters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.