Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Perl: porters

[perl #109726] PL_sv_undef loses identity

 

 

Perl porters RSS feed   Index | Next | Previous | View Threaded


perlbug-followup at perl

Feb 3, 2012, 3:57 AM

Post #1 of 5 (36 views)
Permalink
[perl #109726] PL_sv_undef loses identity

# New Ticket Created by Zefram
# Please include the string: [perl #109726]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=109726 >



This is a bug report for perl from zefram [at] fysh,
generated with the help of perlbug 1.39 running under perl 5.15.7.


-----------------------------------------------------------------
[Please describe your issue here]

Passing the standard undef object to a subroutine as an argument fails
to pass by reference. Instead the sub sees a fresh *and writable*
undefined scalar:

$ perl -lwe 'BEGIN { *x = \undef; } sub mutate($) { print \$_[0], " second"; $_[0] = 456; } print \$x, " first"; mutate($x); print "done";'
SCALAR(0x8dc6500) first
SCALAR(0x8dc9758) second
done

whereas any other scalar has its identity preserved:

$ perl -lwe 'BEGIN { *x = \123; } sub mutate($) { print \$_[0], " second"; $_[0] = 456; } print \$x, " first"; mutate($x); print "done";'
SCALAR(0x86739f8) first
SCALAR(0x86739f8) second
Modification of a read-only value attempted at -e line 1.
$ perl -lwe 'BEGIN { *x = \!1; } sub mutate($) { print \$_[0], " second"; $_[0] = 456; } print \$x, " first"; mutate($x); print "done";'
SCALAR(0x8d5e510) first
SCALAR(0x8d5e510) second
Modification of a read-only value attempted at -e line 1.

The latter case above is PL_sv_no, which is immortal in the same way as
the misbehaving PL_sv_undef. Devel::Peek::Dump() confirms the identity
of these special scalars. Attempting mutation without the intermediation
of a subroutine call consistently produces the expected failure:

$ perl -lwe 'BEGIN { *x = \undef; } $x = 456; print "done";'
Modification of a read-only value attempted at -e line 1.
$ perl -lwe 'BEGIN { *x = \123; } $x = 456; print "done";'
Modification of a read-only value attempted at -e line 1.
$ perl -lwe 'BEGIN { *x = \!1; } $x = 456; print "done";'
Modification of a read-only value attempted at -e line 1.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=core
severity=low
---
Site configuration information for perl 5.15.7:

Configured by zefram at Sat Jan 21 18:33:38 GMT 2012.

Summary of my perl5 (revision 5 version 15 subversion 7) configuration:

Platform:
osname=linux, osvers=2.6.32-5-686, archname=i386-linux-thread-multi
uname='linux vigo.rous.org 2.6.32-5-686 #1 smp thu nov 3 04:23:54 utc 2011 i686 gnulinux '
config_args='-des -Darchname=i386-linux -Dcccdlflags=-fPIC -Dccdlflags=-rdynamic -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52 -Dman1ext=1 -Dman3ext=3perl -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.4.5', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib64
libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.11.2.so, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.11.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7/i386-linux-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:


---
@INC for perl 5.15.7:
/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/site_perl/5.15.7/i386-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/site_perl/5.15.7
/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7/i386-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7
.

---
Environment for perl 5.15.7:
HOME=/home/zefram
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/i686-pc-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
PERL_BADLANG (unset)
SHELL=/usr/bin/zsh


davem at iabyn

Feb 6, 2012, 4:57 AM

Post #2 of 5 (29 views)
Permalink
Re: [perl #109726] PL_sv_undef loses identity [In reply to]

On Fri, Feb 03, 2012 at 03:57:41AM -0800, Zefram wrote:
> Passing the standard undef object to a subroutine as an argument fails
> to pass by reference. Instead the sub sees a fresh *and writable*
> undefined scalar:
>
> $ perl -lwe 'BEGIN { *x = \undef; } sub mutate($) { print \$_[0], " second"; $_[0] = 456; } print \$x, " first"; mutate($x); print "done";'
> SCALAR(0x8dc6500) first
> SCALAR(0x8dc9758) second

The PL_sv_undef pointer in $x is copied to @_. When an array element is
fetched in lvalue context (e.g. \$_[0]), av_fetch() converts PL_sv_undef
int the slot into a real SV:

if (AvARRAY(av)[key] == &PL_sv_undef) {
emptyness:
if (lval)
return av_store(av,key,newSV(0));
return NULL;
}


--
Dave's first rule of Opera:
If something needs saying, say it: don't warble it.


zefram at fysh

Feb 6, 2012, 4:59 AM

Post #3 of 5 (29 views)
Permalink
Re: [perl #109726] PL_sv_undef loses identity [In reply to]

Dave Mitchell wrote:
>fetched in lvalue context (e.g. \$_[0]), av_fetch() converts PL_sv_undef
>int the slot into a real SV:

It's another special-case use of PL_sv_undef to mean something
other than the Perl-visibile standard undef value. Another case for
PL_sv_placeholder?

-zefram


nick at ccl4

Feb 9, 2012, 6:13 AM

Post #4 of 5 (30 views)
Permalink
Re: [perl #109726] PL_sv_undef loses identity [In reply to]

I've been thinking about this for a bit. I've just remembered an
important part. Right now, PL_sv_placeholder is in perlvars.h, *not*
intrpvar.h:

/* Restricted hashes placeholder value.
* The contents are never used, only the address. */
PERLVAR(G, sv_placeholder, SV)

It's a global, not per-interpreter. If I understand things correctly, the
address &PL_sv_placeholder is never actually *returned* anywhere - it's
used to compare against. That means that nothing can fiddle with its
reference count, or accidentally upgrade it, bless it, or anything else.

(I think that Juerd did a lightning talk on undef abuse that included
demonstrating how to bless PL_sv_undef)

Whilst that can be solved by making it per-interpreter, I'm not sure what
interesting side effects will be knock ons from there. eg

On Mon, Feb 06, 2012 at 12:59:33PM +0000, Zefram wrote:
> Dave Mitchell wrote:
> >fetched in lvalue context (e.g. \$_[0]), av_fetch() converts PL_sv_undef
> >int the slot into a real SV:
>
> It's another special-case use of PL_sv_undef to mean something
> other than the Perl-visibile standard undef value. Another case for
> PL_sv_placeholder?

This would mean that &PL_sv_placeholder has the potential to be returned
by APIs that didn't use to. Whilst this is sort of the same problem as
PL_sv_undef propagating further than we might like

a) &PL_sv_undef is well known to XS code that wants to care about it
b) Accidentally propagating PL_sv_placeholder onwards to places where it
shouldn't be (eg in the front door of the hash APIs) will cause
"interesting" bugs.

Historically, PL_sv_placeholder was introduced for 5.8.1. 5.8.0 added
restricted hashes (gah) and the placeholders, but used PL_sv_undef for
the placeholder value. Which broke existing working XS code which stored
PL_sv_undef in hashes, mostly as a token value where the key was
important.

On Fri, Dec 30, 2011 at 01:40:12PM -0800, Father Chrysostomos via RT wrote:

> There is also this interesting bit:
>
> else if (o->op_type != OP_METHOD_NAMED
> && cSVOPo->op_sv == &PL_sv_undef) {
> /* PL_sv_undef is hack - it's unsafe to store it in the
> AV that is the pad, because av_fetch treats values of
> PL_sv_undef as a "free" AV entry and will merrily
> replace them with a new SV, causing pad_alloc to think
> that this pad slot is free. (When, clearly, it is not)
> */
> SvOK_off(PAD_SVl(ix));
> SvPADTMP_on(PAD_SVl(ix));
> SvREADONLY_on(PAD_SVl(ix));
>
> I don't believe &PL_sv_placeholder is ever accessible to Perl, so using
> that instead of &PL_sv_undef in pad.c would allow XS modules to provide
> an OP_CONST that actually returns &PL_sv_undef.

This is still somewhat wooly in my head, but there are several levels in the
hierarchy of what AVs are used for

0: !AvREAL() && !AvREIFY() just the stacks? [and @DB::args currently]
1: !AvREAL() && AvREIFY() @_ [and @DB::args should be here]
2: Pads
2.5: other AVs seen only by C and XS code that can have AVs, HVs etc directly
in them
3: AVs visible to Perl code - ie via a reference, and containing only
scalars and *references* to nested containers


As far as can work out, the stacks store garbage rather than any sort of
pointer for empty parts. From this:

else {
newmax = key < 3 ? 3 : key;
MEM_WRAP_CHECK_1(newmax+1, SV*, oom_array_extend);
Newx(AvALLOC(av), newmax+1, SV*);
ary = AvALLOC(av) + 1;
tmp = newmax;
AvALLOC(av)[0] = &PL_sv_undef; /* For the stacks */
}
if (AvREAL(av)) {
while (tmp)
ary[--tmp] = &PL_sv_undef;
}

I still don't understand that part commented "For the stacks". Why does
the first element, but the first element *only* need to be a valid pointer?


Anyway, I suspect that there's low risk of problems elsewhere if the Pads
switched to using PL_sv_placeholder for a free slot, as they are only
supposed to be accessed via our APIs, which would never actually *return*
it. And Pads are already somewhat insane with their manual reference
counting fakery.

I'm concerned that there would be odd strange breakages if arrays (more
generally) started using PL_sv_placeholder because it would start escaping.

But I suspect the only way to get a handle on *that* would be to tweak the
core and then smoke CPAN against it.

Nicholas Clark


perlbug-followup at perl

Feb 12, 2012, 12:19 AM

Post #5 of 5 (30 views)
Permalink
[perl #109726] PL_sv_undef loses identity [In reply to]

On Thu Feb 09 06:13:44 2012, nicholas wrote:
> 0: !AvREAL() && !AvREIFY() just the stacks? [and @DB::args
> currently]
> 1: !AvREAL() && AvREIFY() @_ [and @DB::args should be here]

@DB::args *does* have AvREIFY set.

> Anyway, I suspect that there's low risk of problems elsewhere if the
> Pads
> switched to using PL_sv_placeholder for a free slot, as they are only
> supposed to be accessed via our APIs, which would never actually
> *return*
> it. And Pads are already somewhat insane with their manual reference
> counting fakery.

Thread cloning would have to know about it, too, as @DB::args can end up
containing pads.


--

Father Chrysostomos


---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=109726

Perl porters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.