Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Perl: porters

[perl #67838] [PATCH] lvalue substr keeping lexical alive

 

 

Perl porters RSS feed   Index | Next | Previous | View Threaded


perlbug-followup at perl

Nov 6, 2009, 5:49 PM

Post #1 of 9 (554 views)
Permalink
[perl #67838] [PATCH] lvalue substr keeping lexical alive

On Fri Nov 06 16:25:33 2009, ikegami [at] adaelis wrote:
> On Thu Nov 05 12:56:29 2009, perl [at] profvince wrote:
> > It's not only lvalue vec(), it's also lvalue pos(), substr() and maybe
> > keys().
>
> Confirmed for all four. A patch to add tests is attached.
>
> A patch to fix the problem will follow shortly.

Two patches are attached.

The first adds tests. It's an updated version of my earlier patch. It
should be used instead of the earlier patch.

The second plugs the leaks by not using TARG when a lvalue is required.
Attachments: 0001-Tests-to-detect-mem-leaks-in-lvalue-ops-RT-67838.patch (2.33 KB)
  0002-Fix-mem-leaks-in-lvalue-ops-RT-67838.patch (5.20 KB)


demerphq at gmail

Nov 7, 2009, 3:12 PM

Post #2 of 9 (515 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

2009/11/7 Eric Brine via RT <perlbug-followup [at] perl>:
> On Fri Nov 06 16:25:33 2009, ikegami [at] adaelis wrote:
>> On Thu Nov 05 12:56:29 2009, perl [at] profvince wrote:
>> > It's not only lvalue vec(), it's also lvalue pos(), substr() and maybe
>> > keys().
>>
>> Confirmed for all four. A patch to add tests is attached.
>>
>> A patch to fix the problem will follow shortly.
>
> Two patches are attached.
>
> The first adds tests. It's an updated version of my earlier patch. It
> should be used instead of the earlier patch.
>
> The second plugs the leaks by not using TARG when a lvalue is required.

Just out of curiosity why does that code decontaminate differently in
the two cases? One time it "decontaminates" taint and utf8, and one
time it just does taint. Is that a bug?

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"


ikegami at adaelis

Nov 7, 2009, 4:01 PM

Post #3 of 9 (514 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

On Sat, Nov 7, 2009 at 6:13 PM, yves orton via RT <perlbug-followup [at] perl
> wrote:

> Just out of curiosity why does that code decontaminate differently in
> the two cases? One time it "decontaminates" taint and utf8, and one
> time it just does taint. Is that a bug?
>

Some opcodes always return the result in the same SV to avoid having to
create a new SV everytime the opcode is encountered. This SV is known as
TARG.

The problem with these ops is
1) that they reference their last return value since they use TARG, and
2) that their return value references one of the opcode's arguments when
they are used as lvalues.

{
my $x = "abc"; # REFCOUNT($x) = 1 (pad)
substr($x, 1, 1) = "d"; # REFCOUNT($x) = 2 (pad,substr)
print($x); # REFCOUNT($x) = 2 (pad,substr)
}
# LEAK! # REFCOUNT($x) = 1 (substr)

The mem will relacaimed the next time that substr instance is called.

The patch has the ops use a fresh SV instead of TARG when they're used as
lvalues, making it so the arg never contains a reference to a variable.

Now to answer your question.

Before TARG is reused by the op, it's untainted. There's no use untaintaing
a freshly created variable, and there's no use untainting TARG when TARG
isn't used, so I moved the untainting into the branch where TARG is used.

ELB


ikegami at adaelis

Nov 7, 2009, 4:11 PM

Post #4 of 9 (516 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

[. Oops, my previous message didn't answer your question. I had misread it.
Let's try again ]

On Sat, Nov 7, 2009 at 6:12 PM, demerphq <demerphq [at] gmail> wrote:

> Just out of curiosity why does that code decontaminate differently in
> the two cases? One time it "decontaminates" taint and utf8, and one
> time it just does taint. Is that a bug?
>

Only one of the four ops plays with the UTF8 flag because three of the ops
return numbers.

- ELB


rgs at consttype

Nov 8, 2009, 5:18 AM

Post #5 of 9 (510 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

2009/11/7 Eric Brine via RT <perlbug-followup [at] perl>:
> On Fri Nov 06 16:25:33 2009, ikegami [at] adaelis wrote:
>> On Thu Nov 05 12:56:29 2009, perl [at] profvince wrote:
>> > It's not only lvalue vec(), it's also lvalue pos(), substr() and maybe
>> > keys().
>>
>> Confirmed for all four. A patch to add tests is attached.
>>
>> A patch to fix the problem will follow shortly.
>
> Two patches are attached.
>
> The first adds tests. It's an updated version of my earlier patch. It
> should be used instead of the earlier patch.
>
> The second plugs the leaks by not using TARG when a lvalue is required.

WIth this patch, the following tests fail :
re/substr.t
(Wstat: 65280 Tests: 328 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 334 tests but ran 328.
op/sub_lval.t
(Wstat: 65280 Tests: 56 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 69 tests but ran 56.
../lib/warnings.t
(Wstat: 0 Tests: 633 Failed: 1)
Failed test: 251

with the error "Can't return a temporary from lvalue subroutine".
That happens in cases like that one :
sub sstr : lvalue { substr($str, 1, 4) }
It seems that we have a trade-off to make here. My opinion would be to
apply your patch, at the expense of forbidding that kind of leaky
constructs. I'd like to hear comments here.

Also, the warnings.t failure apparently is a bug fix rather than a true failure.


perl at profvince

Nov 8, 2009, 5:49 AM

Post #6 of 9 (506 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

> It seems that we have a trade-off to make here. My opinion would be to
> apply your patch, at the expense of forbidding that kind of leaky
> constructs. I'd like to hear comments here.
>

I'm not too sure about this. I'd rather :
- understand why ae389c8a29b487f4434c465442dfb611507a4a38 started
incrementing the refcount of the LvTARG member ;
- if it is decided to stop lvalues from propagating too far, I'd rather
keep those ops using the TARG and decrement its refcount in the magical
callback.

Vincent.


ikegami at adaelis

Nov 8, 2009, 7:57 AM

Post #7 of 9 (508 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

On Sun, Nov 8, 2009 at 8:49 AM, Vincent Pit <perl [at] profvince> wrote:

>
> It seems that we have a trade-off to make here. My opinion would be to
>> apply your patch, at the expense of forbidding that kind of leaky
>> constructs. I'd like to hear comments here.
>>
>>
>
> I'm not too sure about this. I'd rather :
> - understand why ae389c8a29b487f4434c465442dfb611507a4a38 started
> incrementing the refcount of the LvTARG member ;
> - if it is decided to stop lvalues from propagating too far, I'd rather
> keep those ops using the TARG and decrement its refcount in the magical
> callback.
>

Can't:

$x = \substr(...);
print $$x;
print $$x;

$x = \substr(...);
$$x = uc($$x);

What about a weak reference. Is that possible? I haven't looked at how those
work.


hv at crypt

Nov 8, 2009, 8:51 AM

Post #8 of 9 (508 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

Rafael Garcia-Suarez <rgs [at] consttype> wrote:
:2009/11/7 Eric Brine via RT <perlbug-followup [at] perl>:
:> On Fri Nov 06 16:25:33 2009, ikegami [at] adaelis wrote:
:>> On Thu Nov 05 12:56:29 2009, perl [at] profvince wrote:
:>> > It's not only lvalue vec(), it's also lvalue pos(), substr() and maybe
:>> > keys().
:>>
:>> Confirmed for all four. A patch to add tests is attached.
:>>
:>> A patch to fix the problem will follow shortly.
:>
:> Two patches are attached.
:>
:> The first adds tests. It's an updated version of my earlier patch. It
:> should be used instead of the earlier patch.
:>
:> The second plugs the leaks by not using TARG when a lvalue is required.
:
:WIth this patch, the following tests fail :
[...]
:with the error "Can't return a temporary from lvalue subroutine".
:That happens in cases like that one :
:sub sstr : lvalue { substr($str, 1, 4) }
:It seems that we have a trade-off to make here. My opinion would be to
:apply your patch, at the expense of forbidding that kind of leaky
:constructs. I'd like to hear comments here.

Is it possible to restrict the leak only to the lvalue-sub case? (In fact,
is it even a leak in that case?)

I feel it should be possible to have the best of both worlds.

Hugo


ikegami at adaelis

Nov 8, 2009, 10:24 AM

Post #9 of 9 (504 views)
Permalink
Re: [perl #67838] [PATCH] lvalue substr keeping lexical alive [In reply to]

On Sun, Nov 8, 2009 at 12:21 PM, Hugo van der Sanden via RT <
perlbug-followup [at] perl> wrote:

> Is it possible to restrict the leak only to the lvalue-sub case? (In fact,
> is it even a leak in that case?)
>
> I feel it should be possible to have the best of both worlds.
>

As I understand it, yes. That's the "LVRET" in "PL_op->op_flags & OPf_MOD
|| LVRET".

Considering the leak will probably never matter, another option would be to
simply not fix it.

Perl porters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.