Mailing List Archive: Perl: porters

Eval is deadly slow

1 2

View All

Index | Next | Previous | View Threaded

bzm at 2bz

Nov 5, 2009, 5:53 AM

Post #26 of 37 (190 views)
Permalink

Re: Eval is deadly slow [In reply to]

Hi,

I use

$x = pack 'x10000';
$x = '';

which is 10 times faster than creating a list. So SvGROW looks
overdone to me.

Am 05.11.2009 um 08:48 schrieb H.Merijn Brand:

> On Wed, 4 Nov 2009 20:59:50 +0000, Ben Morrow <ben [at] morrow>
> wrote:
>
>> Quoth h.m.brand [at] xs4all ("H.Merijn Brand"):
>>>
>>> IIRC I once suggested to allow length () to do preallocation:
>>>
>>> $ perl -wle'length($a) = 8000'
>>> Can't modify length in scalar assignment at -e line 1, at EOF
>>> Execution of -e aborted due to compilation errors.
>>
>> ~% perl -MDevel::Peek -e'my $x = ("x" x 10_000); $x = ""; Dump $x'
>> SV = PV(0x810109c) at 0x8103b10
>> REFCNT = 1
>> FLAGS = (PADMY,POK,pPOK)
>> PV = 0x8134004 ""\0
>> CUR = 0
>> LEN = 16380
>> ~%
>>
>> so while lvalue length would be a little neater, it's not strictly
>> necessary.
>
> There is quite a big difference here. 'x' x 10_000 actually
> allocates a
> temporary space for the value to be used to initialize $x with, then
> copies the variable and resets the length afterwards. A lot of
> unneeded
> ops.
>
> $ perl -MO=Deparse -we'my $x = ("x" x 10_000); $x = ""'
> BEGIN { $^W = 1; }
> my $x = 'x' x 10000;
> $x = '';
> -e syntax OK
> $ perl -MO=Concise -we'my $x = ("x" x 10_000); $x = ""'
> c <@> leave[1 ref] vKP/REFC ->(end)
> 1 <0> enter ->2
> 2 <;> nextstate(main 1 -e:1) v:{ ->3
> 7 <2> sassign vKS/2 ->8
> 5 <2> repeat[t2] sKP/2 ->6
> 3 <$> const(PV "x") s ->4
> 4 <$> const(IV 10000) s ->5
> 6 <0> padsv[$x:1,2] sRM*/LVINTRO ->7
> 8 <;> nextstate(main 2 -e:1) v:{ ->9
> b <2> sassign vKS/2 ->c
> 9 <$> const(PV "") s ->a
> a <0> padsv[$x:1,2] sRM* ->b
> -e syntax OK
>
> If possible, I would suggest to allow 'length ($x) = 10000' to be
> something ending in
>
> SvGROW ($x, 10000);
>
> one single op!
>
> --
> H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
> using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20,
> 11.00,
> 11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and
> 5.3.
> http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
> http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

--
Boris

h.m.brand at xs4all

Nov 5, 2009, 6:01 AM

Post #27 of 37 (191 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Thu, 5 Nov 2009 14:53:05 +0100, Boris Zentner <bzm [at] 2bz> wrote:

>
> Hi,
>
> I use
>
> $x = pack 'x10000';
> $x = '';

A very good solution, but not very obvious for all-day use. IMHO

length (my $x) = 10_000;

is more obvious, even if would use pack in the internals :p

> which is 10 times faster than creating a list. So SvGROW looks
> overdone to me.

I'm almost certain that pack will internally also use SvGROW to
populate the scalar. Direct or indirect.

> Am 05.11.2009 um 08:48 schrieb H.Merijn Brand:
>
> > On Wed, 4 Nov 2009 20:59:50 +0000, Ben Morrow <ben [at] morrow>
> > wrote:
> >
> >> Quoth h.m.brand [at] xs4all ("H.Merijn Brand"):
> >>>
> >>> IIRC I once suggested to allow length () to do preallocation:
> >>>
> >>> $ perl -wle'length($a) = 8000'
> >>> Can't modify length in scalar assignment at -e line 1, at EOF
> >>> Execution of -e aborted due to compilation errors.
> >>
> >> ~% perl -MDevel::Peek -e'my $x = ("x" x 10_000); $x = ""; Dump $x'
> >> SV = PV(0x810109c) at 0x8103b10
> >> REFCNT = 1
> >> FLAGS = (PADMY,POK,pPOK)
> >> PV = 0x8134004 ""\0
> >> CUR = 0
> >> LEN = 16380
> >> ~%
> >>
> >> so while lvalue length would be a little neater, it's not strictly
> >> necessary.
> >
> > There is quite a big difference here. 'x' x 10_000 actually
> > allocates a
> > temporary space for the value to be used to initialize $x with, then
> > copies the variable and resets the length afterwards. A lot of
> > unneeded
> > ops.
> >
> > $ perl -MO=Deparse -we'my $x = ("x" x 10_000); $x = ""'
> > BEGIN { $^W = 1; }
> > my $x = 'x' x 10000;
> > $x = '';
> > -e syntax OK
> > $ perl -MO=Concise -we'my $x = ("x" x 10_000); $x = ""'
> > c <@> leave[1 ref] vKP/REFC ->(end)
> > 1 <0> enter ->2
> > 2 <;> nextstate(main 1 -e:1) v:{ ->3
> > 7 <2> sassign vKS/2 ->8
> > 5 <2> repeat[t2] sKP/2 ->6
> > 3 <$> const(PV "x") s ->4
> > 4 <$> const(IV 10000) s ->5
> > 6 <0> padsv[$x:1,2] sRM*/LVINTRO ->7
> > 8 <;> nextstate(main 2 -e:1) v:{ ->9
> > b <2> sassign vKS/2 ->c
> > 9 <$> const(PV "") s ->a
> > a <0> padsv[$x:1,2] sRM* ->b
> > -e syntax OK
> >
> > If possible, I would suggest to allow 'length ($x) = 10000' to be
> > something ending in
> >
> > SvGROW ($x, 10000);
> >
> > one single op!

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

h.m.brand at xs4all

Nov 5, 2009, 6:04 AM

Post #28 of 37 (191 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Thu, 05 Nov 2009 13:13:33 +0000, hv [at] crypt wrote:

> demerphq <demerphq [at] gmail> wrote:
> :2009/11/4 Ben Morrow <ben [at] morrow>:
> :> Quoth h.m.brand [at] xs4all ("H.Merijn Brand"):
> :>>
> :>> IIRC I once suggested to allow length () to do preallocation:
> :>>
> :>> $ perl -wle'length($a) = 8000'
> :>> Can't modify length in scalar assignment at -e line 1, at EOF
> :>> Execution of -e aborted due to compilation errors.
> :>
> :> ~% perl -MDevel::Peek -e'my $x = ("x" x 10_000); $x = ""; Dump $x'
> :> SV = PV(0x810109c) at 0x8103b10
> :> REFCNT = 1
> :> FLAGS = (PADMY,POK,pPOK)
> :> PV = 0x8134004 ""\0
> :> CUR = 0
> :> LEN = 16380
> :> ~%
> :>
> :> so while lvalue length would be a little neater, it's not strictly
> :> necessary.
> :
> :Also notice the preallocated space. It was asked for 10k it gave 1638.
>
> An alternative approach:
> ~% perl -MDevel::Peek -e 'my $x=""; vec($x, 10000, 8)=0; $x=""; Dump $x'
> SV = PV(0x8b13b00) at 0x8b12cdc
> REFCNT = 2
> FLAGS = (PADMY,POK,pPOK)
> PV = 0x8b53c20 ""\0
> CUR = 0
> LEN = 10004
> ~%
>
> Hmm, not sure why I get refcount 2. But I think this should be a more
> efficient way to allocate the space then ("x" x 10_000).

And a lot more efficient than ('x" x 10_000), op-wise

$ perl -MO=Concise -we'my $x = ""; vec ($x, 10000, 8)=0; $x=""'
h <@> leave[1 ref] vKP/REFC ->(end)
1 <0> enter ->2
2 <;> nextstate(main 1 -e:1) v:{ ->3
5 <2> sassign vKS/2 ->6
3 <$> const(PV "") s ->4
4 <0> padsv[$x:1,2] sRM*/LVINTRO ->5
6 <;> nextstate(main 2 -e:1) v:{ ->7
c <2> sassign vKS/2 ->d
7 <$> const(IV 0) s ->8
b <@> vec[t3] sKRM*/3 ->c
- <0> ex-pushmark s ->8
8 <0> padsv[$x:1,2] sRM* ->9
9 <$> const(IV 10000) s ->a
a <$> const(IV 8) s ->b
d <;> nextstate(main 2 -e:1) v:{ ->e
g <2> sassign vKS/2 ->h
e <$> const(PV "") s ->f
f <0> padsv[$x:1,2] sRM* ->g

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

h.m.brand at xs4all

Nov 5, 2009, 7:09 AM

Post #29 of 37 (189 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Thu, 5 Nov 2009 14:53:05 +0100, Boris Zentner <bzm [at] 2bz> wrote:

>
> Hi,
>
> I use
>
> $x = pack 'x10000';
> $x = '';
>
> which is 10 times faster than creating a list. So SvGROW looks
> overdone to me.

Better in the number of op's section, but not in the number of mallocs:

% perl5.8.8 -Dm -e'$x="";$x="x"x 10000' |&\
perl -ne'/free/&&next;s/0x\w+/0x####/&&print' >z_x
% perl5.8.8 -Dm -e'$x="";$x=pack"x10000"' |&\
perl -ne'/free/&&next;s/0x\w+/0x####/&&print' >z_p
% diff z_x z_p
764c764
< 0x####: (00779) realloc 2 bytes
---
> 0x####: (00779) realloc 7 bytes
767,768c767,768
< 0x####: (00783) calloc 1 x 48 bytes
< 0x####: (00789) calloc 1 x 56 bytes
---
> 0x####: (00788) calloc 1 x 56 bytes
> 0x####: (00789) calloc 1 x 40 bytes
777c777
< 0x####: (00812) realloc 10008 bytes
---
> 0x####: (00812) realloc 10016 bytes

> Am 05.11.2009 um 08:48 schrieb H.Merijn Brand:
>
> > On Wed, 4 Nov 2009 20:59:50 +0000, Ben Morrow <ben [at] morrow>
> > wrote:
> >
> >> Quoth h.m.brand [at] xs4all ("H.Merijn Brand"):
> >>>
> >>> IIRC I once suggested to allow length () to do preallocation:
> >>>
> >>> $ perl -wle'length($a) = 8000'
> >>> Can't modify length in scalar assignment at -e line 1, at EOF
> >>> Execution of -e aborted due to compilation errors.
> >>
> >> ~% perl -MDevel::Peek -e'my $x = ("x" x 10_000); $x = ""; Dump $x'
> >> SV = PV(0x810109c) at 0x8103b10
> >> REFCNT = 1
> >> FLAGS = (PADMY,POK,pPOK)
> >> PV = 0x8134004 ""\0
> >> CUR = 0
> >> LEN = 16380
> >> ~%
> >>
> >> so while lvalue length would be a little neater, it's not strictly
> >> necessary.
> >
> > There is quite a big difference here. 'x' x 10_000 actually
> > allocates a
> > temporary space for the value to be used to initialize $x with, then
> > copies the variable and resets the length afterwards. A lot of
> > unneeded
> > ops.
> >
> > $ perl -MO=Deparse -we'my $x = ("x" x 10_000); $x = ""'
> > BEGIN { $^W = 1; }
> > my $x = 'x' x 10000;
> > $x = '';
> > -e syntax OK
> > $ perl -MO=Concise -we'my $x = ("x" x 10_000); $x = ""'
> > c <@> leave[1 ref] vKP/REFC ->(end)
> > 1 <0> enter ->2
> > 2 <;> nextstate(main 1 -e:1) v:{ ->3
> > 7 <2> sassign vKS/2 ->8
> > 5 <2> repeat[t2] sKP/2 ->6
> > 3 <$> const(PV "x") s ->4
> > 4 <$> const(IV 10000) s ->5
> > 6 <0> padsv[$x:1,2] sRM*/LVINTRO ->7
> > 8 <;> nextstate(main 2 -e:1) v:{ ->9
> > b <2> sassign vKS/2 ->c
> > 9 <$> const(PV "") s ->a
> > a <0> padsv[$x:1,2] sRM* ->b
> > -e syntax OK
> >
> > If possible, I would suggest to allow 'length ($x) = 10000' to be
> > something ending in
> >
> > SvGROW ($x, 10000);
> >
> > one single op!

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

Tim.Bunce at pobox

Nov 5, 2009, 9:48 AM

Post #30 of 37 (190 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Wed, Nov 04, 2009 at 08:58:03PM +0100, H.Merijn Brand wrote:
> On Wed, 4 Nov 2009 20:38:32 +0100, demerphq <demerphq [at] gmail> wrote:
>
> > 2009/11/4 Horsley, Tom <Tom.Horsley [at] ccur>:
> > >> The length of the string is very important here as the graphs show. If
> > >> they are short i bet you see minimal difference.
> > >
> > > I know nothing about how perl works internally, but I've seen things like
> > > this in many places besides perl where programs try to accumulate
> > > string data and have to keep reallocing the accumulator to make it
> > > bigger as the string continues to grow. Memory gets fragmented and
> > > not reused because the string is getting bigger and none of the
> > > old fragments are big enough, then when finally done, the program
> > > starts over again and goes through the exact same thing on the
> > > next string. Sometimes something simple like keeping track of
> > > the biggest string seen, and preallocating that much space for
> > > new strings can work wonders. Freeing extra space once at the
> > > end is usually less overhead than constant reallocation during
> > > the string creation.
> > >
> > > Of course, this may be completely irrelevant, depending on how
> > > perl actually parses strings today.
> >
> > That looks to be the same process we are using. If my cursory analysis
> > is right we preallocate 80 bytes, then grow the string incrementally
> > after that.
>
> IIRC I once suggested to allow length () to do preallocation:
>
> $ perl -wle'length($a) = 8000'
> Can't modify length in scalar assignment at -e line 1, at EOF
> Execution of -e aborted due to compilation errors.

That's fine and good, but it's not relevant to the original thread.

The benchmark I used is measuring compile time and shows that
_parsing_ long literal strings is currently inefficient.

Tim.

demerphq at gmail

Nov 6, 2009, 1:24 AM

Post #31 of 37 (184 views)
Permalink

Re: Eval is deadly slow [In reply to]

2009/11/5 Tim Bunce <Tim.Bunce [at] pobox>:
> On Wed, Nov 04, 2009 at 08:58:03PM +0100, H.Merijn Brand wrote:
>> On Wed, 4 Nov 2009 20:38:32 +0100, demerphq <demerphq [at] gmail> wrote:
>>
>> > 2009/11/4 Horsley, Tom <Tom.Horsley [at] ccur>:
>> > >> The length of the string is very important here as the graphs show. If
>> > >> they are short i bet you see minimal difference.
>> > >
>> > > I know nothing about how perl works internally, but I've seen things like
>> > > this in many places besides perl where programs try to accumulate
>> > > string data and have to keep reallocing the accumulator to make it
>> > > bigger as the string continues to grow. Memory gets fragmented and
>> > > not reused because the string is getting bigger and none of the
>> > > old fragments are big enough, then when finally done, the program
>> > > starts over again and goes through the exact same thing on the
>> > > next string. Sometimes something simple like keeping track of
>> > > the biggest string seen, and preallocating that much space for
>> > > new strings can work wonders. Freeing extra space once at the
>> > > end is usually less overhead than constant reallocation during
>> > > the string creation.
>> > >
>> > > Of course, this may be completely irrelevant, depending on how
>> > > perl actually parses strings today.
>> >
>> > That looks to be the same process we are using. If my cursory analysis
>> > is right we preallocate 80 bytes, then grow the string incrementally
>> > after that.
>>
>> IIRC I once suggested to allow length () to do preallocation:
>>
>> $ perl -wle'length($a) = 8000'
>> Can't modify length in scalar assignment at -e line 1, at EOF
>> Execution of -e aborted due to compilation errors.
>
> That's fine and good, but it's not relevant to the original thread.
>
> The benchmark I used is measuring compile time and shows that
> _parsing_ long literal strings is currently inefficient.

I agree. Preallocation is only relevent to this thread insofar as the
core appears to not do enough. But this thread is about the problem of
inefficient parsing as a whole not the question of preallocation.

yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

mark at mark

Nov 6, 2009, 8:18 AM

Post #32 of 37 (182 views)
Permalink

Re: Eval is deadly slow [In reply to]

On 11/06/2009 04:24 AM, demerphq wrote:
> I agree. Preallocation is only relevent to this thread insofar as the
> core appears to not do enough. But this thread is about the problem of
> inefficient parsing as a whole not the question of preallocation.
>

Would it be correct to say that this problem applies to all Perl
scripts, modules, and eval strings that have tokens > 64 characters?

Cheers,
mark

--
Mark Mielke<mark [at] mielke>

Tim.Bunce at pobox

Nov 6, 2009, 11:58 AM

Post #33 of 37 (181 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Fri, Nov 06, 2009 at 11:18:27AM -0500, Mark Mielke wrote:
> On 11/06/2009 04:24 AM, demerphq wrote:
>> I agree. Preallocation is only relevent to this thread insofar as the
>> core appears to not do enough. But this thread is about the problem of
>> inefficient parsing as a whole not the question of preallocation.
>>
>
> Would it be correct to say that this problem applies to all Perl scripts,
> modules, and eval strings that have tokens > 64 characters?

I dont know. But this code spends >30% of its time inside S_scan_str:

use strict;
use warnings;

my $str = "x" x 1000;
my $code = join "\n", map { qq{\$a .= "$str";\n} } 1..1000;

warn "compiling...\n";
eval "return 1; sub { my \$a; $code }" or die
for 1..1000;

gprof would shed light on exactly where, and thus why.

Tim.

h.m.brand at xs4all

Nov 6, 2009, 2:08 PM

Post #34 of 37 (181 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Thu, 5 Nov 2009 14:53:05 +0100, Boris Zentner <bzm [at] 2bz> wrote:

> Hi,
>
> I use
>
> $x = pack 'x10000';
> $x = '';
>
> which is 10 times faster than creating a list. So SvGROW looks
> overdone to me.

:) Next version of Data::Peek will have DGrow (), just as a proof of
concept:

$ perl -MDP -e'my$x="";DGrow($x,10000);DDump$x'
SV = PV(0x8265048) at 0x8268310
REFCNT = 1
FLAGS = (PADMY,POK,pPOK)
PV = 0x82869c0 ""\0
CUR = 0
LEN = 10000

--8<---
use strict;
use warnings;
use Data::Peek qw( DGrow );

use Benchmark qw( cmpthese );

cmpthese (-2, {
pack => q{my $x = ""; $x = pack "x20000"; $x = "";},
op_x => q{my $x = ""; $x = "x" x 20000; $x = "";},
grow => q{my $x = ""; DGrow ($x, 20000); $x = "";},
});
-->8---

Rate op_x pack grow
op_x 62127/s -- -59% -96%
pack 152046/s 145% -- -91%
grow 1622943/s 2512% 967% --

> Am 05.11.2009 um 08:48 schrieb H.Merijn Brand:
>
> > On Wed, 4 Nov 2009 20:59:50 +0000, Ben Morrow <ben [at] morrow>
> > wrote:
> >
> >> Quoth h.m.brand [at] xs4all ("H.Merijn Brand"):
> >>>
> >>> IIRC I once suggested to allow length () to do preallocation:
> >>>
> >>> $ perl -wle'length($a) = 8000'
> >>> Can't modify length in scalar assignment at -e line 1, at EOF
> >>> Execution of -e aborted due to compilation errors.
> >>
> >> ~% perl -MDevel::Peek -e'my $x = ("x" x 10_000); $x = ""; Dump $x'
> >> SV = PV(0x810109c) at 0x8103b10
> >> REFCNT = 1
> >> FLAGS = (PADMY,POK,pPOK)
> >> PV = 0x8134004 ""\0
> >> CUR = 0
> >> LEN = 16380
> >> ~%
> >>
> >> so while lvalue length would be a little neater, it's not strictly
> >> necessary.
> >
> > There is quite a big difference here. 'x' x 10_000 actually
> > allocates a
> > temporary space for the value to be used to initialize $x with, then
> > copies the variable and resets the length afterwards. A lot of
> > unneeded
> > ops.
> >
> > $ perl -MO=Deparse -we'my $x = ("x" x 10_000); $x = ""'
> > BEGIN { $^W = 1; }
> > my $x = 'x' x 10000;
> > $x = '';
> > -e syntax OK
> > $ perl -MO=Concise -we'my $x = ("x" x 10_000); $x = ""'
> > c <@> leave[1 ref] vKP/REFC ->(end)
> > 1 <0> enter ->2
> > 2 <;> nextstate(main 1 -e:1) v:{ ->3
> > 7 <2> sassign vKS/2 ->8
> > 5 <2> repeat[t2] sKP/2 ->6
> > 3 <$> const(PV "x") s ->4
> > 4 <$> const(IV 10000) s ->5
> > 6 <0> padsv[$x:1,2] sRM*/LVINTRO ->7
> > 8 <;> nextstate(main 2 -e:1) v:{ ->9
> > b <2> sassign vKS/2 ->c
> > 9 <$> const(PV "") s ->a
> > a <0> padsv[$x:1,2] sRM* ->b
> > -e syntax OK
> >
> > If possible, I would suggest to allow 'length ($x) = 10000' to be
> > something ending in
> >
> > SvGROW ($x, 10000);
> >
> > one single op!

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

demerphq at gmail

Nov 7, 2009, 1:38 AM

Post #35 of 37 (173 views)
Permalink

Re: Eval is deadly slow [In reply to]

2009/11/6 H.Merijn Brand <h.m.brand [at] xs4all>:
> On Thu, 5 Nov 2009 14:53:05 +0100, Boris Zentner <bzm [at] 2bz> wrote:
>
>> Hi,
>>
>> I use
>>
>> � $x = pack 'x10000';
>> � $x = '';
>>
>> which is 10 times faster than creating a list. So SvGROW looks
>> overdone to me.
>
> :) Next version of Data::Peek will have DGrow (), just as a proof of
> concept:
>

IMO this is a bad idea. Dont put this in Data::Peek as "a proof of
concept". A module intended to be used to introspect objects should
IMO not be used as a means to alter it.

Otherwise the inevitable conversation will start: "But we already have
that in Devel::Peek", "Yes but that is a Devel module that shouldnt be
used in production", "Yes but we already have that in Devel::Peek".
Etc etc, puke, gnarl, grrr. :-)

Not only that but the concept is already proved. As such IMO you
should just put it in the new namespace we discussed some months back
and make it accessable to all perl code without requiring a use. (I
still have a todo related to this... Sigh)

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

h.m.brand at xs4all

Nov 7, 2009, 2:40 AM

Post #36 of 37 (173 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Sat, 7 Nov 2009 10:38:12 +0100, demerphq <demerphq [at] gmail> wrote:

> 2009/11/6 H.Merijn Brand <h.m.brand [at] xs4all>:
> > On Thu, 5 Nov 2009 14:53:05 +0100, Boris Zentner <bzm [at] 2bz> wrote:
> >
> >> Hi,
> >>
> >> I use
> >>
> >> $x = pack 'x10000';
> >> $x = '';
> >>
> >> which is 10 times faster than creating a list. So SvGROW looks
> >> overdone to me.
> >
> > :) Next version of Data::Peek will have DGrow (), just as a proof of
> > concept:
>
> IMO this is a bad idea. Dont put this in Data::Peek as "a proof of
> concept". A module intended to be used to introspect objects should
> IMO not be used as a means to alter it.

Well, why not? I cannot imagine anyone loading a full blown module to
gain a few microseconds on big string allocation. And if they really
need it, they're free to steal the code.

> Otherwise the inevitable conversation will start: "But we already have
> that in Devel::Peek", "Yes but that is a Devel module that shouldnt be
> used in production", "Yes but we already have that in Devel::Peek".
> Etc etc, puke, gnarl, grrr. :-)

I understand that, but I *do* use Data::Peek in production, as I get
the most unwieldy data thrown at me, and having all of DDump, DPeek,
DDisplay and DDumper available is great. (Imagine the easy part of a
-v9 option to display database data that actually shows you the Unicode
encoding)

> Not only that but the concept is already proved. As such IMO you
> should just put it in the new namespace we discussed some months back

Sorry, I don't remember, but willing to be freshened on my memory.

> and make it accessable to all perl code without requiring a use. (I
> still have a todo related to this... Sigh)

I'll cooperate!

(But DGrow () is in, and Data-Peek-0.28 released yesterday. I do see
DGrow in the same scope as triplevar (). I'm not going to use it)

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, OpenSuSE 10.3, 11.0, and 11.1, AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

perl at greerga

Nov 8, 2009, 9:30 PM

Post #37 of 37 (162 views)
Permalink

Re: Eval is deadly slow [In reply to]

On Fri, 6 Nov 2009, Tim Bunce wrote:

> On Fri, Nov 06, 2009 at 11:18:27AM -0500, Mark Mielke wrote:
>> On 11/06/2009 04:24 AM, demerphq wrote:
>>> I agree. Preallocation is only relevent to this thread insofar as the
>>> core appears to not do enough. But this thread is about the problem of
>>> inefficient parsing as a whole not the question of preallocation.
>>>
>>
>> Would it be correct to say that this problem applies to all Perl scripts,
>> modules, and eval strings that have tokens > 64 characters?
>
> I dont know. But this code spends >30% of its time inside S_scan_str:
>
> use strict;
> use warnings;
>
> my $str = "x" x 1000;
> my $code = join "\n", map { qq{\$a .= "$str";\n} } 1..1000;
>
> warn "compiling...\n";
> eval "return 1; sub { my \$a; $code }" or die
> for 1..1000;
>
> gprof would shed light on exactly where, and thus why.

valgrind + kcachegrind is relatively awesome. It shows that 90% of
S_scan_str's time on your test is in the S_scan_str block:

if (PL_multi_open == PL_multi_close) {
...
}

It looks like anything doing "*s" takes a lot more time than you would
initially expect.

(Incidentally, I kept wishing for Devel::NYTProf for C but kcachegrind is
as close as I could find.)

I used:
$ git clean -dxqf
$ ./Configure -des -Dusedevel -Doptimize='-g3' && make perl
$ time valgrind --tool=callgrind ./perl -I lib ../test.p5-scan_str.pl
$ kcachegrind callgrind.out.19558 &
(where '19558' is the PID that the first line happened to be)

I recomend looking at "Source Code" in upper left and "Call Graph" in
lower right initially. Setting "Percentage Relative to Parent" can help
too.

--
George Greer

1 2

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com