Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Perl: porters

Memory leak with s/// and hashes

 

 

Perl porters RSS feed   Index | Next | Previous | View Threaded


rgarciasuarez at gmail

Nov 13, 2006, 8:59 AM

Post #1 of 7 (373 views)
Permalink
Memory leak with s/// and hashes

Here's a small script, courtesy of an ex-coworker.

When executing it, the memory usage increases, but no leak is detected
by valgrind. For example, for 1000 loops, memory usage here goes from
412 to 1580. It doesn't leak with 5.8.8.

Not sure how to trace this. It seems to involve hashes and
substitutions. A problem with shared keys ?

#!perl
# usage : $0 <nb_loops>

my $str1 = "My name is_BABA_ and I do a _TINY_ _WORK_\n";
my $str2 = "I work in _ENTERPRISE_. I'm very _FEELING_ and I _ACTION_.\n";

my %values1 = (BABA => " Bibi", TINY => "Small", WORK => "Job");
my %values2 = (ENTERPRISE => "xyz", FEELING => "happy", ACTION => "sleep");

sub subtitute {
my ($string, $param) = @_;
my $r = join '|' => keys %$param;
$string =~ s/$r//g;
return $string;
}

sub printMemSize {
system("ps -C perl -o cmd=,size=");
}

printMemSize();

for (0 .. ( $ARGV[0] || 1 )) {
$foo1 = subtitute($str1,\%values1);
$foo2 = subtitute($str2,\%values2);
}

printMemSize();
__END__


davem at iabyn

Nov 13, 2006, 10:44 AM

Post #2 of 7 (354 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On Mon, Nov 13, 2006 at 05:59:08PM +0100, Rafael Garcia-Suarez wrote:
> Not sure how to trace this. It seems to involve hashes and
> substitutions. A problem with shared keys ?

Both the hash and the substitution are red herrings. This also leaks:

system("ps -C perl -o cmd=,size=");
eval 'qr(a|b)' for 0 .. ($ARGV[0]||1);
system("ps -C perl -o cmd=,size=");

It appears that compiling a pattern containing an alternation leaks.

--
The Enterprise's efficient long-range scanners detect a temporal vortex
distortion in good time, allowing it to be safely avoided via a minor
course correction.
-- Things That Never Happen in "Star Trek" #21


demerphq at gmail

Nov 13, 2006, 12:18 PM

Post #3 of 7 (352 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On 11/13/06, Dave Mitchell <davem [at] iabyn> wrote:
> On Mon, Nov 13, 2006 at 05:59:08PM +0100, Rafael Garcia-Suarez wrote:
> > Not sure how to trace this. It seems to involve hashes and
> > substitutions. A problem with shared keys ?
>
> Both the hash and the substitution are red herrings. This also leaks:
>
> system("ps -C perl -o cmd=,size=");
> eval 'qr(a|b)' for 0 .. ($ARGV[0]||1);
> system("ps -C perl -o cmd=,size=");
>
> It appears that compiling a pattern containing an alternation leaks.

I guess that makes it my fault. :-(

Ill investigate. Any further info?

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"


demerphq at gmail

Nov 13, 2006, 1:27 PM

Post #4 of 7 (359 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On 11/13/06, Dave Mitchell <davem [at] iabyn> wrote:
> On Mon, Nov 13, 2006 at 05:59:08PM +0100, Rafael Garcia-Suarez wrote:
> > Not sure how to trace this. It seems to involve hashes and
> > substitutions. A problem with shared keys ?
>
> Both the hash and the substitution are red herrings. This also leaks:
>
> system("ps -C perl -o cmd=,size=");
> eval 'qr(a|b)' for 0 .. ($ARGV[0]||1);
> system("ps -C perl -o cmd=,size=");
>
> It appears that compiling a pattern containing an alternation leaks.

Is it possible that this is something OS related?

The code:

..\perl -e"sleep(10); eval 'qr(a|b)' for 0 .. ($ARGV[0]||1); sleep(10)"

uses the same memory regardless of how many iterations I do.

Yves


--
perl -Mre=debug -e "/just|another|perl|hacker/"


davem at iabyn

Nov 13, 2006, 2:40 PM

Post #5 of 7 (354 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On Mon, Nov 13, 2006 at 10:27:21PM +0100, demerphq wrote:
> Is it possible that this is something OS related?

Well, on my linux system, the following code:

$r = '0|2';
qr/$r/, substr($r,0,1) ^= 1 for 0 .. 10;

leaks three SVs during each call to pp_regcomp.

These SVs are: an AV allocated by

TRIE_REVCHARMAP(trie) = newAV();

and two SVs pushed onto the array by

TRIE_STORE_REVCHAR;

This is bleed [at] 2924

HTH.

[. the substr is there just to defeat the pattern caching code ]




--
"You're so sadly neglected, and often ignored.
A poor second to Belgium, When going abroad."
-- Monty Python, "Finland"


nick at ccl4

Nov 13, 2006, 2:48 PM

Post #6 of 7 (358 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On Mon, Nov 13, 2006 at 10:40:45PM +0000, Dave Mitchell wrote:
> On Mon, Nov 13, 2006 at 10:27:21PM +0100, demerphq wrote:
> > Is it possible that this is something OS related?
>
> Well, on my linux system, the following code:
>
> $r = '0|2';
> qr/$r/, substr($r,0,1) ^= 1 for 0 .. 10;
>
> leaks three SVs during each call to pp_regcomp.
>
> These SVs are: an AV allocated by
>
> TRIE_REVCHARMAP(trie) = newAV();
>
> and two SVs pushed onto the array by
>
> TRIE_STORE_REVCHAR;
>
> This is bleed [at] 2924
>
> HTH.

Well, it certainly explains why valgrind reports that no memory is leaked
by program exit.
(I was midway through building perl with a copy of Devel::Arena to see if
the scalar count was a giveaway)

Nicholas Clark


rgarciasuarez at gmail

Nov 14, 2006, 1:36 AM

Post #7 of 7 (351 views)
Permalink
Re: Memory leak with s/// and hashes [In reply to]

On 14/11/06, demerphq <demerphq [at] gmail> wrote:
> Does this fix it?

Yes. Thanks, applied as #29268.

Perl porters RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.