Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: kinosearch: discuss

memory leak in KinoSearch::Util::BitVector (v .15)

 

 

kinosearch discuss RSS feed   Index | Next | Previous | View Threaded


matthew at openlist

Aug 15, 2007, 4:56 PM

Post #1 of 3 (743 views)
Permalink
memory leak in KinoSearch::Util::BitVector (v .15)

Am running into a bit of a snafu using a BitCollector in a persistent
environment. After calling the following code, memory utilization
grows arithmetically:

my $bit_vector = $collector->get_bit_vector();
my $result_ids = $bit_vector->to_arrayref();

From what I can tell, the array returned to perl is never freed up.
From BitVector.pm:

Kino_BitVec_to_array(BitVector* bit_vec) {
U32 num = 0;
AV *out_av;

out_av = newAV();
while (1) {
num = Kino_BitVec_next_set_bit(bit_vec, num);
if (num == KINO_BITVEC_SENTINEL)
break;
av_push( out_av, newSViv(num) );
num++;
}
return out_av;
}

Anyone run into this problem? Any suggestions for whether I'm on
base, or potentially how to solve for this?

Thanks,

Matthew Berk
Lead Search Architect
t 206.331.3528
f 206.331.3695
matthew [at] openlist

Marchex, Inc.
http://www.marchex.com

This email message and any attachments are solely for intended
recipients, and may contain information that is privileged and
confidential. If you are not the intended recipient, any
dissemination, distribution or copying is strictly prohibited. If you
believe that you may have received this message in error, please
immediately notify the sender by replying to this e-mail message.


matthew at openlist

Aug 15, 2007, 5:16 PM

Post #2 of 3 (686 views)
Permalink
memory leak in KinoSearch::Util::BitVector (v .15) [In reply to]

Just doing a bit of digging. Would adding sv_2mortal make a difference?

PPCODE:
out_av = Kino_BitVec_to_array(bit_vec);
/* XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av ))); */
XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av )));
XSRETURN(1);


Thanks again,

Matthew


On Aug 15, 2007, at 4:39 PM, Matthew Berk wrote:

> Am running into a bit of a snafu using a BitCollector in a
> persistent environment. After calling the following code, memory
> utilization grows arithmetically:
>
> my $bit_vector = $collector->get_bit_vector();
> my $result_ids = $bit_vector->to_arrayref();
>
> From what I can tell, the array returned to perl is never freed up.
> From BitVector.pm:
>
> Kino_BitVec_to_array(BitVector* bit_vec) {
> U32 num = 0;
> AV *out_av;
>
> out_av = newAV();
> while (1) {
> num = Kino_BitVec_next_set_bit(bit_vec, num);
> if (num == KINO_BITVEC_SENTINEL)
> break;
> av_push( out_av, newSViv(num) );
> num++;
> }
> return out_av;
> }
>
> Anyone run into this problem? Any suggestions for whether I'm on
> base, or potentially how to solve for this?
>
> Thanks,
>
> Matthew Berk
> Lead Search Architect
> t 206.331.3528
> f 206.331.3695
> matthew [at] openlist
>
> Marchex, Inc.
> http://www.marchex.com
>
> This email message and any attachments are solely for intended
> recipients, and may contain information that is privileged and
> confidential. If you are not the intended recipient, any
> dissemination, distribution or copying is strictly prohibited. If
> you believe that you may have received this message in error,
> please immediately notify the sender by replying to this e-mail
> message.
>
>
>
> _______________________________________________
> KinoSearch mailing list
> KinoSearch [at] rectangular
> http://www.rectangular.com/mailman/listinfo/kinosearch

Matthew Berk
Lead Search Architect
t 206.331.3528
f 206.331.3695
matthew [at] openlist

Marchex, Inc.
http://www.marchex.com

This email message and any attachments are solely for intended
recipients, and may contain information that is privileged and
confidential. If you are not the intended recipient, any
dissemination, distribution or copying is strictly prohibited. If you
believe that you may have received this message in error, please
immediately notify the sender by replying to this e-mail message.


marvin at rectangular

Aug 16, 2007, 6:04 AM

Post #3 of 3 (687 views)
Permalink
memory leak in KinoSearch::Util::BitVector (v .15) [In reply to]

On Aug 15, 2007, at 5:00 PM, Matthew Berk wrote:

> Just doing a bit of digging. Would adding sv_2mortal make a
> difference?
>
> PPCODE:
> out_av = Kino_BitVec_to_array(bit_vec);
> /* XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av ))); */
> XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av )));
> XSRETURN(1);

This originally confused me, because the commented-out line is
identical to the one that follows. In the real text, though, the
sv_2mortal is indeed missing.

XPUSHs(newRV_noinc( (SV*)out_av ));

The Valgrind testing I do before each release did not pick this up. :
( Presumably Perl reclaims the leaked elements during global
destruction, at least under PERL_DESTRUCT_LEVEL=2.

Your assessment and remedy are both correct, Matthew. I am swamped
right now, but I will release KS 0.16 with this fix when I get a
moment. Thank you.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

kinosearch discuss RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.