Memory leak 
Author Message
 Memory leak

Hi,

  Is 'eval' command leaks memory. Is it possible to find out causes of
memory leak in perl program. I am using eval command to about 5000-6000
times.  Each time i evaluate the command in different context. Program
grows to about 100MB from 12MB.

Can anyone give me pointers on how to fix this or understand what's
going on.

Regards,
sreekanth



Fri, 16 Apr 2004 22:56:51 GMT  
 Memory leak
: Hi,

:   Is 'eval' command leaks memory. Is it possible to find out causes of
: memory leak in perl program. I am using eval command to about 5000-6000
: times.  Each time i evaluate the command in different context. Program
: grows to about 100MB from 12MB.

I doubt the "eval"  statement itself is to blame, but the code *inside*
the eval could easily leak memory, (depending on what it does).

To say anything more specific you would need to show the code that
produces the problem.

Also, the file perltoc.pod has a section called "Debugging Perl memory
usage", plus the faq's have various entries about memory usage.



Sat, 17 Apr 2004 06:22:02 GMT  
 Memory leak

Quote:

> Hi,
>   Is 'eval' command leaks memory.

What version of perl, and string or block eval?

Failed string evals leak on most versions of perl. That
was fixed recently, but I don't know whether it's in 5.6.1
or if you need the dev release of perl to get the fix.

                                Dan



Sat, 17 Apr 2004 07:08:11 GMT  
 Memory leak
I seem to have found why program size increases. Can't understand why
this should increase the memory size. Can anybody explain. How can i
use the case3 and still maintain the memory consumption of case1.

Here is sample code. Essentially this is what i do in the main
code(Except while loop which i added for testing). $curobj is object
handle to which i populate the attributes. Within SetAttribute method i
just assign theattribute value pair in hash reference.

  $code = '$curobj->SetAttribute( attrib1 => value1,
                    attrib2  => <abc> <def> <ghi> ,
                    env       => [USER , xyz, PWD, "/home/xyz"]);';

  $package = "package temppack; no strict; use FlowParser";
  while(1){
        $index++;
        eval "$package; $code;";
        &printMemoryConsumed();
  }

sub SetAttribute {
    my $this = shift;

#Sample code for single attrib, val assignment
    $this->{$attrib1} = $val1;  

Quote:
}

Here is  memory consumption comparison

Case1 :
 $package = "package temppack;
 Process memory at the end of 7000 iteration : 13040

Case2 :
 $package = "package temppack; no strict";
 Process memory at the end of 7000 iteration : 24504

Case3 :
 $package = "package temppack; no strict; use FlowParser";
 Process memory at the end of 7000 iteration : 32712

sreekanth



Sun, 18 Apr 2004 10:37:27 GMT  
 Memory leak

Quote:

> I seem to have found why program size increases. Can't understand why
> this should increase the memory size. Can anybody explain. How can i
> use the case3 and still maintain the memory consumption of case1.

You are asking the wrong question.  Once you have discoved a construct
that leaks memory then the only way to avoid memory leak is not to use
that contruct (or at least to avoid using it in loops).

The quantity of memory leaked per iteration depends on what else is
going on in memory constructs that are referenced by whatever is not
getting freed at the right time.

Quote:
> Here is sample code.

[snip]

Way too complicated - and incomplete.

Here is a much simpler way to illustrate a memory leak using a method
call within eval:

my $code = 'main->foo';
sub foo {}
while(1) {
    eval $code for 1..1000;
    system "ps l $$";

Quote:
}

The m{*filter*}is do not use eval()ed strings as a replacement for code
references, especially not in loops.

If you can't eliminate the eval() altogether then at least move it
outside the loop.

my $code = 'main->foo';

sub foo {}
while(1) {
    &$code for 1..1000;
    system "ps l $$";

Quote:
}

--
     \\   ( )
  .  _\\__[oo

 .  l___\\
  # ll  l\\
 ###LL  LL\\


Mon, 19 Apr 2004 03:18:27 GMT  
 Memory leak
[A complimentary Cc of this posting was sent to

Quote:
> Here is a much simpler way to illustrate a memory leak using a method
> call within eval:

> my $code = 'main->foo';
> sub foo {}
> while(1) {
>     eval $code for 1..1000;
>     system "ps l $$";
> }

Hmm?!!!

env PERL_DEBUG_MSTATS=1 perl -wle 'sub F::f{} eval "F->f" for 1..1000'
Memory allocation statistics after execution:   (buckets 4(4)..8184(8192)
   19676 free:   183    64    54    22    11   1   1     2   1 0 0
              451   114    46    49    16
   41032 used:    72    63    72    40     5   7   3    10   0 1 1
               60    56   124    77     9
Total sbrk(): 62408/8:156. Odd ends: pad+heads+chain+tail: 968+732+0+0.

The problem with the initial code is that the guy did not check for
the success of eval.  IIRC, his eval won't even compile...

Ilya



Mon, 19 Apr 2004 08:24:15 GMT  
 Memory leak

Quote:

> The problem with the initial code is that the guy did not check for
> the success of eval.  IIRC, his eval won't even compile...

Which is a guaranteed leak, since failed compilations leak memory.

                                        Dan



Wed, 21 Apr 2004 09:55:07 GMT  
 Memory leak

I fixed leak because of eval statement. Still memory grows although at
slower
rate. Is there any document which describes all possible causes of
memory leak
and any mechanism to detect it.



Wed, 21 Apr 2004 11:36:19 GMT  
 Memory leak

Quote:

> Is there any document which describes all possible causes of
> memory leak and any mechanism to detect it.

Nope. Best you can do is build perl with its own malloc and with debugging
enabled, and do some judicious stats dumping.

Generally speaking memory leaks in perl are due to perl bugs, but not
always.

                                Dan



Thu, 22 Apr 2004 06:29:03 GMT  
 Memory leak

Quote:


> > Is there any document which describes all possible causes of
> > memory leak and any mechanism to detect it.

> Nope. Best you can do is build perl with its own malloc and with debugging
> enabled, and do some judicious stats dumping.

You can also use Devel::Leak to help track things down.

--
Kevin Michael Vail | a billion stars go spinning through the night,

.. . . . . . . . .  | But _in_ you is the presence that
 . . . . . . . . . | will be, when all the stars are dead.  (Rainer Maria Rilke)



Thu, 22 Apr 2004 06:52:14 GMT  
 Memory leak

Quote:

> [A complimentary Cc of this posting was sent to

> > Here is a much simpler way to illustrate a memory leak using a method
> > call within eval:

> > my $code = 'main->foo';
> > sub foo {}
> > while(1) {
> >     eval $code for 1..1000;
> >     system "ps l $$";
> > }

> Hmm?!!!

> env PERL_DEBUG_MSTATS=1 perl -wle 'sub F::f{} eval "F->f" for 1..1000'
> Memory allocation statistics after execution:   (buckets 4(4)..8184(8192)
>    19676 free:   183    64    54    22    11   1   1     2   1 0 0
>               451   114    46    49    16
>    41032 used:    72    63    72    40     5   7   3    10   0 1 1
>                60    56   124    77     9
> Total sbrk(): 62408/8:156. Odd ends: pad+heads+chain+tail: 968+732+0+0.

I have no idea what that's supposed to show.  You can't detect a leak
without trying at least two different numbers of iterations and
comparing the memory usage.  

Or are you simply trying to point out that I could have avoided the
use of system() by recompiling perl with debug options or that I could
have saved a few keytrokes by calling my subroutine F::f rather than
main::foo?

Quote:
> The problem with the initial code is that the guy did not check for
> the success of eval.

No, that a red herring.

Quote:
> IIRC, his eval won't even compile...

We all like to punish people who retype and paraphrase their example
code such that it is no longer a valid example but don't pretend like
you actually believe that the transcription errors were really the
orginal problem.  That way lies Godzilla.

There is a memory leak in method invocation within eval(STRING) in
5.6.0.

--
     \\   ( )
  .  _\\__[oo

 .  l___\\
  # ll  l\\
 ###LL  LL\\



Thu, 22 Apr 2004 18:25:57 GMT  
 Memory leak

Quote:


>> The problem with the initial code is that the guy did not check for
>> the success of eval.
> No, that a red herring.

Not quite a red herring. Failed string evals have leaked for a long
time. All of perl 5 I think, though generally nobody's made much mention
of it. (The yacc-generated stuff leaks on failure IIRC, and fixing that's
always been non-trivial, so nobody took it on until recently)

Quote:
> We all like to punish people who retype and paraphrase their example
> code such that it is no longer a valid example but don't pretend like
> you actually believe that the transcription errors were really the
> orginal problem.  That way lies Godzilla.

Oh, in ways most people don't understand... ;-)

Quote:
> There is a memory leak in method invocation within eval(STRING) in
> 5.6.0.

Yup. That's not the bug the original poster's triggering, though. (And it
*is* a bug in perl)

                                        Dan



Fri, 23 Apr 2004 06:08:06 GMT  
 Memory leak
[A complimentary Cc of this posting was sent to

Quote:
> > env PERL_DEBUG_MSTATS=1 perl -wle 'sub F::f{} eval "F->f" for 1..1000'
> > Memory allocation statistics after execution:   (buckets 4(4)..8184(8192)
> >    19676 free:   183    64    54    22    11   1   1     2   1 0 0
> >               451   114    46    49    16
> >    41032 used:    72    63    72    40     5   7   3    10   0 1 1
> >                60    56   124    77     9
> > Total sbrk(): 62408/8:156. Odd ends: pad+heads+chain+tail: 968+732+0+0.

> I have no idea what that's supposed to show.  You can't detect a leak
> without trying at least two different numbers of iterations and
> comparing the memory usage.  

Of course you can.  The above info shows that no bucket has a count
above 1000.  Thus there is no memory-leak-on-each-iteration.

Quote:
> There is a memory leak in method invocation within eval(STRING) in
> 5.6.0.

This is not what you initially wrote.

env PERL_DEBUG_MSTATS=1 perl5.6.0 -wle 'sub F::f{} for (1..100) {eval "F->f" for 1..100}'
Memory allocation statistics after execution:   (buckets 4(4)..8184(8192)
   15880 free:   183    65    34    12    11   2   1     1   1 0 0
              160   108    38    32    22
  370208 used:    72    62    29    50     5   6   3   297   0 1 1
            10060    62   132    52     3
Total sbrk(): 392280/59:207. Odd ends: pad+heads+chain+tail: 1112+3032+0+2048.

As you see, there is a leak of one 4-byte bucket per iteration.
[Strangely enough, it is not seen without embedded loops.]

Hope this helps,
Ilya



Sat, 24 Apr 2004 02:04:47 GMT  
 
 [ 13 post ] 

 Relevant Pages 

1. PASCAL LISTING PROGRAM NEEDED!!!!

2. memory leak caused by memory fragmentation?

3. Memory limitation & memory leak

4. Memory leak while calling Perl from C?

5. Memory Leak?

6. Memory Leak on Solaris?

7. Memory leak in Perl5...

8. perl5: memory leaks

9. Memory leak using perl 5.000's split() and a decrement

10. Embedding perl into an MFC application causes memory leaks

11. perl push() has memory leak!!?

12. Memory leak in Perl 5.8

 

 
Powered by phpBB® Forum Software