Good Programming Practices - Avoiding Memory Leaks 
Author Message
 Good Programming Practices - Avoiding Memory Leaks

I'm having a problem with a small memory leak (~20MB/day).  I have traced
the problem back to my Perl scripts.  I was wondering if someone here can
give me pointers on what NOT to do with Perl.  If anyone is willing to
impart their expert Perl wisdom on me, I would be EXTREMELY grateful.

I am using Active State Perl v5.6.1 build 626 on Windows 2000 Server.

I've written a 250 line Perl script and a bunch of smaller scripts that are
~50 lines.  All of these scripts run every five minutes, working together to
extract WMI information from servers (I'm also running MRTG, which could
conceivably be the culprit).

Here are some examples of what I'm using:

- I use a bunch of regular expressions
- I use DBI
- I am reading from one file handle
- I am writing to one file handle
- I use Win32::OLE for getting WMI info.

-------------------------------------------

This is what I've done/know so far:

As far as DBI, I am closing my recordset and database connection with this
code:

$rsh->finish;
$dbh->disconnect;

I'm also closing my file handles.

I'm using transliteration in a couple of places (tr/a-z/A-Z/).

I'm using system() in two places, but I could easily change that.

I'm NOT using strict or declaring variables with "my" (I know this is
probably bad, but could it cause leaks?).

I'm NOT using local().

I don't think I have any circular references, but how can I know for sure?

In a couple of cases I open a file handle and read the contents of the file,
then close the handle.  I unlink the file.  Then I create a new file with
the same name using the same file handle, write to that file, then close the
handle.
-------------------------------------------

I've googled this for a long time, but I still haven't found what I'm
looking for.  I'm still sort of a beginner (maybe an intermediate) with
respect to Perl, so I welcome any criticism you have.

-Kevin



Mon, 03 Oct 2005 19:49:15 GMT  
 Good Programming Practices - Avoiding Memory Leaks

Quote:
> I'm NOT using strict or declaring variables with "my" (I know this is
> probably bad, but could it cause leaks?).

This will cause memory leaks. Any variable you don't declare with "my"
is a package variable. Package variables stick around forever unless
you explicitly get rid of them.


Tue, 04 Oct 2005 04:08:11 GMT  
 Good Programming Practices - Avoiding Memory Leaks

Quote:
>I'm having a problem with a small memory leak (~20MB/day). I have
>traced the problem back to my Perl scripts.
...
>I've written a 250 line Perl script and a bunch of smaller scripts that
>are ~50 lines. All of these scripts run every five minutes, working
>together to extract WMI information from servers (I'm also running
>MRTG, which could conceivably be the culprit).

So, are you saying that these scripts (also the 250-liner) are started
fresh every five minutes (not that they just sleep for five minutes, do
something, and go to sleep again but do not exit)?

If it is so that the scripts are started fresh, the memory leak is
not in Perl - but in your OS. Ok, something you do in Perl might be
triggering the leak, but the leak still is on the OS side - which makes
the Win32::OLE routines the most probable place of problems. You might
try to check this by having the same script, but instead of calling
things through Win32::OLE just return some fixed (or random) data.

And as was mentioned in the other reply, use 'my' for declaring your
variables, and limit them to the smallest possible scope. This also
helps to keep your program(s) maintainable (and so do, after you cross
the initial "pain", turning on warnings and strict). This doesn't help
much in catching memory leaks, but it might catch some other problems
that are waiting to bite you.
--
Wolf  a.k.a.  Juha Laiho     Espoo, Finland

         PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
"...cancel my subscription to the resurrection!" (Jim Morrison)



Tue, 04 Oct 2005 07:12:00 GMT  
 Good Programming Practices - Avoiding Memory Leaks
Thank you so much for the response.  I appreciate your help.

I'm definitely starting the process anew every five minutes.  The task
scheduler runs the script every five minutes.

I am setting up a test server right now.  I will try to get back here in a
day or so and post whether or not disabling Win32::OLE did the trick.

Below is  the main part of a smaller program that uses Win32::OLE.  Notice
the "OleCleanup" sub routine at the end of the script.  That was my initial
attempt at closing the connection.  If returning a static value as you
mentioned prevents the leak, then how do I go back to using Win32::OLE?
I've searched through the man pages for Win32::OLE, but I wasn't able to
come up with anything else.  Any ideas you have would be greatly
appreciated.

-Kevin

------------------------------------------

$locator = Win32::OLE->CreateObject('WbemScripting.SWbemLocator',
\&OleCleanup) or die "Could not access WMI: \n", Win32::OLE->LastError;
$services = $locator->ConnectServer("$Machine", "root/cimv2", "", "")  or
die "ConnectServer failed: \n", Win32::OLE->LastError;
$coll = $services->InstancesOf( "Win32_PerfRawData_PerfOS_Memory" );

foreach $item ( in( $coll ) )
{
  $name = "$item->{Caption}";
  $usage = "$item->{AvailableMBytes}";

Quote:
} # end foreach

print "$usage\n$usage\n";

sub OleCleanup {
  $self = shift;
  $self->Quit;

Quote:
}



Quote:

> >I'm having a problem with a small memory leak (~20MB/day). I have
> >traced the problem back to my Perl scripts.
> ...
> >I've written a 250 line Perl script and a bunch of smaller scripts that
> >are ~50 lines. All of these scripts run every five minutes, working
> >together to extract WMI information from servers (I'm also running
> >MRTG, which could conceivably be the culprit).

> So, are you saying that these scripts (also the 250-liner) are started
> fresh every five minutes (not that they just sleep for five minutes, do
> something, and go to sleep again but do not exit)?

> If it is so that the scripts are started fresh, the memory leak is
> not in Perl - but in your OS. Ok, something you do in Perl might be
> triggering the leak, but the leak still is on the OS side - which makes
> the Win32::OLE routines the most probable place of problems. You might
> try to check this by having the same script, but instead of calling
> things through Win32::OLE just return some fixed (or random) data.

> And as was mentioned in the other reply, use 'my' for declaring your
> variables, and limit them to the smallest possible scope. This also
> helps to keep your program(s) maintainable (and so do, after you cross
> the initial "pain", turning on warnings and strict). This doesn't help
> much in catching memory leaks, but it might catch some other problems
> that are waiting to bite you.
> --
> Wolf  a.k.a.  Juha Laiho     Espoo, Finland

>          PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
> "...cancel my subscription to the resurrection!" (Jim Morrison)



Tue, 04 Oct 2005 15:30:51 GMT  
 Good Programming Practices - Avoiding Memory Leaks

<URL::">
## > I'm NOT using strict or declaring variables with "my" (I know this is
## > probably bad, but could it cause leaks?).
##  
##  This will cause memory leaks. Any variable you don't declare with "my"
##  is a package variable. Package variables stick around forever unless
##  you explicitly get rid of them.

That's not a leak, unless the code has a mechanism to construct
variables with new names.

Note that if you call this a leak, my variables leak *as well*.
When leaving a block, Perl doesn't free up all the memory that
makes a variable, because blocks often get reentered (think of
loops and subroutines), and this saves reallocating the memory.

Abigail
--
$_ = "\112\165\163\1648\141\156\157\164\150\145\1628\120\145"
   . "\162\1548\110\141\143\153\145\162\0128\177"  and &japh;



Tue, 04 Oct 2005 17:31:36 GMT  
 Good Programming Practices - Avoiding Memory Leaks
Just an update on what's been going on...

I set up a test server and started running the program every five minutes.
Even with all the Win32::OLE stuff commented out, the memory leak persists.
I'm losing about 10MB of RAM per day.

I'm going to turn on strict (ugh...) and fix my program.  Maybe that will do
the trick.  I'll let you know.

-Kevin


Quote:

> >I'm having a problem with a small memory leak (~20MB/day). I have
> >traced the problem back to my Perl scripts.
> ...
> >I've written a 250 line Perl script and a bunch of smaller scripts that
> >are ~50 lines. All of these scripts run every five minutes, working
> >together to extract WMI information from servers (I'm also running
> >MRTG, which could conceivably be the culprit).

> So, are you saying that these scripts (also the 250-liner) are started
> fresh every five minutes (not that they just sleep for five minutes, do
> something, and go to sleep again but do not exit)?

> If it is so that the scripts are started fresh, the memory leak is
> not in Perl - but in your OS. Ok, something you do in Perl might be
> triggering the leak, but the leak still is on the OS side - which makes
> the Win32::OLE routines the most probable place of problems. You might
> try to check this by having the same script, but instead of calling
> things through Win32::OLE just return some fixed (or random) data.

> And as was mentioned in the other reply, use 'my' for declaring your
> variables, and limit them to the smallest possible scope. This also
> helps to keep your program(s) maintainable (and so do, after you cross
> the initial "pain", turning on warnings and strict). This doesn't help
> much in catching memory leaks, but it might catch some other problems
> that are waiting to bite you.
> --
> Wolf  a.k.a.  Juha Laiho     Espoo, Finland

>          PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
> "...cancel my subscription to the resurrection!" (Jim Morrison)



Tue, 11 Oct 2005 15:30:31 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. How to avoid memory leaks in perl programs ?

2. Finding memory leaks in a Perl program

3. Perl/Tk memory leak example program.

4. memory leak caused by memory fragmentation?

5. Memory limitation & memory leak

6. Good practice for sub returns?

7. good perl practice?

8. Good Techniques and Practices

9. local *FILENAME within procedures....good practice?

10. Best practices....

11. Morse Code practice program

12. DBI, Error Handling, and accepted programming practices

 

 
Powered by phpBB® Forum Software