store leaks (drip drip...) 
Author Message
 store leaks (drip drip...)

Robin asks:

>> Gordon Dakin is suffering a store leak on a POP program for vision
>> that we have been writing.
>>  [text deleted]
>> (ii) Are there any tools available to help localise the problem. E.g. one
>> might do a kind of garbage collect analogue to discover how much store
>> was hung on a particular variable.
Steve writes:
> No, nor is it possible, even in principle, for users to write such
> tools. This is a matter of considerable frustration for me.  Even C users
> are (these days) supplied with much better tools for reviewing store use.
> Mind you, they have a greater incentive!

Certainly there are difficulties. But I don't think a useful tool is
-impossible-. The problem of course is not unrelated to garbage-collection,
persistence etc. I must go and jump on a what the Rev. Awbry would doubtless
characterise as a female of dubious inclinations - so more later when I have
had a hack on Mr Twinkle Flipflops, aka my 486.


Fri, 17 May 1996 23:19:36 GMT  
 store leaks (drip drip...)

Well, Steve, I do appreciate your greater experience in worming out the
secrets of where resources go to in Poplog. But what is hidden in procedures
(apart from manifest things like frozvals, updaters)?  -datasize-  presumably
gives the amount of code. Problems seem to be:

(i)   Getting hold of file-lexicals.
(ii)  Internal structure of procedures.
(iii) Devices are opaque
(iv)  Processes and most of the call-stack are opaque.
(v)   XptDescriptors are opaque.
(vi)  External pointers are opaque.

Whether the limited tool below is of any use I don't know. It would help me,
since I memoise functions and forget about them... But would not help
X-related leaks.
       Example of use

: profile_store(200);
vedupperfile  720
emacsescapetable  817
vedbufferlist  76441
sysstring  512
vveddump  5184

This runs on Poplog 14.0 without datasize.

Call profile_store(n) to obtain a listing of words whose dependent store is
larger than n words.
section profile => profile_store;

vars procedure(
    marked = newassoc([]),

define profile_store(n);
  lvars n;
  newassoc([]) -> marked;

define profile_word(W,n); lvars n,W;
  unless W == "marked_profile" or not(isdefined(W))
  then lvars V = valof(W), n1 = size_profile(V);
    if  n1>n  then pr(W); sp(2); npr(n1);

define size_profile(V);
  lvars V; ;;; V -> VV;
  if iscompound(V) and isinheap(V) and not(marked(V))
    and is_profilable(V)
      true -> marked(V);
      lvars n = datalength(V);
      appdata(V, procedure(d); size_profile(d)+n->n endprocedure);
  else 0

define is_profilable(V); lvars V;
     isprocedure(V) or isdevice(V) or isprocess(V)
    or isexternal_ptr(V) or isXptDescriptor(V)



Sat, 18 May 1996 23:55:24 GMT  
 store leaks (drip drip...)
It may seem an obvious suggestion but have you tried using popgctrace?

popgctrace -> BOOL or INT                                     [variable]
BOOL or INT -> popgctrace
        If this variable (default value <false>) is true then a  message
        is output after each garbage collection, giving the GC time  and
        details of memory allocation (the characters of the message  are
        output through -cucharout-).

        The first line of the message has the form

            GC-<code>(X) <t>  MEM: u <u> + f <f> + s <s> = <m>

        where <code> is a 4-letter code indicating why the GC  happened,
        and <t> is the time for the garbage collection in seconds. X  is
        a letter indicating which garbage collection algorithm was  used
        (C = copying, N = non-copying).

        <code> is one of the following:

            auto    The combined area for relocatable structures and the
                    user stack was full.

            fixd    The area for fixed-address structures was full.

            ftbl    The internal table  of open  files/devices was  full
                    when trying to open a  device. (A GC can free  space
                    in  this  table  by  closing  and  removing  garbage

            fopn    The operating-system imposed limit on the number  of
                    open devices  was  reached  when  trying  to  open a
                    device. (By closing garbage  devices, a GC can  make
                    more available.)

            plog    The  combined  area   for  the   Prolog  trail   and
                    continuation stack was full.

            save    GC done by -syssave-.

            lock    GC done by -sys_lock_system-.

            user    Direct call of -sysgarbage- from a user program.

        The MEM figures refer  to totals for all  heap segments: <u>  is
        the number of words used by structures, <f> the number free, <s>
        the number taken by  the user stack, and  <m> the total. If  the
        heap is  locked  with  -sys_lock_heap- then  an  additional  <l>
        figure precedes <u>, i.e.

            GC-<code>(X) <t>  MEM: l <l> + u <u> + f <f> + s <s> = <m>

        where <l> is the total number of words in the locked part of the
        heap (note this includes both used and unused space).

        If -popgctrace- has any non-false  value other than an  integer,
        only  the  first  line  is  output.  Additional  information  is
        obtained by  giving  it  an integer  value,  whose  bits  select
        further message lines as follows:

            Bit     Meaning
            ---     -------
             0      Same as setting  both bits 2  and 3 (for  upward
                    compatibility  in   assigning  1   to  get   the
                    callstack and Prolog information).

             1      Fixed-address structures; prints

                      FIXED: u <u> + fb <fb> + f <f> = <m>

                    where  <u>  is  the  number  of  words  used  by
                    structures,  <fb>   the   number  used   up   in
                    (discontiguous) free blocks,  <f> the number  of
                    (contiguous) free,  and  <m>  the  total.  These
                    values are  included in  the first-line  totals,
                    <fb> and <f> being subsumed together under total
                    words free.
                        (Note   that    space   for    fixed-address
                    structures is  allocated  from  free  blocks  if
                    possible, but because  of fragmentation a  given
                    figure for <fb> doesn't imply that that  much is
                    actually usable in any context.)

             2      Callstack; prints

                      CALLSTACK: <c>

                    where  <c>  is  the  number  of  words  used  by
                    procedure-call stack frames.

             3      Prolog store area; prints

                      PROLOG: trail <t> + free <f> + contn <c> = <m>

                    where <t> is  the number  of words  used by  the
                    trail, <f> the number free, <c> the number  used
                    by the continuation stack, and <m> the total.

Sun, 19 May 1996 03:10:26 GMT  
 [ 3 post ] 

 Relevant Pages 

1. Store leaks (drip drip...)

2. Drip, Glip, and Strip

3. Hot Dripping Sluts...

4. memory leak and leak-fixing 'patterns'

5. StORE/PostgreSQL: Store.DbRegistry update5i2

6. Zombie Objects Leaks With Automatic GC

7. Possible Dolphi R4 memory leak using ODBC

8. GETDSAB and memory leaks

9. How to pinpoint memory leaks

10. TopLink errors and memory leaks

11. memory leak in gawk 3.1.0 ?

12. ENFIN Memory Leak


Powered by phpBB® Forum Software