Malloc & free acting strange 
Author Message
 Malloc & free acting strange

Hi all,

I am currently trying to nail down a tenacious bug in some code under a
Linux system compiled with gcc.

I have a program that recursively goes through a directory structure,
and scans and modifies each file.

When applied to an individual file, the scan process is successful. When
applied to an entire directory, at some point in the execution of the
scan of the many files, the program stops with a segmentation fault.
This point was the same until I modified a line of code, then it bombed
in a different place.

Further analysis proved that the program bombed at calls to "free()". As
a result, I assumed memory leaks and went through the code looking for
them.

To do this I wrote a malloc and free wrapper which stored some malloc
info while the program ran, eg how many descriptors were outstanding
(add one for each malloc, subtract one for each free), as well as the
peak no of mallocs outstanding.

The bug remained with no mallocs outstanding at the end of the program.

I then added more functionality to the wrapper to actually track the
memory addresses and sizes, and included a magic number at the end of
the alloced block to check for overruns.

All the above tests passed, calls to free still bombed for no readily
explanable reason.

Anyone had problems with the malloc library like this before?

Here is the wrapper code:

/*
 *      L I B R A R Y
 *      = = = = = = =
 *
 *      Memory wrapper routine library
 *
 *
 *      ObJoke:
 *      -------
 *
 *              Two seals walked into a club.
 *
 *
 *
 *      (c) 1997 Graham Leggett
 */

#include "libmalloc.h"

#include <stdlib.h>

/* number of descriptors allocated */
long int libmemc=0;
/* peak number of descriptors allocated */
long int libmemp=0;

/* array of memory descriptors */
#define MAGIC_INT 28752
#define LIBMEM_MAXD 1024
typedef struct
{
        char *addr;
        size_t size;

Quote:
} libmemdesc;

libmemdesc libmemd[LIBMEM_MAXD];

/*
 *      Initialise
 *      ==========
 */
void libmeminit()
{
        /* clear out libmemd array */
        memset(&libmemd,0,sizeof(libmemdesc)*LIBMEM_MAXD);
        return;

Quote:
}

/*
 *      Malloc Wrapper
 *      ==============
 *
 */
void *libmalloc(size_t size)
{
        void *ptr;
        long int x=0;

        ptr=malloc(size+2);

        if(ptr)
        {
                while((libmemd[x]).addr!=NULL) x++;
                if(x==LIBMEM_MAXD)
                {
                        fprintf(stderr,"libmalloc: too many mallocs outstanding
(>%d)",LIBMEM_MAXD-1);
                        exit(0);
                }
                else
                {
                        (libmemd[x]).addr=ptr;
                        (libmemd[x]).size=size;
                        (int)*((int *)(ptr+size))=MAGIC_INT;
                }
                libmemc++;
                if(libmemc>libmemp) libmemp=libmemc;
        }
        else
        {
                fprintf(stderr,"libmalloc: malloc returned NULL.\n");
                exit(0);
        }

        return(ptr);

Quote:
}

/*
 *      Realloc Wrapper
 *      ===============
 *
 */
void *librealloc(void *ptr, size_t size)
{
        long int x=0;

        if(ptr==NULL)
        {
                /* behave like malloc */
                ptr=libmalloc(size);
                return(ptr);
        }
        if(size=0)
        {
                /* behave like free */
                libfree(ptr);
                return(NULL);
        }

        while((x<LIBMEM_MAXD)&&((libmemd[x]).addr!=ptr)) x++;
        if(x==LIBMEM_MAXD)
        {
                fprintf(stderr,"libmalloc: realloc from a non exitstant address.\n");
                exit(0);
        }
        else
        {
                if( (int) *((int *)((libmemd[x]).addr+(libmemd[x]).size)) != MAGIC_INT
)
                {
                        fprintf(stderr,"libfree: magic number killed!! (Memory overrun)\n");
                        exit(0);
                }

        }

        ptr=realloc(ptr,size+2);

        if((ptr==NULL)&&(size!=0))
        {
                fprintf(stderr,"librealloc: realloc returned NULL.\n");
                exit(0);
        }

        /* set new alloced parms */
        (libmemd[x]).addr=ptr;
        (libmemd[x]).size=size;
        (int)*((int *)(ptr+size))=MAGIC_INT;

        return(ptr);

Quote:
}

/*
 *      Calloc Wrapper
 *      ==============
 */
void *libcalloc(size_t nmemb, size_t size)
{
        void *ptr;

        ptr=libmalloc(nmemb*size);

        return(ptr);

Quote:
}

/*
 *      Free Wrapper
 *      ============
 */
void libfree(void *ptr)
{
        long int x=0;

        if(ptr==NULL)
        {
                fprintf(stderr,"libfree: passed NULL pointer to free().\n");
                exit(0);
        }
        else
        {
                while((x<LIBMEM_MAXD)&&((libmemd[x]).addr!=ptr)) x++;
                if(x==LIBMEM_MAXD)
                {
                        fprintf(stderr,"libfree: attempt to free unallocated memory\n");
                        exit(0);
                }
                else
                {
                        if( (int) *((int *)((libmemd[x]).addr+(libmemd[x]).size)) !=
MAGIC_INT )
                        {
                                fprintf(stderr,"libfree: magic number killed!! (Memory overrun)\n");
                                exit(0);
                        }
                        (libmemd[x]).addr=NULL;
                        (libmemd[x]).size=0;
                }
                --libmemc;
        }

        free(ptr);

        return;

Quote:
}

/*
 *      Wrapper Info
 *      ============
 */
void libmeminfo()
{
        printf("libmeminfo: %d malloc pointers outstanding.\n",libmemc);
        return;
Quote:
}



Sun, 28 Nov 1999 03:00:00 GMT  
 Malloc & free acting strange

Quote:

> Hi all,

> I am currently trying to nail down a tenacious bug in some code under a
> Linux system compiled with gcc.

> I have a program that recursively goes through a directory structure,
> and scans and modifies each file.

> When applied to an individual file, the scan process is successful. When
> applied to an entire directory, at some point in the execution of the
> scan of the many files, the program stops with a segmentation fault.
> This point was the same until I modified a line of code, then it bombed
> in a different place.

> Further analysis proved that the program bombed at calls to "free()". As
> a result, I assumed memory leaks and went through the code looking for
> them.

> To do this I wrote a malloc and free wrapper which stored some malloc
> info while the program ran, eg how many descriptors were outstanding
> (add one for each malloc, subtract one for each free), as well as the
> peak no of mallocs outstanding.

> The bug remained with no mallocs outstanding at the end of the program.

> I then added more functionality to the wrapper to actually track the
> memory addresses and sizes, and included a magic number at the end of
> the alloced block to check for overruns.

> All the above tests passed, calls to free still bombed for no readily
> explanable reason.

This is usually the result of writing through a bad pointer, either
because the pointer was never initialized or because it's{*filter*} on to
a value that's already been free()'d. A less common cause is that
somewhere the code attempted to free a block that had never been
allocated, e.g. a variable on the stack. That usually fails immediately,
however.
        -- Pete


Sun, 28 Nov 1999 03:00:00 GMT  
 Malloc & free acting strange


Quote:
>Hi all,

>I am currently trying to nail down a tenacious bug in some code under a
>Linux system compiled with gcc.

[ snip ]

Quote:
>Further analysis proved that the program bombed at calls to "free()". As
>a result, I assumed memory leaks and went through the code looking for
>them.

>To do this I wrote a malloc and free wrapper which stored some malloc
>info while the program ran, eg how many descriptors were outstanding
>(add one for each malloc, subtract one for each free), as well as the
>peak no of mallocs outstanding.

>The bug remained with no mallocs outstanding at the end of the program.

Sounds like a object overrun. Or you could be freeing something twice, and then
not freeing something else. This would make your malloc ``ledger'' look
balanced.

Save yourself a lot of hassles and install ElectricFence. Using the memory
protection facilities found in Linux or UNIX, it will detect most cases of
overrun or the multiple freeing. All you have to do is compile a little library
that replaces malloc and friends.



Sun, 28 Nov 1999 03:00:00 GMT  
 Malloc & free acting strange

Quote:

> Hi all,

> I am currently trying to nail down a tenacious bug in some code under a
> Linux system compiled with gcc.

> I have a program that recursively goes through a directory structure,
> and scans and modifies each file.

> When applied to an individual file, the scan process is successful. When
> applied to an entire directory, at some point in the execution of the
> scan of the many files, the program stops with a segmentation fault.
> This point was the same until I modified a line of code, then it bombed
> in a different place.

> Further analysis proved that the program bombed at calls to "free()". As
> a result, I assumed memory leaks and went through the code looking for
> them.

> To do this I wrote a malloc and free wrapper which stored some malloc
> info while the program ran, eg how many descriptors were outstanding
> (add one for each malloc, subtract one for each free), as well as the
> peak no of mallocs outstanding.

> The bug remained with no mallocs outstanding at the end of the program.

> I then added more functionality to the wrapper to actually track the
> memory addresses and sizes, and included a magic number at the end of
> the alloced block to check for overruns.

> All the above tests passed, calls to free still bombed for no readily
> explanable reason.

> Anyone had problems with the malloc library like this before?

> Here is the wrapper code:

<< code snipped >>

Ah, maybe I'm not going insane after all.  I have a system running on
a Sparcstation, also using the gcc compiler.  (I assume your Linux
system is on a PC ? )  I've had malloc problems as well, although it
was very hard to track down.  We're using another tool, SDT, that takes
graphical SDL (Specification and Description Language) constructs and
then generates C code from that.  To get a view into the actual C code,
I had to run it outside of the SDL environment to find the error.  I
managed to get info down to the actual library call in malloc.  It would
appear as though something in the internally maintained memory
structures
used by the malloc library to keep track of allocated and deallocated
blocks
is getting corrupted, by what, I don't know.  I had added an external C
linked list library to this SDL generated system, which had worked
before, without it.  I also ran tests on the linked list library
separately,
and within a smaller, SDL generated system, and it worked fine.  But,
combine the two, and I get a disaster.

I didn't go to the extent you did, with the wrapper, to track
allocations.
I had inserted a bunch of printf debug code, and just logged all of the,
I thought, offending linked list calls that were added.  They all seemed
to
behave well, but after only a few cycles through the main procesing
loop,
things died.  Since it happens so quickly, its not a matter of exhauting
any resources.  

Your problem sounds very similar, just manifesting itself differently.
The location of my blowup also moved, from the simulator SDL environment
I had originally been testing, to the standalone host application I used
to try to isolate the problem.  

I've basically given up for now.  I've left it as something to deal with
later, while I make progress on other parts of the system.  I'd be
interested in finding what you might find.  Our local gcc tools guru
wasn't much help.

--
=======================================================================

  Tektronix Inc.                            DSTBU Software Engineering
  Measurement Business Division                    
  PO Box 500 Mail Stop: 39-732                 Phone: (503) 627-1357
  Beaverton OR.  97077                         FAX:   (503) 627-5548
=======================================================================



Sun, 28 Nov 1999 03:00:00 GMT  
 Malloc & free acting strange

Quote:

> I am currently trying to nail down a tenacious bug in some code under a
> Linux system compiled with gcc.
> Further analysis proved that the program bombed at calls to "free()". As
> a result, I assumed memory leaks and went through the code looking for
> them.
> Anyone had problems with the malloc library like this before?

I haven't had any trouble like that you describe. You could try
linking with a debugging malloc() like Electric Fence, or a different
libc (I think 5.4.x and glibc 2.0 have a different malloc from
5.3.x). Also try compiling on a different platform, if your code
is portable. A Linux-specific newsgroup would probably be able to
provide more info about known malloc bugs.

Good luck,

--
Edwin



Tue, 30 Nov 1999 03:00:00 GMT  
 Malloc & free acting strange


: > I am currently trying to nail down a tenacious bug in some code under a
: > Linux system compiled with gcc.

: > Further analysis proved that the program bombed at calls to "free()". As
: > a result, I assumed memory leaks and went through the code looking for
: > them.

: > Anyone had problems with the malloc library like this before?

: I haven't had any trouble like that you describe. You could try
: linking with a debugging malloc() like Electric Fence, or a different
: libc (I think 5.4.x and glibc 2.0 have a different malloc from
: 5.3.x). Also try compiling on a different platform, if your code
: is portable. A Linux-specific newsgroup would probably be able to
: provide more info about known malloc bugs.

: Good luck,

: --
: Edwin

        problem may originate with attempt
        to
                free pointer
        not currently
        ^^^^^^^^^^^^^
        used for allocation;
        ( at given point in program run )
        !

--

Ralph Silverman



Mon, 06 Dec 1999 03:00:00 GMT  
 Malloc & free acting strange

[his usual cryptic reply]

Here we post again into the wide world of programming thanks to the
courtesy of Mr. Ralph Silverman.

He seems to think that his posts are so important that they should not
be limited to one newsgroup.

Stephan



Tue, 07 Dec 1999 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. malloc & free in c#

2. malloc&free

3. simple question about malloc & free

4. malloc/calloc & free

5. if acting strange

6. Strange way CStatusBar acts with CDialogBar

7. Changing CEdit font is acting very strange

8. CCriticalSection acting strange??..

9. HELP! Tree Control Acts Strange After Many Selects

10. Act X & FireEvent

11. Act X & 2 interfaces

12. memory to free or not to free, to malloc or not to malloc ?

 

 
Powered by phpBB® Forum Software