Hi there,

I got a really cute problem with malloc/cute.
situation: I need to allocate the memory for a LOT of objects/structures at
runtime
problem: When I try to use "malloc(1)" / "new char" for about 1000000 times,
it eats up 200MB+ (!) of memory. When I use "malloc (1000000)" / "new
char[1000000]" it uses only 1MB. Am I missing something?

Thanks to anyone who got the patience to explain this to me.

Markus

--

When you allocate a block of memory, malloc/realloc has to store
housekeeping information, so that it knows how much memory to release
when you come to free the block. The simplest approach is to allocate
additional space for a single void *, which points to the next
available block, but more sophisticated schemes may be used to improve
performance. Also, because malloc must return a pointer that is
properly aligned for any data type, the block you are returned may
contain padding.

An overhead of 199 bytes does seem excessive, though. I'd expect
something closer to the order of 2*sizeof(long double)-1. Either

a) You're using debug versions of the memory allocation functions (try
changing your compiler options).
b) Your implementation has been tuned heavily towards performance, at
the expense of memory effiency (again, see if there are any options to
control this).
c) Your implementation is horribly inefficient (get a new one).

-- Mat.
--

I can guess.

1 - malloc() always returns a pointer which is aligned for any data
type.  So if doubles are 8 bytes and must be aligned, the malloc will
actually allocate 8 bytes.

2 - malloc() reserves a hidden area for the block size and some linked
list pointers.

3 - The implementation might use a minimum block size (to avoid
fragmentation?).  This might be the biggest factor.

For some programs, you might not care.  For others, the memory
consumption, or the performance hit of the malloc(), might cause you
to think about a redesign.

--

--

For each allocation with malloc/new there is more memory allocated
then requested, so that some information about the allocated memory
can be stored. This information is used for example when the memory is
freed to free the corrct amount of memory.

Bart v Ingen Schenau
--

Remeber that malloc adds some memory overhead so that it keeps things straight
(like how much to release when you call free). So when you do many mallocs, you
get a lot of overhead. With just a single malloc, you get virtually unnoticed
overhead.

200 bytes seems excessive overhead to me, so I am not sure that is the full
extent of your situation, but it is definately most of it.

- Jim

--

What's the problem?  Did you not like the answers you{*filter*}the last time?

--

__________________________________________________________
Fight spam now!
Get your free anti-spam service: http://www.*-*-*.com/
--

This makes absolute sense.  When you allocate a chunk of memory the chunk is
frequently preceded by header information.  Also, the memory functions will
attempt to allocate memory on double-word boundaries (or larger) to make sure
whatever it is you need pointed to, the CPU won't fault because it's not
aligned properly.  Some (most) CPUs require integers to be at least,
word-aligned, and float double-word aligned.
--
Open Source middleware available at http://pweb.netcom.com/~tgagne
--

says...

This is pretty common.  malloc (and calloc as well as new under C++,
for that matter) normally allocates memory in blocks.  It'll normally
have a minimum amount of memory to allocate at a time, often something
like 16 or 32 bytes.  In addition, it has to store some extra
information to keep track of each block of memory that gets allocated.  
This means that a typical call to malloc will use a minimum of
something like 32 bytes, and possibly more depending on the exact
implementation you're using.

--
    Later,
    Jerry.

The Universe is a figment of its own imagination.
--

What is happening is that when you ask malloc to allocate
you a block of memory it doesn't just give you a piece of
memory of the size asked for. It actually allocates a block
which contains information for garbage collection, this
information takes up the same amount of space irrespective
of the amount of space you asked for. This explains why
calling malloc more times causes more memory to be used.

Hope this clears this up for you.
--

: >
: > I got a really cute problem with malloc/cute.
: > situation: I need to allocate the memory for a LOT of objects/structures at
: > runtime
:
: What's the problem?  Did you not like the answers you{*filter*}the last time?

Be nice.  The last time was about the same time in
comp.lang.c++.moderated.  Check the date here and the X-Original-Date in
clc++m.

For technical reasons crossposts to the two groups do not work well.  It
is also rare to have a question like this which is reasonable for both
groups.  The languages and the folks in the groups are different in at
least as many ways as they are alike.  This multipost makes sense,
although the answers were much the same.

John
--