Malloc fails in large program - Help. 
Author Message
 Malloc fails in large program - Help.

I have a Met data decode and display program which I wrote about a
year ago and is in use by many people with no problems at all for
that time. The code is now around 25,000 lines.

Recently I decided to extend the code to more functions, another
couple of thousand lines of code. Only to find that the program failed
in one of the early modules in malloc:-

        if (( buffer = malloc (size)) == NULL)

The particular functions that failed have all worked fine since the
program was finished.

By removing a chunk of code anywhere in any function it started to
work again.

I seem to have hit some sort of limit, where adding more code even
though that code does not call malloc, causes earlier malloc's
to fail.

The compiled program is about 600k in the huge model, using Borland
4.5.

Can anyone help or have I reached a maximum code size.

John



Tue, 17 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:
>I have a Met data decode and display program which I wrote about a
>year ago and is in use by many people with no problems at all for
>that time. The code is now around 25,000 lines.

>Recently I decided to extend the code to more functions, another
>couple of thousand lines of code. Only to find that the program failed
>in one of the early modules in malloc:-

>        if (( buffer = malloc (size)) == NULL)

>The particular functions that failed have all worked fine since the
>program was finished.

>By removing a chunk of code anywhere in any function it started to
>work again.

>I seem to have hit some sort of limit, where adding more code even
>though that code does not call malloc, causes earlier malloc's
>to fail.

>The compiled program is about 600k in the huge model, using Borland
>4.5.

>Can anyone help or have I reached a maximum code size.

Try asking in comp.os.msdos.programmer. C doesn't have maximum code
sizes.

Mike Shannon
Houston, Texas

<!>  Once we had computers and dumb terminals.
<!>  Now we have computers and stupid terminals.
<!>  Thanks, Microsoft.



Wed, 25 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:

>Try asking in comp.os.msdos.programmer. C doesn't have maximum code
>sizes.

While I approve of the sentiment, this is flatly untrue.  There are
minimum maximum code sizes, just as there are minimum maximum integral
values.  Due to an unfortunate flaw in the wording of that section,
a compiler is exempt from any requirement to do anything.  But if you
disregard that, there are still limits on the number of objects,
declarations, etc. that can be around in a program.  Or rather, there
are minimum numbers which a compiler *must* support, and it is
welcome to refuse to support anything more, although in practice
must support significantly more.

For instance, a compiler must be able to handle one object of at least
32,767 bytes.  There is no requirement that it handle, say, a 2 megabyte
array, although I refuse (on principle) to use any which can't.

-s
--

Unix/C Wizard - send mail for help, or send money for consulting!
The *other* C FAQ, the hacker FAQ, et al.  See web page above.
Unsolicited email (junk mail and ads) is unwelcome, and will be billed for.



Thu, 26 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:

...

>>By removing a chunk of code anywhere in any function it started to
>>work again.

>>I seem to have hit some sort of limit, where adding more code even
>>though that code does not call malloc, causes earlier malloc's
>>to fail.

>>The compiled program is about 600k in the huge model, using Borland
>>4.5.

>>Can anyone help or have I reached a maximum code size.
>Try asking in comp.os.msdos.programmer. C doesn't have maximum code
>sizes.
>Mike Shanno

I do not write in the same utopian world as my collegue, I have
encountered that unseen wall on unix,dos and nt systems. You
may consider DLL's, shared mem modules, and of course parallel
processing.


Sat, 28 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:


>>Try asking in comp.os.msdos.programmer. C doesn't have maximum code
>>sizes.

>While I approve of the sentiment, this is flatly untrue. There are
>minimum maximum code sizes, just as there are minimum maximum integral
>values.  Due to an unfortunate flaw in the wording of that section,
>a compiler is exempt from any requirement to do anything.  But if you
>disregard that, there are still limits on the number of objects,
>declarations, etc. that can be around in a program.  Or rather, there
>are minimum numbers which a compiler *must* support, and it is
>welcome to refuse to support anything more, although in practice
>must support significantly more.

Obviously there are practical limits in any implementation, of memory
allocation per process, processes per user, open files per process, etc,
none of which are specified in the C standard.

The original post says "600k in the huge model, using Borland". This pertains
to "memory models" which are a feature of DOS. Intel chips in real mode
operation restrict offset registers to 16 bit values and thus a range of 0 -
65535 bytes beyond some segment address. There are several "models",
from "small" to "huge" in which a program may have varying numbers of
(executable) code and data segments, none of which may exceed 65536
bytes in size. These are implementation specific limits specified by Intel,
not by the C standard.

Mike Shannon
Houston, Texas

<!>  Once we had computers and dumb terminals.
<!>  Now we have computers and stupid terminals.
<!>  Thanks, Microsoft.



Sun, 29 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:


>....

>>>By removing a chunk of code anywhere in any function it started to
>>>work again.

>>>I seem to have hit some sort of limit, where adding more code even
>>>though that code does not call malloc, causes earlier malloc's
>>>to fail.

>>>The compiled program is about 600k in the huge model, using Borland
>>>4.5.

>>>Can anyone help or have I reached a maximum code size.

>>Try asking in comp.os.msdos.programmer. C doesn't have maximum code
>>sizes.

>>Mike Shanno

>I do not write in the same utopian world as my collegue, I have
>encountered that unseen wall on unix,dos and nt systems. You
>may consider DLL's, shared mem modules, and of course parallel
>processing.

<sigh>

Here, on the third planet, implementation specific constraints are defined by
the implementors, not by the C standard.

Mike Shannon
Houston, Texas

<!>  Once we had computers and dumb terminals.
<!>  Now we have computers and stupid terminals.
<!>  Thanks, Microsoft.



Sun, 29 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.

Quote:
>For instance, a compiler must be able to handle one object of at least
>32,767 bytes.  

What do you define as an object? (Novice's question)
is it any data-type (eg, int, char...)?

Quote:
>-s

--
Ted


Sun, 29 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.



Quote:
>>For instance, a compiler must be able to handle one object of at least
>>32,767 bytes.  

>What do you define as an object? (Novice's question)
>is it any data-type (eg, int, char...)?

From the standard:

"3.14 object: A region of data storage in the execution environment, the
 contents of which can represent values. Except for bit-fields, objects are
 composed of contiguous sequences of one or more bytes, the number, order,
 and encoding of which are either explicitly specified or
 implementation-defined. When referenced, an object may be interpreted as
 having a particular type"

There are 3 categories of types in C:

1. Object types. These have a known size at compile time. int, int [10], etc.
   are object types. Anything that you can apply sizeof to has an object
   type.

2. Incomplete type. These don't define a size. E.g. void, int [],
   struct atype (with no declared member list) are all incomplete types.

3. Function types. Fairly obvious, but note that functions are not objects.

--
-----------------------------------------


-----------------------------------------



Sun, 29 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.


Quote:

>Obviously there are practical limits in any implementation, of memory
>allocation per process, processes per user, open files per process, etc,
>none of which are specified in the C standard.

Actually, FOPEN_MAX *is* specified in the C standard; it shall be at least
eight, including the three standard text streams.  Processes are obviously out
of scope, but memory usage is (in large part) described.

Quote:
>The original post says "600k in the huge model, using Borland". This pertains
>to "memory models" which are a feature of DOS. Intel chips in real mode
>operation restrict offset registers to 16 bit values and thus a range of 0 -
>65535 bytes beyond some segment address. There are several "models",
>from "small" to "huge" in which a program may have varying numbers of
>(executable) code and data segments, none of which may exceed 65536
>bytes in size. These are implementation specific limits specified by Intel,
>not by the C standard.

And which, surprisingly enough, *exceed* the requirements of the C standard.

Admittedly, discussion of how to change the limits is totally beyond the scope
of the language.

-s
--

Unix/C Wizard - send mail for help, or send money for consulting!
The *other* C FAQ, the hacker FAQ, et al.  See web page above.
Unsolicited email (junk mail and ads) is unwelcome, and will be billed for.



Mon, 30 Nov 1998 03:00:00 GMT  
 Malloc fails in large program - Help.



Quote:


>>>For instance, a compiler must be able to handle one object of at least
>>>32,767 bytes.  

>>What do you define as an object? (Novice's question)
>>is it any data-type (eg, int, char...)?

>From the standard:

>"3.14 object: A region of data storage in the execution environment, the
> contents of which can represent values. Except for bit-fields, objects are
> composed of contiguous sequences of one or more bytes, the number, order,
> and encoding of which are either explicitly specified or
> implementation-defined. When referenced, an object may be interpreted as
> having a particular type"

>There are 3 categories of types in C:

>1. Object types. These have a known size at compile time. int, int [10], etc.
>   are object types. Anything that you can apply sizeof to has an object
>   type.

>2. Incomplete type. These don't define a size. E.g. void, int [],
>   struct atype (with no declared member list) are all incomplete types.

>3. Function types. Fairly obvious, but note that functions are not objects.

Thanks  Lawrence that has cleared up one "those" definitions (you know
the one's that you come across in a really interesting piece of text,
that you're reading and you have a fair idea what it means but you say
to yourself, "I must remember to look up that exact definition of so-
and-so", but after reading for about two hours you forget to look it up.
So once again thanks!
--
Ted


Mon, 30 Nov 1998 03:00:00 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Help with smaller programs within a larger program

2. CEdit::CharFromPos failing for large text, need help

3. HELP: Malloc() failed..

4. need help with a C program on largest integer

5. HELP: MS Malloc/free wont allow program to run continuosly

6. array and malloc ?(very large memory)

7. Possible bug in MSC malloc (Large Memory Model)

8. Malloc Trouble with Large Memory Model

9. Large malloc problems

10. Large malloc problems

11. Good malloc routine for large chunks

12. How large overhead has malloc got?

 

 
Powered by phpBB® Forum Software