sizeof(struct) and padding 
Author Message
 sizeof(struct) and padding

Some time ago the issue of the value of sizeof(S) where S is
a structure requiring alignment came up in this news group.
It turns out that sizeof(S) will include not only all the
content of S. It also counts any padding necessary to ensure
the allignment of another S, even if S is not declared as
part of an array. This is good if your usage is:

   s_array = malloc(n*sizeof(S)); /* Where s_array is a pointer
                                     to type S */

But in other cases it is not so good. If sizeof() is being used
to get the number of bytes for memcpy() the padding bytes will
be copied unnecessarily. If space is being allocated for a single
structure extra memory may be allocated, but this probably won't
make any difference since the space for the padding would usually
be wasted anyway.

"Big deal!" you say? Actually it could make an important differnce
if the structure is being written as a record to a file. Then  extra
file space will be used. Unless I intend to read or write arrays of
records some of the time, changing the number of records handled,
with each operation, this is a real waste.

The $64K question is: How do I portably get the actual size of
a structure without padding when I need that rather than
what sizeof() tells me? I have some methods that will work,
but I am not sure how portable they'd be and I hope someone
has a better way.




Wed, 07 Apr 1993 02:46:14 GMT  
 sizeof(struct) and padding

Quote:
> The $64K question is: How do I portably get the actual size of
> a structure without padding when I need that rather than
> what sizeof() tells me?

You don't.

If you want to add up the sizes of the structure elements, do that.
Write each one to disk individually, and take compactness over speed.

To put it differently, if you want to write a data structure to a file,
you should respect its rights as a *structure*, rather than treating it
as a byte array. As you observed, byte arrays are fine for some things,
like allocating memory, but not for others.

---Dan



Wed, 07 Apr 1993 05:07:13 GMT  
 sizeof(struct) and padding

[sizeof() reflects padding at ends of structures.]

Quote:
>"Big deal!" you say? Actually it could make an important differnce
>if the structure is being written as a record to a file. Then  extra
>file space will be used.
>The $64K question is: How do I portably get the actual size of
>a structure without padding when I need that rather than
>what sizeof() tells me?

My $64,000 question is: why are so many poor souls condemned to
try to read and write binary data files?  Yes, it is easy and
quick to use fwrite and sizeof to write whole structs to a file,
but those are the only two good things which can be said about it.
Binary files are otherwise quite difficult to work with, not to
mention unportable, sometimes even to the same machine on which
they were written.  More flexible (e.g. text) file formats offer
hosts of advantages, and can be implemented reasonably easily and
efficiently as well.  (This debate appears here endlessly, and I
should add it to the FAQ if I could think of a succinct way.  I
won't repeat all the pros and cons right now.)

If a binary format has not already been forced upon you by some
prior decision or external piece of software (which it sounds
like it has not, since you are trying to figure out how to change
the format by eliminating the padding you feel is spurious) you
should strongly consider dropping binary formats entirely, and
moving to a simple text format.  I'd describe good ways to do so,
but I recently discovered that everything I'd say about data
files has already been published, in Jon Bently's book
Programming Pearls (or perhaps the sequel, More Programming
Pearls).

If you must compute the size of a structure less any trailing
padding, one way to do so would be

        struct x x;
        offsetof(struct x, last_member) + sizeof(x.last_member)

offsetof is a (relatively) new macro, standardized by ANSI, and
#defined in <stddef.h>.

(By the way, it is generally agreed that the advantages of having
sizeof account for trailing padding far outweigh any disadvantages.
Yet this issue apparently belongs in the FAQ as well.)

                                            Steve Summit



Wed, 07 Apr 1993 08:32:22 GMT  
 sizeof(struct) and padding

Quote:

>My $64,000 question is: why are so many poor souls condemned to
>try to read and write binary data files?

  My $0.02 answer is: Using a binary (or other fixed-length) format
makes random access easy; and allocating records large enough to
handle the largest textual representation of the data may be
impractical.  (OK, so you could do the equivalent of uuencode on the
data and it wouldn't blow up too much.)

  For sequential files where space and speed aren't issues, text
formats are great; otherwise, a binary format is probably good.

  All IMHO, of course.

        Anton

+---------------------------+------------------+-------------+

+---------------------------+------------------+-------------+



Thu, 08 Apr 1993 06:36:15 GMT  
 sizeof(struct) and padding

Quote:
> "Big deal!" you say? Actually it could make an important differnce
> if the structure is being written as a record to a file. Then  extra
> file space will be used. Unless I intend to read or write arrays of
> records some of the time, changing the number of records handled,
> with each operation, this is a real waste.
> The $64K question is: How do I portably get the actual size of
> a structure without padding when I need that rather than
> what sizeof() tells me?

There is no such thing as "the actual size of a structure without padding".
Much of the padding required for alignment is likely to be within the
record.  Consider
        struct foo { char a; double d; short s; void *p; };
                            ^                  ^
I've marked the two places that padding is likely to go on a machine that
requires 32-bit alignment for doubles and data pointers.  They are *inside*.

Suppose we could squeeze the padding out.  The size of the record with
padding is 20 bytes.  Without padding it'd be 15.  Big deal.  If you
order the fields in a struct thus:
        first long double
        then  double
        then  float
        then  pointer-to-function
        then  pointer
        then  long
        then  int
        then  short
        last  char
then all the padding required for alignment is likely to be at the end,
and on a 32-bit byte-addressed machine the total amount of padding is
likely to be at most 3 bytes.  This isn't going to be perfect on all
machines, but it's pretty good for most.  (It's a generalisation of the
ordering prudent fortran programmers use within COMMON blocks.)

If you are only writing one or two records of a particular type to a
file, why do you care about the padding?  If you are writing thousands,
then you may well want to read and write arrays of them.

If it is absolutely essential that you write your records in the most
compact form, then don't use ``raw'' fwrite and fread.  Write your own

        int fwrite_foo(FILE *f, struct foo *p) { ... }
        int fread_foo(FILE *f, struct foo *p)  { ... }

functions that write/read the fields separately.

But above all, keep in mind that in C, there is no such thing as "the
actual size of a structure without padding".  "sizeof" is as actual
as it ever gets.

--
Fear most of all to be in error.        -- Kierkegaard, quoting Socrates.



Fri, 09 Apr 1993 09:07:49 GMT  
 sizeof(struct) and padding

Quote:
> Many of us use binary files because our OS does not differentiate
> between text and binary files.

I'm sorry, but this is just so totally wierd.  I can understand the
reasoning that goes "this operating system forces me to distinguish
between binary and text, and this is one of the cases where it wants
binary, so I'm stuck with binary".  But I don't see why an operating
system that doesn't force you to use a binary representation is a
reason for doing so anyway.

Quote:
> My Suns at school do not even have the ability to choose which one to
> open -- look at my recent question about reading/writing binary files...

Oh come on.  The Suns at your school *do* give you the choice of opening
a file for binary transput or opening it for text transput.  Either way,
you write exactly the same code.  This is like complaining that your
local milk company doesn't let you choose whether to drink from the bottle
or out of a glass because it gives you exactly the same bottle in both
cases.

The realistic way to put it is that UNIX-based systems like the Suns
let you have *any* mix of text and binary within the same file.  You
can, for example, put source code (text), object code (binary), and
archives (structured) into archives (.a files, managed by the 'ar'
command).  This is a facility that operating systems which distinguish
between text and data *deny* you.

Stop and think about it.  The ability to mix text and binary in one file
gives us the best of both worlds.  Suppose we structure a file like this:
        <directory>
        <text version>
        <binary version>
where the <directory> is
        <binary edition> <pointer to binary version>
and <binary edition> is some magic tag that lets you determine whether
the binary version is appropriate for this machine&os&compiler&release.
If it is, the <pointer to binary version> is a decimal integer or integers
encoding the address of the binary version, you just go there and slurp it
up.  If the binary version isn't appropriate, you read the text version,
rewrite the binary version, and update the header.  That way you get the
speed benefit of reading a binary version, and the portability benefit of
having a text version, and you've still got only one file to worry about.

What makes this possible?  *NOT* distinguishing between text and binary
in the os.

Quote:
> So, I'd just as soon stick with my binary files...might as well with binary
> data.

You'd better stick with your machine too...

--
Fear most of all to be in error.        -- Kierkegaard, quoting Socrates.



Fri, 09 Apr 1993 09:34:46 GMT  
 sizeof(struct) and padding

Quote:

> The $64K question is: How do I portably get the actual size of
> a structure without padding when I need that rather than
> what sizeof() tells me?

You don't. When you write structures directly to disc, it is non-portable,
period -- whether or not you write the tail-padding. Different machines will
have different internal padding, even different data formats. For most
kinds of data, text (ASCII or EBCDIC) is the perfect solution. Use fprintf()
or get fancy with a library package. I think there are some public domain
ones. Somebody will also mention Sun's XDR, which may work well for you,
particularly if your machine uses the same data-representations that Sun
machines do.


Mon, 12 Apr 1993 11:49:44 GMT  
 sizeof(struct) and padding

Quote:

> When you write structures directly to disc, it is non-portable,
> period -- whether or not you write the tail-padding. Different machines will
> have different internal padding, even different data formats.

Sometimes you have to manually swap memory to disk for efficiency.
You're never going to use the temporary file on another machine, so the
format doesn't have to be portable. All you need is a guarantee from the
OS that memory can be written out and read back in without corruption.

I agree that the original problem is best solved with a text format.

---Dan



Tue, 13 Apr 1993 04:41:39 GMT  
 sizeof(struct) and padding

<The $64K question is: How do I portably get the actual size of
<a structure without padding when I need that rather than
<what sizeof() tells me? I have some methods that will work,
<but I am not sure how portable they'd be and I hope someone
<has a better way.

What you need is to turn off the alignment for a struct. There is no
portable way to do it. Compilers typically provide a command line switch
or a pragma to turn off alignment.

If you want to guarantee that alignment is off for a particular struct,
try this:

        struct ABC { int a; char c; };
        ...
        assert(sizeof(struct ABC) == sizeof(int) + sizeof(char));



Sun, 18 Apr 1993 16:31:56 GMT  
 
 [ 11 post ] 

 Relevant Pages 

1. struct padding and sizeof

2. C++ 5.0 and sizeof (struct) / padding ?

3. sizeof (struct) ! = sizeof struct elements

4. SizeOf Misreporting Struct w/ Bitfield, Union, and Nested Struct

5. padding in a statically initialized struct

6. Padding in structs

7. padding a struct to a defined size

8. padding in structs..

9. padding in structs

10. Q: howto prevent struct padding

11. create a padded struct?

12. preventing struct padding

 

 
Powered by phpBB® Forum Software