Strange byte order using fread() and fwrite() 
Author Message
 Strange byte order using fread() and fwrite()

Apologies if I am stating the obvious here, but having scoured FAQ's and
reference books, I'm none the wiser.

First of all assume the following: int's are 16bit char's are 8bit. If you
are unhappy with this, just use a bucket load of sizeof's and other fancy
checks, it's the principle I want to understand, not the syntax!

1)You have an integer, you break it up into high and low bytes (use masking
and bit shifting).

2)The low byte is saved as element 0, the high as element 1.

3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

4)fread(ptr, 2, 1, fp) - the same file is read as a single int

5)display the int and it's the same as the original number, Uh???

Now, I thought Intel's stored bytes high:low, so what is happening here?
Surely the standard says how fread&write mess with numbers, so am I right in
thinking these functions are supposed to fiddle the byte order? Considering
how this would be a tricky thing to make portable, why bother? I would love
to know why the above happens, and I am convinced it's to do with the
_standard_ function I'm using....any opinions?
___________________________________

ICQ UIN : 997147 'Billee'



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Wild Bill said:

Quote:
>Apologies if I am stating the obvious here, but having scoured FAQ's and
>reference books, I'm none the wiser.

That's your problem.  You are scouring the books when you are supposed to be
_reading_ them.  After that, make sure that you understand them.  If not go
back to the read step.

Quote:
>First of all assume the following: int's are 16bit char's are 8bit.

I refuse on both counts.  The size of an int is sizeof(int) and the number of
bits in a char is CHAR_BIT.  To assume otherwise is just plain stupid.

Quote:
>If you
>are unhappy with this, just use a bucket load of sizeof's and other fancy
>checks, it's the principle I want to understand, not the syntax!

I use lots and lots of compilers every day.  The one I like the least and use
the least often does agree with the 16 bit int crap, but none of the others do.
 If we want to go by 'bucket load of sizeof' for some kind of voting system,
then 16 bit integer loses.

Quote:
>1)You have an integer, you break it up into high and low bytes (use masking
>and bit shifting).

Ok.  But I won't assume two bytes.

Quote:
>2)The low byte is saved as element 0, the high as element 1.

Ok.  And so forth, up to the actual size.

Quote:
>3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

I refuse.  I won't assume two bytes.  I will write sizeof int bytes to the
file.

Quote:
>4)fread(ptr, 2, 1, fp) - the same file is read as a single int

See above.

Quote:
>5)display the int and it's the same as the original number, Uh???

Shift and mask to put it back together.  That is probably what you don't
understand.

Quote:
>Now, I thought Intel's stored bytes high:low, so what is happening here?
>Surely the standard says how fread&write mess with numbers, so am I right in
>thinking these functions are supposed to fiddle the byte order? Considering
>how this would be a tricky thing to make portable, why bother? I would love
>to know why the above happens, and I am convinced it's to do with the
>_standard_ function I'm using....any opinions?

If you want to read and write integers, DO NOT BREAK THEM INTO CHUNKS.  Sorry
for screaming, but I had a bur under my saddle.  Just write them out.  If you
want it to be portable to other architectures, write them as formatted text and
you will be able to read them anywhere.

--
C-FAQ ftp sites: ftp://ftp.eskimo.com ftp://rtfm.mit.edu
Hypertext C-FAQ: http://www.eskimo.com/~scs/C-faq/top.html
C-FAQ Book: ISBN 0-201-84519-9.
Want Software?  Algorithms?  Pubs? http://www.infoseek.com



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()


Quote:
>If you want to read and write integers, DO NOT BREAK THEM INTO CHUNKS.
>Sorry for screaming, but I had a bur under my saddle.  Just write them
>out.  If you want it to be portable to other architectures, write them
>as formatted text and you will be able to read them anywhere.

Except for those pesky machines with wierd CHAR_BIT sizes.  :-)

--

http://www.cs.wustl.edu/~jxh/        Washington University in Saint Louis

Quote:
>>>>>>>>>>>>> I use *SpamBeGone* <URL:http://www.internz.com/SpamBeGone/>



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

: Apologies if I am stating the obvious here, but having scoured FAQ's and
: reference books, I'm none the wiser.

: First of all assume the following: int's are 16bit char's are 8bit. If you
: are unhappy with this, just use a bucket load of sizeof's and other fancy
: checks, it's the principle I want to understand, not the syntax!

: 1)You have an integer, you break it up into high and low bytes (use masking
: and bit shifting).

: 2)The low byte is saved as element 0, the high as element 1.

: 3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

: 4)fread(ptr, 2, 1, fp) - the same file is read as a single int

: 5)display the int and it's the same as the original number, Uh???

: Now, I thought Intel's stored bytes high:low, so what is happening here?
: Surely the standard says how fread&write mess with numbers, so am I right in
: thinking these functions are supposed to fiddle the byte order? Considering
: how this would be a tricky thing to make portable, why bother? I would love
: to know why the above happens, and I am convinced it's to do with the
: _standard_ function I'm using....any opinions?

There are two ways of ordering high and low bytes, and you've hit the
right one (for Intel).  That's really all there is to it.  Try writing
an int to a file as an int, and compare the byte order with writing the
int as two characters (use debug, or od, or whatever).  You'll find the
byte orders match.  If you reverse the order of writing the characters,
then you'll find that you've messed up the int value.  Intel is little-
endian, least significant byte has lowest address; it's therefore easier
for it to read numbers low-byte first, which is what it's doing here.

Will



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:

> Apologies if I am stating the obvious here, but having scoured FAQ's and
> reference books, I'm none the wiser.

Hi Will Scott,

That's very sad. I was a lot wiser after having read the comp.lang.c
FAQ.

Quote:
> First of all assume the following: int's are 16bit char's are 8bit. If you
> are unhappy with this, just use a bucket load of sizeof's and other fancy
> checks, it's the principle I want to understand, not the syntax!

It is still a very risky assumption, especially the 16 bit "int"s.
(They are mostly 32 bits nowadays)

Quote:
> 1)You have an integer, you break it up into high and low bytes (use masking
> and bit shifting).

I assume that you mean "byte" in the sense of the C standard, ie.
"char" (which in your example corresponds to 8 bits).

Quote:
> 2)The low byte is saved as element 0, the high as element 1.

Saved where ? This is a vital piece of information. I will assume that
you use an array of 2 "char"s.

Quote:
> 3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

Are you writing the "int" or the array of 2 chars here ? Again I will
assume that you are talking about the array of 2 chars.

Quote:
> 4)fread(ptr, 2, 1, fp) - the same file is read as a single int

If this is supposed to be different from the "fwrite()" above, you
should betetr use a different pointer to emphasise the difference.

Quote:
> 5)display the int and it's the same as the original number, Uh???

> Now, I thought Intel's stored bytes high:low, so what is happening here?

You should have posted a short but complete code fragment to illustrate
your problem. Based on your presentation I can only assume that you
actually write and read the same thing, ie. an integer. Your use of
"ptr" indicates that. Or maybe there is a subtle bug in your way of
masking out the single bytes. Please do post a complete little "main()"
(preverable in ANSI-C) for a more thorough analysis.

Quote:
> Surely the standard says how fread&write mess with numbers, so am I right in
> thinking these functions are supposed to fiddle the byte order?

No you are not really right. You see "fread()" and "fwrite()" do not
know about numbers and number formats. They write *memory contents*
to file and read them back to memory. The processor or compiler
decides how various data types are stored in memory. The exact
replica of this memory storage layout is written/read.

Quote:
> Considering how this would be a tricky thing to make portable, why
> bother? I would love to know why the above happens, and I am
> convinced it's to do with the
> _standard_ function I'm using....any opinions?

We are talking about the representation of standard data types in
memory here. This is absolutely *compiler and system specific* !
The standard only defines value ranges of the variaous data types.
The implementation is up to the compiler and there are LOT'S of
different implementations for different hardware architectures.
Sometimes you will even find differing methods of storing numbers
in memory for the very same processor.

The only really portable way of writing data to file and read it back
is to use *text* files.

You can get the FAQ at http://www.eskimo.com/~scs/C-faq/top.html or
at ftp://rtfm.mit.edu/pub/usenet/comp.lang.c/C-FAQ-list and it gets
posted to this newsgroup and to news.answers regularly (at the
beginning of each month).

Stephan
(initiator of the campaign against grumpiness in c.l.c)



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:

>First of all assume the following: int's are 16bit char's are 8bit. If you
>are unhappy with this, just use a bucket load of sizeof's and other fancy
>checks, it's the principle I want to understand, not the syntax!

>1)You have an integer, you break it up into high and low bytes (use masking
>and bit shifting).

>2)The low byte is saved as element 0, the high as element 1.

>3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

>4)fread(ptr, 2, 1, fp) - the same file is read as a single int

>5)display the int and it's the same as the original number, Uh???

>Now, I thought Intel's stored bytes high:low, so what is happening here?
>Surely the standard says how fread&write mess with numbers, so am I right in
>thinking these functions are supposed to fiddle the byte order? Considering
>how this would be a tricky thing to make portable, why bother? I would love
>to know why the above happens, and I am convinced it's to do with the
>_standard_ function I'm using....any opinions?

No, Intel x86 processors store multi-byte values with the lower
bytes first.

Thus

int main(void)
{
  char *cp;
  int i;
  FILE *input_file;
  FILE *output_file;

  i = 1234;
  cp = (char *) &i;
  /* ... */
  fwrite(&i, sizeof(int), 1, output_file);
  /* ... */
  fread(cp, 1, 2, input_file);
  printf("%d\n", i);

Quote:
}

(with appropriate opens, closes, and error checking) will give
you your original number back.  Note that this is non-portable,
because some processors *do* store values with the high bytes
first.  In that case the byte swapping you were expecting may
well take place.

As a general rule, if your fread() calls don't match your
fwrite() calls, your program is probably broken.  Nonsense like
that above can be fun and educational, but it has no place in
professional software.

                              Marc Leonard



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:


>>If you want to read and write integers, DO NOT BREAK THEM INTO CHUNKS.
>>Sorry for screaming, but I had a bur under my saddle.  Just write them
>>out.  If you want it to be portable to other architectures, write them
>>as formatted text and you will be able to read them anywhere.

>Except for those pesky machines with wierd CHAR_BIT sizes.  :-)

If written in binary, they are only definitely readable on the same machine
with the same compiler.  If we write them out as text strings, there might
possibly be a problem with some character sets, but the the number of bits in
char bit will be irrelevant.  There may be a problem of data types if one
machine has enormous ints and the seconds has smaller ones.  It might even be
extremely difficult to overcome if one machine has a native int of 64 bits,
and the second has 16 bit ints and no [gag... cough...] __int64 or long long
data type.  In such a case, if you wanted portable results, you would have to
resort to extended precision subroutines.
--
Hypertext C-FAQ: http://www.eskimo.com/~scs/C-faq/top.html
C-FAQ ftp: ftp://rtfm.mit.edu, C-FAQ Book: ISBN 0-201-84519-9
Try "C Programming: A Modern Approach" ISBN 0-393-96945-2
Want Software?  Algorithms?  Pubs? http://www.infoseek.com


Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:
>No, Intel x86 processors store multi-byte values with the lower
>bytes first.

Thank you, I had been misinformed.

Quote:
>Nonsense like that above can be fun and educational, but it has
>no place in professional software.

I deliberately did not state the application of this code. It is indeed a
nonsense, but it is also the way that soundblaster cards output thier data,
[low][high]. This is not, AFAIK, debatable, so anyone writing code for them
must take counter measures. In my case, it seems I have to do nothing, data
is automatically stored in the correct order.

The above is the off topic part, I believe my question was fair in the way
that it appeared to hinge (to me at least) on how fread & fwrite intepret
data. Now I know that it simply reads bytes, and that Intel store
[low][high] - I stand corrected.

There have been a number of remarks on the 16bit int I use. Am I really so
alone? I use 2 Borland compilers, and DJGPP, only the latter does not have
16bit int's. My degree course looks at industrial applications of computer
control where smaller int's are obviously favourable. Am I to believe that
Borland are out on a limb on this? BTW, I thought the remarks were getting a
little personal, never judge a man by the size of his int's ;-)
___________________________________

ICQ UIN : 997147 'Billee'



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

[...]
: There have been a number of remarks on the 16bit int I use. Am I really so
: alone? I use 2 Borland compilers, and DJGPP, only the latter does not have
: 16bit int's. My degree course looks at industrial applications of computer
: control where smaller int's are obviously favourable. Am I to believe that
: Borland are out on a limb on this? BTW, I thought the remarks were getting a
: little personal, never judge a man by the size of his int's ;-)

I tend to feel that the value of sizeof(int) is something that the programmer
shouldn't want to know; or at least, shouldn't want to keep in the front of
their mind.  But then I write a lot of my code for MSDOS and Unix, which keeps
me honest in this regard.  Embedded systems still use a lot of 8/16 bit
processors, too.

Will



Fri, 04 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()


says...

Quote:

>Now, I thought Intel's stored bytes high:low, so what is happening here?

  You thought wrong.  And the fact that you mention Intel should clue you that
fwrite/fread with types of different sizes is, just like casting with such
types, subject to vagaries of the particular implementation.

David G



Sat, 05 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:

>I deliberately did not state the application of this code. It is indeed a
>nonsense, but it is also the way that soundblaster cards output thier data,
>[low][high]. This is not, AFAIK, debatable, so anyone writing code for them
>must take counter measures. In my case, it seems I have to do nothing, data
>is automatically stored in the correct order.

>The above is the off topic part, I believe my question was fair in the way
>that it appeared to hinge (to me at least) on how fread & fwrite intepret
>data. Now I know that it simply reads bytes, and that Intel store
>[low][high] - I stand corrected.

>There have been a number of remarks on the 16bit int I use. Am I really so
>alone? I use 2 Borland compilers, and DJGPP, only the latter does not have
>16bit int's. My degree course looks at industrial applications of computer
>control where smaller int's are obviously favourable. Am I to believe that
>Borland are out on a limb on this? BTW, I thought the remarks were getting a
>little personal, never judge a man by the size of his int's ;-)

Yes, people in this newsgroup sometimes like to believe that no
legitimate C code can do anything which is hardware-dependent or
OS-dependent.

Fortunately, real-world programming is not like that.  The
hardware-dependent stuff is the fun stuff.  In my opinion,
questions relating to "what is the best way to write C code
which meets the data requirements of this device" are on-topic
in this newsgroup.  (Not to be confused with questions such as
"how do I make my SoundBlaster play a .WAV file").

I would recommend declaring your variables to be "short int" if
you need 16 bits.  As the purists are sure to point out, short
does not guarantee any particular size.  However, in my
experience (with 2 Borland compilers and DJGPP), short gives you
16 bits even when int is 32 bits.  If you want to protect these
assumptions against future compilers, put something like the
following somewhere in your program:

#if CHAR_BITS * sizeof(short int) != 16
  #error Type size trouble!  This program needs 16-bit integers.
#endif

This at least tells a future programmer at compile-time that
something is amiss.  It is up to him to figure out how to fix it
:-).

Another approach is to use typedefs to emphasize the requirement
for a particular bit size.  E.g.

typedef signed char int8;
typedef short int int16;
typedef long int int32;

This helps identify places in your code where a particular size
is critical, not just suggested.

Good luck.

                               Marc Leonard



Sat, 05 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:


>>There have been a number of remarks on the 16bit int I use. Am I really so
>>alone?
>Yes, people in this newsgroup sometimes like to believe that no
>legitimate C code can do anything which is hardware-dependent or
>OS-dependent.

Who says that?  I don't believe anyone says so.  For example, nearly
everyone here programs with the assumption that sizeof (char) is less
than sizeof (int) on an everyday basis.

Quote:
>Fortunately, real-world programming is not like that.  The
>hardware-dependent stuff is the fun stuff.  In my opinion,
>questions relating to "what is the best way to write C code
>which meets the data requirements of this device" are on-topic
>in this newsgroup.  (Not to be confused with questions such as
>"how do I make my SoundBlaster play a .WAV file").

It depends on how the question is worded.

(1) For any substantial problem, there is almost never a "best way" to
write a C program that solves it.  There are always alternative ways,
and each different way has its advantages and disadvantages.

(2) A question that has "meets the data requirements of this device" is also
vague.  If the device is say a mouse, then the answer would depend on how
the mouse is interacting with the system (e.g., bus, serial, parallel, or
other), and what hardware and OS the mouse is interacting with (e.g.,
Linux-FreeBSD-Win32-DOS-OS/2-BSD-Solaris on PC architecture,
Linux-AU/X-MacOS-WinNT on Mac/PowerPC architecture, BSD-Solaris-SunOS on
Sparc architecture, IRIX-BSD on mips architecture, BSD-VMS on VAX, and many
others).

There are just too many answers to give, and so it has been generally
accepted that the best place to receive an answer would be a newsgroup
that specializes to the platform the person asking the question is
using.

It is possible for the question to be worded in a way that makes the
question topical for this newsgroup however.

Examples of good questions:

(1) I am trying to use the blah () system call on my computer.
It has the prototype

       int blah (void *buf, struct tm *timeout);

Where timeout is some time in the future.  How would I set timeout to be 10
seconds into the future?

(2) The manual for my FOO device says that to read and write data to it
from my machine, I have to read a byte from address 0xF0934E and write a
byte to address 0xE234AB.  How do I do this?

Examples of bad questions:

(1) I am trying to use the blah () system call on my computer.
It has the prototype

       int blah (void *buf, struct tm *timeout);

But, when I call it, it doesn't do what the documentation says it should.
What did I do wrong?

(2) I know lots of people have a FOO device on their computer.  My
question is:  How do I directly control the FOO device from a C program?
I know that I can do it by using the program that come with it.  It lets
me read bytes from it and write bytes to it.  Does this help?  Thanks in
advance.

--

http://www.cs.wustl.edu/~jxh/        Washington University in Saint Louis

Quote:
>>>>>>>>>>>>> I use *SpamBeGone* <URL:http://www.internz.com/SpamBeGone/>



Sat, 05 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

[snip]

Quote:
>I would recommend declaring your variables to be "short int" if
>you need 16 bits.  As the purists are sure to point out, short
>does not guarantee any particular size.  However, in my
>experience (with 2 Borland compilers and DJGPP), short gives you
>16 bits even when int is 32 bits.  If you want to protect these
>assumptions against future compilers, put something like the
>following somewhere in your program:

>#if CHAR_BITS * sizeof(short int) != 16
>  #error Type size trouble!  This program needs 16-bit integers.
>#endif

.... if all the compilers you use, or will ever use, can evaluate
sizeof in the pre-processor.

Will



Sun, 06 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()

Quote:


>>#if CHAR_BITS * sizeof(short int) != 16
>>  #error Type size trouble!  This program needs 16-bit integers.
>>#endif
>.... if all the compilers you use, or will ever use, can evaluate
>sizeof in the pre-processor.

One could use USHORT_MAX instead, but this doesn't mean much if the
machine decides to use more bytes to represent a short than it will
actually use for values.  So, it offers little protection against byte
order assumptions.

--

http://www.cs.wustl.edu/~jxh/        Washington University in Saint Louis

Quote:
>>>>>>>>>>>>> I use *SpamBeGone* <URL:http://www.internz.com/SpamBeGone/>



Sun, 06 Aug 2000 03:00:00 GMT  
 Strange byte order using fread() and fwrite()


Quote:

> Wild Bill said:
>> 3)fwrite(ptr, 1, 2, fp) - the two elements are saved to a binary file

> I refuse.  I won't assume two bytes.  I will write sizeof int bytes to the
> file.

I disagree.

There really isn't a right answer here.  If you use fread or
fwrite to read or write sizeof(int) bytes, you're asserting that
the data file will only be read or written by one copy of the
program compiled for one machine by one compiler.  If, on the
other hand, you have any hopes of making the data file format at
all portable (though by presuming a binary format, you've got
several strikes against you), there's a strong argument in favor
of using literal numbers like 2 and 4 in the I/O calls, heretical
an idea though it may sound.  But think about it: if the data
file is defined as having, say, 2-byte integers, than anything
other than 2 that you use as the third argument to fread is at
least one extra step removed from reality -- there's at least one
more place for something to go wrong, and a human reader of the
code has at least one more hoop to jump through to verify that
the code does, in fact, meet its specification by reading 2
bytes.

Of course, merely nailing down the size of the integer in the
data file doesn't solve all your problems; you've also got to
worry about byte order, and when you do, fread & co. become less
and less attractive.  You may have to perform byte swapping
before a write and after a read, and if the variable you're
reading to or writing from isn't 2 bytes but is big-endian,
the code is unlikely to work correctly.

If you want portable data files, the easiest route is to use text.
If you want portable binary data files, the easiest route is to
eschew fread and fwrite altogether, and use getc and putc (which
obviously give you complete control over each byte read and written).
If you want efficient portable binary data files, you've got a
very hard problem to solve indeed, and I'd recommend that you
take a look at an existing data file standard, such as XDR or one
of the others mentioned in the FAQ list.  (I'm pretty sure there
are formats out there, and libraries implementing them, which
claim to address all three criteria.)  Garden-variety binary data
file formats and I/O routines, however, are almost invariably
either not portable or not efficient (or both).

(Of course, text file formats are typically even less efficient
to read and write.  One of these days I've got to implement an
XDR-like library for efficient text data file I/O.  C9X's new %a
format will help.)

                                        Steve Summit



Sun, 06 Aug 2000 03:00:00 GMT  
 
 [ 19 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Problem with struct/array inside a struct using fread/fwrite

2. byte order and bit order

3. Writting low-order or high-order bytes

4. fread, fwrite probs

5. Please Help (fread, fwrite)

6. fwrite(), fread() portability.

7. problem with fwrite / fread

8. fopen() fseek() fread() fwrite()

9. fread and fwrite

10. fread and fwrite sync

11. microsoft endian problem with fread/fwrite

12. fwrite + fread problem

 

 
Powered by phpBB® Forum Software