cross platform byte storage
Author |
Message |
<- Chameleon -> #1 / 21
|
 cross platform byte storage
My question is this: (about byte ordering) If I save a float number to a file, then when load this number from file, in any platform, I will have the same number? I speak about C float. I know this is not working with int and short but with float? If it is not working what I can do? Split the float into bytes is not a solution (in case of int is a solution) thanks
|
Thu, 29 Sep 2005 22:10:39 GMT |
|
 |
Tom St Deni #2 / 21
|
 cross platform byte storage
Quote: > My question is this: (about byte ordering) > If I save a float number to a file, then when load this number from file, in > any platform, I will have the same number? > I speak about C float. > I know this is not working with int and short but with float? > If it is not working what I can do? > Split the float into bytes is not a solution (in case of int is a
solution) Extract the mantissa and exponent. They're just integers. Or failing that output with a formated output function. Tom
|
Thu, 29 Sep 2005 22:37:19 GMT |
|
 |
Thomas Stege #3 / 21
|
 cross platform byte storage
Quote:
> Extract the mantissa and exponent. They're just integers.
Won't solve the issue of endianess -- Thomas.
|
Thu, 29 Sep 2005 21:46:17 GMT |
|
 |
CBFalcone #4 / 21
|
 cross platform byte storage
Quote:
> My question is this: (about byte ordering) > If I save a float number to a file, then when load this number > from file, in any platform, I will have the same number? > I speak about C float. > I know this is not working with int and short but with float? > If it is not working what I can do? > Split the float into bytes is not a solution (in case of int > is a solution)
Of course it works. For example: float realnums[COUNT]; size_t n; FILE *fpw, *fpr; ... /* open suitable file(s) */ ... n = fwrite(&realnums, sizeof float, COUNT, fpw); if (n != COUNT) handlewriteerror(); ... n = fread(&realnums, sizeof float, COUNT, fpr); if (n != COUNT) eoforerroroccured(); ... but the file is likely to be useless if not used on the same breed of machine with the same compiler. Byte ordering, format, etc. don't matter. You can change float to any other type with impunity as long as no pointers are involved. --
Available for consulting/temporary embedded and systems. <http://cbfalconer.home.att.net> USE worldnet address!
|
Thu, 29 Sep 2005 23:37:28 GMT |
|
 |
Michael B. Alle #5 / 21
|
 cross platform byte storage
Quote:
> My question is this: (about byte ordering) > If I save a float number to a file, then when load this number from > file, in any platform, I will have the same number?
How about encoding the float with the below enc_floatbe function: #include <stddef.h> #include <stdint.h> union convert_float { float f; uint32_t i; Quote: };
size_t enc_uint32be(uint32_t i, unsigned char *dst) { dst[0] = (i >> 24) & 0xFF; dst[1] = (i >> 16) & 0xFF; dst[2] = (i >> 8) & 0xFF; dst[3] = i & 0xFF; return 4; Quote: }
size_t enc_floatbe(const float f, unsigned char *dst) { union convert_float c; c.f = f; return enc_uint32be(c.i, dst); Quote: }
and then decode it with the dec_floatbe: uint32_t dec_uint32be(const unsigned char *src) { return ((unsigned)src[0] << 24) | ((unsigned)src[1] << 16) | ((unsigned)src[2] << 8) | src[3]; Quote: }
float dec_floatbe(const unsigned char *src) { union convert_float c; c.i = dec_uint32be(src); return c.f; Quote: }
Meaning, write the float to a sequence of 4 bytes and write the bytes to a file. When you want to read the numbers back, read out 4 bytes and use the dec function. Mike -- A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived.
|
Fri, 30 Sep 2005 03:16:45 GMT |
|
 |
bd #6 / 21
|
 cross platform byte storage
Quote:
> My question is this: (about byte ordering) > If I save a float number to a file, then when load this number from file, in > any platform, I will have the same number?
I don't think it's guarenteed. Quote: > I speak about C float. > I know this is not working with int and short but with float? > If it is not working what I can do?
You can output it in text form, but you'll lose some accuraccy, possibly. Quote: > Split the float into bytes is not a solution (in case of int is a solution) > thanks
|
Fri, 30 Sep 2005 03:40:38 GMT |
|
 |
bd #7 / 21
|
 cross platform byte storage
Quote:
>> Extract the mantissa and exponent. They're just integers. > Won't solve the issue of endianess
Integers can be easily forced into a network byte order: struct network_long { unsigned char data[4]; Quote: };
struct network_long htonl(unsigned long n){ int i; struct network_long temp = {{0}}; for(i = 3; i > 0; i--){ temp.data[i] = n &0xffu; n >>= 8; } return temp; Quote: }
(note: Untested. Use at your own peril.)
|
Fri, 30 Sep 2005 04:03:40 GMT |
|
 |
<- Chameleon -> #8 / 21
|
 cross platform byte storage
but have float type 4 bytes? Is this standard in C Specification? (I mean 4 bytes) I know int type is not standard 4 bytes. float is 4 bytes?
|
Fri, 30 Sep 2005 05:55:45 GMT |
|
 |
those who know me have no need of my nam #9 / 21
|
 cross platform byte storage
in comp.lang.c i read: Quote: >but have float type 4 bytes? >Is this standard in C Specification? (I mean 4 bytes)
nope. -- bringing you boring signatures for 17 years
|
Fri, 30 Sep 2005 06:19:48 GMT |
|
 |
those who know me have no need of my nam #10 / 21
|
 cross platform byte storage
in comp.lang.c i read: Quote: >My question is this: (about byte ordering) >If I save a float number to a file, then when load this number from file, in >any platform, I will have the same number?
no. some solutions have been provided, but i don't think that you've been given a direct answer to that part of your question. Quote: >Split the float into bytes is not a solution (in case of int is a solution)
it's not a solution even with int if you directly access the bytes, you must use the value. -- bringing you boring signatures for 17 years
|
Fri, 30 Sep 2005 06:23:12 GMT |
|
 |
Jack Klei #11 / 21
|
 cross platform byte storage
On Sun, 13 Apr 2003 15:16:45 -0400, "Michael B. Allen"
Quote:
> > My question is this: (about byte ordering) > > If I save a float number to a file, then when load this number from > > file, in any platform, I will have the same number? > How about encoding the float with the below enc_floatbe function: > #include <stddef.h> > #include <stdint.h> > union convert_float { > float f; > uint32_t i;
Note that not all implementations, even C99 conforming one, will provide a uint32_t type. Quote: > }; > size_t > enc_uint32be(uint32_t i, unsigned char *dst) > { > dst[0] = (i >> 24) & 0xFF; > dst[1] = (i >> 16) & 0xFF; > dst[2] = (i >> 8) & 0xFF; > dst[3] = i & 0xFF; > return 4;
What is sizeof(unit32_t) is not 4? I know of one implementation where it is 2, and another where it is 1, and both are perfectly conforming. Even more important, what if sizeof(float) != sizeof(unit32_t). Quote: > } > size_t > enc_floatbe(const float f, unsigned char *dst) > { > union convert_float c; > c.f = f; > return enc_uint32be(c.i, dst);
Accessing c.i after storing c.f is undefined behavior, of course. Quote: > } > and then decode it with the dec_floatbe: > uint32_t > dec_uint32be(const unsigned char *src) > { > return ((unsigned)src[0] << 24) | ((unsigned)src[1] << 16) | > ((unsigned)src[2] << 8) | src[3]; > } > float > dec_floatbe(const unsigned char *src) > { > union convert_float c; > c.i = dec_uint32be(src); > return c.f;
Accessing c.f after storing c.i is undefined behavior, of course. Quote: > } > Meaning, write the float to a sequence of 4 bytes and write the bytes > to a file. When you want to read the numbers back, read out 4 bytes and > use the dec function. > Mike
The only guarantee you have is that if a conforming C99 compiler provides uint32_t, it will contain exactly 32 bits and have no trap representations. Specifically it does not guarantee that those 32 bits will occupy 4 bytes, because conforming C compilers are not required to provide 8 bit chars. Specifically, for example, the Analog Devices SHARC has CHAR_BIT 32 and sizeof(uint32_t) is 1. The TI 28xx series has CHAR_BIT 16 and sizeof(uint32_t) is 2. And there is no guarantee, requirement, or even vague hint in the standard that sizeof(uint32_t) == sizeof(float). This happens to be true on some platforms, but is by no means universal. -- Jack Klein Home: http://JK-Technology.Com FAQs for comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html comp.lang.c++ http://www.parashift.com/c++-faq-lite/ alt.comp.lang.learn.c-c++ ftp://snurse-l.org/pub/acllc-c++/faq
|
Fri, 30 Sep 2005 07:32:59 GMT |
|
 |
Joel Re #12 / 21
|
 cross platform byte storage
Quote: > My question is this: (about byte ordering) > If I save a float number to a file, then when load this number from file, in > any platform, I will have the same number? > I speak about C float. > I know this is not working with int and short but with float? > If it is not working what I can do? > Split the float into bytes is not a solution (in case of int is a solution) > thanks
Split it into bits? ;) <math.h> in my really old reference book here declares these two functions, which may be of some help: double frexp( double x, int * exponentPtr ); double ldexp( double x, int exponent ); frexp() effectively extracts the fractional portion and exponent. ldexp() effectively puts them back together again. Caveats about the precision, of course. You might be tempted to write the bit pattern of the fractional portion, but you really shouldn't depend on the bit patterns of floating point, even between different versions of the same compiler on the same CPU. (You can get lucky, of course.) What you'd probably want to do is something like this: double fractionalPart = frexp( valueToWrite, &exponentPart ); int exponentPart, i; long accumulator = 0; char outbuff[ 6 ] = { 0 }; for ( i = 0; i < 32; ++i ) { fractionalPart *= 2; accumulator <<= 1; if ( fractionalPart > 1 ) { accumulator |= 1; fractionalPart -= 1; } } outbuff[ 1 ] = (char) exponentPart; outbuff[ 0 ] = (char) exponentPart >> CHAR_BIT; for ( i = 5; i > 1; --i ) { outbuff[ i ] = (char) accumulator; /* Masking with UCHAR_MAX would be redundant, really. */ accumulator >>= CHAR_BIT; } Ant then write the six bytes out to a file opened for binary, etc. On reading, you have to be careful with the sign bit bringing the bytes in, of course. I personally prefer to use unsigned char for things like this, but you still have to use casts. No guarantees on the above. It is liable to drop bits and invent bits. But there should be enough padding to avoid many implementation issues. You get close to binary output, with the ability to read and write most of the significant portion portably. A little more care might suppress the invention of bits. I don't really have time to go after that. Restoring the fraction and doing doubles are left as an exercise. Incidentally, if file size is not an issue, it might be preferable to just use printf()'s formatting and store the numbers as text.
|
Fri, 30 Sep 2005 13:36:49 GMT |
|
 |
Michael B. Alle #13 / 21
|
 cross platform byte storage
Quote:
>> union convert_float { >> float f; >> uint32_t i; > Note that not all implementations, even C99 conforming one, will provide > a uint32_t type.
Sure. It's still the correct type to use for explicit memory manipulation like this. I have a defines.h for platform specific stuff although at the moment it's not used to typedef unit{8,16,32,64}_t like it should. Quote: >> }; >> size_t >> enc_uint32be(uint32_t i, unsigned char *dst) { >> dst[0] = (i >> 24) & 0xFF; >> dst[1] = (i >> 16) & 0xFF; >> dst[2] = (i >> 8) & 0xFF; >> dst[3] = i & 0xFF; >> return 4; > What is sizeof(unit32_t) is not 4? I know of one implementation where > it is 2, and another where it is 1, and both are perfectly conforming. > Even more important, what if sizeof(float) != sizeof(unit32_t).
I didn't know that. Thanks. Natrually this code is platform/implementation specific and there would need to be some conditional complilation for exceptional platforms. Quote: >> float >> dec_floatbe(const unsigned char *src) { >> union convert_float c; >> c.i = dec_uint32be(src); >> return c.f; > Accessing c.f after storing c.i is undefined behavior, of course.
So I have been told several times. Unfortunately I have never been able to determine if there was any practical merit behind the concern insinuated by the expression "undefined behavior" in the case of using a union like this. Are you aware of an implementation or architecture on which this union technique would actually break? Granted there might need to be additional manipulation (e.g. on StrongArm I believe the first two bytes and last two bytes used to store floats are reversed?) but my understanding is that the issue is platform/implementation specific and therefore the behavior is actually quite well defined for a given platform/implementation. Also, I just want to point out that even though the code is not portable across all architectures the binary files it creates *are* portable across architectures provided the {enc,dec} functions can be ported (and with a little understanding there's a good possibility of that). Of course in this case using the union over manually extracting the mantissa and exponent is a no-brainer for the platforms that support it (which is the popular majority: IA32, Sparc, etc). Quote: >> Meaning, write the float to a sequence of 4 bytes and write the bytes >> to a file. When you want to read the numbers back, read out 4 bytes and >> use the dec function. > The only guarantee you have is that if a conforming C99 compiler > provides uint32_t, it will contain exactly 32 bits and have no trap > representations. > Specifically it does not guarantee that those 32 bits will occupy 4 > bytes, because conforming C compilers are not required to provide 8 bit > chars. Specifically, for example, the Analog Devices SHARC has CHAR_BIT > 32 and sizeof(uint32_t) is 1. The TI 28xx series has CHAR_BIT 16 and > sizeof(uint32_t) is 2.
I did some work on TMS3C59x once. It had 16bit addressable units. But I also recall it used IEEE754 representation for floats so in theory I could provide my own implementation of the {enc,dec} functions. The point is that the interface for encoding and decoding floats does not change regardless of the memory representation. It's either char * in float out or float in char * out. Quote: > And there is no guarantee, requirement, or even vague hint in the > standard that sizeof(uint32_t) == sizeof(float). This happens to be > true on some platforms, but is by no means universal.
I have to admit I had not fully considered all the possibilities. Thanks for your insight. It will be very useful when I do need to implement these {enc,dec} functions on one of these exceptional platforms. I suppose my documentation should be more carefully worded as well. Thanks, Mike -- A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived.
|
Fri, 30 Sep 2005 14:31:24 GMT |
|
 |
Michael B. Alle #14 / 21
|
 cross platform byte storage
On Sun, 13 Apr 2003 18:19:48 -0400, those who know me have no need of my Quote:
> in comp.lang.c i read: >>but have float type 4 bytes? >>Is this standard in C Specification? (I mean 4 bytes) > nope.
Actually the vast magority of floating point processors use the IEEE754 format which I believe defines 32bit and 64bit representations so the memory representation is fairly restricted. Although as Jack just pointed out this may not be a sequence of 4 memory units. However, the interface for encoding and decoding these numbers would not change so your code will not break when compling on a new architecture provided the {enc,dec} functions can be ported (likely). Mike -- A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived.
|
Fri, 30 Sep 2005 14:43:55 GMT |
|
 |
Kevin Easto #15 / 21
|
 cross platform byte storage
Quote:
>>> union convert_float { >>> float f; >>> uint32_t i;
[...] Quote: >>> float >>> dec_floatbe(const unsigned char *src) { >>> union convert_float c; >>> c.i = dec_uint32be(src); >>> return c.f; >> Accessing c.f after storing c.i is undefined behavior, of course. > So I have been told several times. Unfortunately I have never been > able to determine if there was any practical merit behind the concern > insinuated by the expression "undefined behavior" in the case of using > a union like this. Are you aware of an implementation or architecture > on which this union technique would actually break?
Because the cases like this are undefined behaviour, optimising compilers are free to make the assumption that a store to an int and a read from a float do not ever refer to the same object - so you have the danger on any architecture that optimisation will mean that the UB manifests itself as garbage values. You can probably avert this by declaring the union as volatile. The other reason it's UB is because the resulting float value may be illegal, and cause some kind of floating point trap. - Kevin.
|
Fri, 30 Sep 2005 16:16:27 GMT |
|
|
Page 1 of 2
|
[ 21 post ] |
|
Go to page:
[1]
[2] |
|