cross platform byte storage 
Author Message
 cross platform byte storage

My question is this: (about byte ordering)

If I save a float number to a file, then when load this number from file, in
any platform, I will have the same number?

I speak about C float.

I know this is not working with int and short but with float?

If it is not working what I can do?

Split the float into bytes is not a solution (in case of int is a solution)

thanks



Thu, 29 Sep 2005 22:10:39 GMT  
 cross platform byte storage



Quote:
> My question is this: (about byte ordering)

> If I save a float number to a file, then when load this number from file,
in
> any platform, I will have the same number?

> I speak about C float.

> I know this is not working with int and short but with float?

> If it is not working what I can do?

> Split the float into bytes is not a solution (in case of int is a

solution)

Extract the mantissa and exponent.  They're just integers.

Or failing that output with a formated output function.

Tom



Thu, 29 Sep 2005 22:37:19 GMT  
 cross platform byte storage

Quote:

> Extract the mantissa and exponent.  They're just integers.

Won't solve the issue of endianess

--
Thomas.



Thu, 29 Sep 2005 21:46:17 GMT  
 cross platform byte storage

Quote:

> My question is this: (about byte ordering)

> If I save a float number to a file, then when load this number
> from file, in any platform, I will have the same number?

> I speak about C float.

> I know this is not working with int and short but with float?

> If it is not working what I can do?

> Split the float into bytes is not a solution (in case of int
> is a solution)

Of course it works.  For example:

   float   realnums[COUNT];
   size_t  n;
   FILE   *fpw, *fpr;
...
   /* open suitable file(s) */
...
   n = fwrite(&realnums, sizeof float, COUNT, fpw);
   if (n != COUNT) handlewriteerror();
...
   n = fread(&realnums, sizeof float, COUNT, fpr);
   if (n != COUNT) eoforerroroccured();
...

but the file is likely to be useless if not used on the same breed
of machine with the same compiler.  Byte ordering, format, etc.
don't matter.  You can change float to any other type with
impunity as long as no pointers are involved.

--

   Available for consulting/temporary embedded and systems.
   <http://cbfalconer.home.att.net>  USE worldnet address!



Thu, 29 Sep 2005 23:37:28 GMT  
 cross platform byte storage

Quote:

> My question is this: (about byte ordering)

> If I save a float number to a file, then when load this number from
> file, in any platform, I will have the same number?

How about encoding the float with the below enc_floatbe function:

#include <stddef.h>
#include <stdint.h>

union convert_float {
        float f;
        uint32_t i;

Quote:
};

size_t
enc_uint32be(uint32_t i, unsigned char *dst)
{
        dst[0] = (i >> 24) & 0xFF;
        dst[1] = (i >> 16) & 0xFF;
        dst[2] = (i >> 8) & 0xFF;
        dst[3] = i & 0xFF;
        return 4;
Quote:
}

size_t
enc_floatbe(const float f, unsigned char *dst)
{
        union convert_float c;
        c.f = f;
        return enc_uint32be(c.i, dst);

Quote:
}

and then decode it with the dec_floatbe:

uint32_t
dec_uint32be(const unsigned char *src)
{
        return ((unsigned)src[0] << 24) | ((unsigned)src[1] << 16) |
           ((unsigned)src[2] << 8) | src[3];

Quote:
}

float
dec_floatbe(const unsigned char *src)
{
        union convert_float c;
        c.i = dec_uint32be(src);
        return c.f;

Quote:
}

Meaning, write the float to a sequence of 4 bytes and write the bytes
to a file. When you want to read the numbers back, read out 4 bytes and
use the dec function.

Mike

--
A  program should be written to model the concepts of the task it
performs rather than the physical world or a process because this
maximizes  the  potential  for it to be applied to tasks that are
conceptually  similar and, more important, to tasks that have not
yet been conceived.



Fri, 30 Sep 2005 03:16:45 GMT  
 cross platform byte storage

Quote:

> My question is this: (about byte ordering)

> If I save a float number to a file, then when load this number from file, in
> any platform, I will have the same number?

I don't think it's guarenteed.

Quote:
> I speak about C float.

> I know this is not working with int and short but with float?

> If it is not working what I can do?

You can output it in text form, but you'll lose some accuraccy, possibly.
Quote:
> Split the float into bytes is not a solution (in case of int is a solution)

> thanks



Fri, 30 Sep 2005 03:40:38 GMT  
 cross platform byte storage

Quote:


>> Extract the mantissa and exponent.  They're just integers.

> Won't solve the issue of endianess

Integers can be easily forced into a network byte order:
struct network_long {
  unsigned char data[4];

Quote:
};

struct network_long htonl(unsigned long n){
  int i;
  struct network_long temp = {{0}};
  for(i = 3; i > 0; i--){
    temp.data[i] = n &0xffu;
    n >>= 8;
  }
  return temp;

Quote:
}

(note: Untested. Use at your own peril.)


Fri, 30 Sep 2005 04:03:40 GMT  
 cross platform byte storage
but have float type 4 bytes?
Is this standard in C Specification? (I mean 4 bytes)
I know int type is not standard 4 bytes. float is 4 bytes?


Fri, 30 Sep 2005 05:55:45 GMT  
 cross platform byte storage
in comp.lang.c i read:

Quote:
>but have float type 4 bytes?
>Is this standard in C Specification? (I mean 4 bytes)

nope.

--
bringing you boring signatures for 17 years



Fri, 30 Sep 2005 06:19:48 GMT  
 cross platform byte storage
in comp.lang.c i read:

Quote:
>My question is this: (about byte ordering)

>If I save a float number to a file, then when load this number from file, in
>any platform, I will have the same number?

no.  some solutions have been provided, but i don't think that you've been
given a direct answer to that part of your question.

Quote:
>Split the float into bytes is not a solution (in case of int is a solution)

it's not a solution even with int if you directly access the bytes, you
must use the value.

--
bringing you boring signatures for 17 years



Fri, 30 Sep 2005 06:23:12 GMT  
 cross platform byte storage
On Sun, 13 Apr 2003 15:16:45 -0400, "Michael B. Allen"

Quote:

> > My question is this: (about byte ordering)

> > If I save a float number to a file, then when load this number from
> > file, in any platform, I will have the same number?

> How about encoding the float with the below enc_floatbe function:

> #include <stddef.h>
> #include <stdint.h>

> union convert_float {
>    float f;
>    uint32_t i;

Note that not all implementations, even C99 conforming one, will
provide a uint32_t type.

Quote:
> };
> size_t
> enc_uint32be(uint32_t i, unsigned char *dst)
> {
>    dst[0] = (i >> 24) & 0xFF;
>    dst[1] = (i >> 16) & 0xFF;
>    dst[2] = (i >> 8) & 0xFF;
>    dst[3] = i & 0xFF;
>    return 4;

What is sizeof(unit32_t) is not 4?  I know of one implementation where
it is 2, and another where it is 1, and both are perfectly conforming.

Even more important, what if sizeof(float) != sizeof(unit32_t).

Quote:
> }
> size_t
> enc_floatbe(const float f, unsigned char *dst)
> {
>    union convert_float c;
>    c.f = f;
>    return enc_uint32be(c.i, dst);

Accessing c.i after storing c.f is undefined behavior, of course.

Quote:
> }

> and then decode it with the dec_floatbe:

> uint32_t
> dec_uint32be(const unsigned char *src)
> {
>    return ((unsigned)src[0] << 24) | ((unsigned)src[1] << 16) |
>            ((unsigned)src[2] << 8) | src[3];
> }
> float
> dec_floatbe(const unsigned char *src)
> {
>    union convert_float c;
>    c.i = dec_uint32be(src);
>    return c.f;

Accessing c.f after storing c.i is undefined behavior, of course.

Quote:
> }

> Meaning, write the float to a sequence of 4 bytes and write the bytes
> to a file. When you want to read the numbers back, read out 4 bytes and
> use the dec function.

> Mike

The only guarantee you have is that if a conforming C99 compiler
provides uint32_t, it will contain exactly 32 bits and have no trap
representations.

Specifically it does not guarantee that those 32 bits will occupy 4
bytes, because conforming C compilers are not required to provide 8
bit chars.  Specifically, for example, the Analog Devices SHARC has
CHAR_BIT 32 and sizeof(uint32_t) is 1.  The TI 28xx series has
CHAR_BIT 16 and sizeof(uint32_t) is 2.

And there is no guarantee, requirement, or even vague hint in the
standard that sizeof(uint32_t) == sizeof(float).  This happens to be
true on some platforms, but is by no means universal.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++ ftp://snurse-l.org/pub/acllc-c++/faq



Fri, 30 Sep 2005 07:32:59 GMT  
 cross platform byte storage

Quote:
> My question is this: (about byte ordering)

> If I save a float number to a file, then when load this number from file, in
> any platform, I will have the same number?

> I speak about C float.

> I know this is not working with int and short but with float?

> If it is not working what I can do?

> Split the float into bytes is not a solution (in case of int is a solution)

> thanks

Split it into bits? ;)

<math.h> in my really old reference book here declares these two
functions, which may be of some help:

    double frexp( double x, int * exponentPtr );
    double ldexp( double x, int exponent );

frexp() effectively extracts the fractional portion and exponent.
ldexp() effectively puts them back together again.

Caveats about the precision, of course.

You might be tempted to write the bit pattern of the fractional
portion, but you really shouldn't depend on the bit patterns of
floating point, even between different versions of the same compiler
on the same CPU. (You can get lucky, of course.) What you'd probably
want to do is something like this:

    double fractionalPart = frexp( valueToWrite, &exponentPart );
    int exponentPart, i;
    long accumulator = 0;
    char outbuff[ 6 ] = { 0 };
    for ( i = 0; i < 32; ++i )
    {   fractionalPart *= 2;
        accumulator <<= 1;
        if ( fractionalPart > 1 )
        {   accumulator |= 1;
            fractionalPart -= 1;
        }
    }
    outbuff[ 1 ] = (char) exponentPart;
    outbuff[ 0 ] = (char) exponentPart >> CHAR_BIT;
    for ( i = 5; i > 1; --i )
    {   outbuff[ i ] = (char) accumulator;
        /* Masking with UCHAR_MAX would be redundant, really. */
        accumulator >>= CHAR_BIT;
    }

Ant then write the six bytes out to a file opened for binary, etc.

On reading, you have to be careful with the sign bit bringing the
bytes in, of course. I personally prefer to use unsigned char for
things like this, but you still have to use casts.

No guarantees on the above. It is liable to drop bits and invent bits.
But there should be enough padding to avoid many implementation
issues. You get close to binary output, with the ability to read and
write most of the significant portion portably.

A little more care might suppress the invention of bits. I don't
really have time to go after that.

Restoring the fraction and doing doubles are left as an exercise.

Incidentally, if file size is not an issue, it might be preferable to
just use printf()'s formatting and store the numbers as text.



Fri, 30 Sep 2005 13:36:49 GMT  
 cross platform byte storage

Quote:

>> union convert_float {
>>        float f;
>>        uint32_t i;

> Note that not all implementations, even C99 conforming one, will provide
> a uint32_t type.

Sure. It's still the correct type to use for explicit memory manipulation
like this. I have a defines.h for platform specific stuff although at
the moment it's not used to typedef unit{8,16,32,64}_t like it should.

Quote:
>> };
>> size_t
>> enc_uint32be(uint32_t i, unsigned char *dst) {
>>        dst[0] = (i >> 24) & 0xFF;
>>        dst[1] = (i >> 16) & 0xFF;
>>        dst[2] = (i >> 8) & 0xFF;
>>        dst[3] = i & 0xFF;
>>        return 4;

> What is sizeof(unit32_t) is not 4?  I know of one implementation where
> it is 2, and another where it is 1, and both are perfectly conforming.

> Even more important, what if sizeof(float) != sizeof(unit32_t).

I didn't know that. Thanks. Natrually this code is platform/implementation
specific and there would need to be some conditional complilation for
exceptional platforms.

Quote:
>> float
>> dec_floatbe(const unsigned char *src) {
>>        union convert_float c;
>>        c.i = dec_uint32be(src);
>>        return c.f;

> Accessing c.f after storing c.i is undefined behavior, of course.

So I have been told several times. Unfortunately I have never been
able to determine if there was any practical merit behind the concern
insinuated by the expression "undefined behavior" in the case of using
a union like this. Are you aware of an implementation or architecture
on which this union technique would actually break? Granted there might
need to be additional manipulation (e.g. on StrongArm I believe the first
two bytes and last two bytes used to store floats are reversed?) but
my understanding is that the issue is platform/implementation specific
and therefore the behavior is actually quite well defined for a given
platform/implementation.

Also, I just want to point out that even though the code is not portable
across all architectures the binary files it creates *are* portable
across architectures provided the {enc,dec} functions can be ported
(and with a little understanding there's a good possibility of that).

Of course in this case using the union over manually extracting the
mantissa and exponent is a no-brainer for the platforms that support it
(which is the popular majority: IA32, Sparc, etc).

Quote:
>> Meaning, write the float to a sequence of 4 bytes and write the bytes
>> to a file. When you want to read the numbers back, read out 4 bytes and
>> use the dec function.

> The only guarantee you have is that if a conforming C99 compiler
> provides uint32_t, it will contain exactly 32 bits and have no trap
> representations.

> Specifically it does not guarantee that those 32 bits will occupy 4
> bytes, because conforming C compilers are not required to provide 8 bit
> chars.  Specifically, for example, the Analog Devices SHARC has CHAR_BIT
> 32 and sizeof(uint32_t) is 1.  The TI 28xx series has CHAR_BIT 16 and
> sizeof(uint32_t) is 2.

I did some work on TMS3C59x once. It had 16bit addressable units. But
I also recall it used IEEE754 representation for floats so in theory
I could provide my own implementation of the {enc,dec} functions. The
point is that the interface for encoding and decoding floats does not
change regardless of the memory representation. It's either char *
in float out or float in char * out.

Quote:
> And there is no guarantee, requirement, or even vague hint in the
> standard that sizeof(uint32_t) == sizeof(float).  This happens to be
> true on some platforms, but is by no means universal.

I have to admit I had not fully considered all the possibilities. Thanks
for your insight. It will be very useful when I do need to implement these
{enc,dec} functions on one of these exceptional platforms. I suppose my
documentation should be more carefully worded as well.

Thanks,
Mike

--
A  program should be written to model the concepts of the task it
performs rather than the physical world or a process because this
maximizes  the  potential  for it to be applied to tasks that are
conceptually  similar and, more important, to tasks that have not
yet been conceived.



Fri, 30 Sep 2005 14:31:24 GMT  
 cross platform byte storage
On Sun, 13 Apr 2003 18:19:48 -0400, those who know me have no need of my

Quote:

> in comp.lang.c i read:

>>but have float type 4 bytes?
>>Is this standard in C Specification? (I mean 4 bytes)

> nope.

Actually the vast magority of floating point processors use the IEEE754
format which I believe defines 32bit and 64bit representations so the
memory representation is fairly restricted. Although as Jack just pointed
out this may not be a sequence of 4 memory units. However, the interface
for encoding and decoding these numbers would not change so your code
will not break when compling on a new architecture provided the {enc,dec}
functions can be ported (likely).

Mike

--
A  program should be written to model the concepts of the task it
performs rather than the physical world or a process because this
maximizes  the  potential  for it to be applied to tasks that are
conceptually  similar and, more important, to tasks that have not
yet been conceived.



Fri, 30 Sep 2005 14:43:55 GMT  
 cross platform byte storage

Quote:

>>> union convert_float {
>>>      float f;
>>>      uint32_t i;

[...]

Quote:
>>> float
>>> dec_floatbe(const unsigned char *src) {
>>>      union convert_float c;
>>>      c.i = dec_uint32be(src);
>>>      return c.f;

>> Accessing c.f after storing c.i is undefined behavior, of course.

> So I have been told several times. Unfortunately I have never been
> able to determine if there was any practical merit behind the concern
> insinuated by the expression "undefined behavior" in the case of using
> a union like this. Are you aware of an implementation or architecture
> on which this union technique would actually break?

Because the cases like this are undefined behaviour, optimising
compilers are free to make the assumption that a store to an int and a
read from a float do not ever refer to the same object - so you have the
danger on any architecture that optimisation will mean that the UB
manifests itself as garbage values.

You can probably avert this by declaring the union as volatile.

The other reason it's UB is because the resulting float value may be
illegal, and cause some kind of floating point trap.

        - Kevin.



Fri, 30 Sep 2005 16:16:27 GMT  
 
 [ 21 post ]  Go to page: [1] [2]

 Relevant Pages 

1. High-performance, cross-platform string support

2. Cross-Platform Development in C

3. slightly OT: cross-platform binary compatibility?

4. Regarding chapter 22 Cross platform development of C Unleashed

5. Cross platform development strategy

6. embedded eeditor -> cross platform

7. cross platform

8. cross platform sockets library

9. Storing cross-platform struct data

10. cross platform compatability

11. Cross-platform program

 

 
Powered by phpBB® Forum Software