Floating-point roundoff from static_casting to int?
Author Message
Floating-point roundoff from static_casting to int?

Hi,

This might be a ridiculously easy question, but when I run the code
below on Win2K/Pentium III with VC6 SP5, the integer n comes back as 9
instead of 10.  I can work around this by including <cmath> and
wrapping the division with a ceil() function, but I'm frankly really
surprised that I have to do this.  I assume this is due to
floating-point roundoff error, but does anyone have any thoughts?

void main()
{
int n = static_cast<int>(0.25f / 0.025f);  // n = 9 ?!

Quote:
}

Whitney Kew

Sun, 27 Mar 2005 02:36:15 GMT
Floating-point roundoff from static_casting to int?

Quote:

> This might be a ridiculously easy question, but when I run the code
> below on Win2K/Pentium III with VC6 SP5, the integer n comes back as 9
> instead of 10.  I can work around this by including <cmath> and
> wrapping the division with a ceil() function, but I'm frankly really
> surprised that I have to do this.  I assume this is due to
> floating-point roundoff error, but does anyone have any thoughts?

> void main()
> {
>    int n = static_cast<int>(0.25f / 0.025f);  // n = 9 ?!
> }

0.25 can be exactly represented, but I don't believe that's the case
for 0.025, in which case the observed behavior would be caused by the
limited precision of floating point calculations.

Sun, 27 Mar 2005 03:04:26 GMT
Floating-point roundoff from static_casting to int?

Quote:

> Hi,

> This might be a ridiculously easy question, but when I run the code
> below on Win2K/Pentium III with VC6 SP5, the integer n comes back as 9
> instead of 10.  I can work around this by including <cmath> and
> wrapping the division with a ceil() function, but I'm frankly really
> surprised that I have to do this.  I assume this is due to
> floating-point roundoff error, but does anyone have any thoughts?

In ANSI C/C++, casting from float to integer truncates the fractional part. In
other words, it ALWAYS rounds down. So, the result of that division mst be
slightly less than 10.0f, so it rounds down to 9.

Quote:
> void main()
> {
>    int n = static_cast<int>(0.25f / 0.025f);  // n = 9 ?!
> }

> Whitney Kew

Phil Frisbie, Jr.
Hawk Software
http://www.hawksoft.com

Sun, 27 Mar 2005 04:48:56 GMT
Floating-point roundoff from static_casting to int?

Quote:
> Hi,

> This might be a ridiculously easy question, but when I run the code
> below on Win2K/Pentium III with VC6 SP5, the integer n comes back as 9
> instead of 10.  I can work around this by including <cmath> and
> wrapping the division with a ceil() function, but I'm frankly really
> surprised that I have to do this.  I assume this is due to
> floating-point roundoff error, but does anyone have any thoughts?

> void main()
> {
>    int n = static_cast<int>(0.25f / 0.025f);  // n = 9 ?!
> }

If you intend to do a lot of work with floating point numbers you need to

http://docs.sun.com/db/doc/800-7895?q=What+Every+Computer+Scientist+S...