Double Precision bug?? 
Author Message
 Double Precision bug??

Please help!!!!

I am trying to do something simple:
Double x = 5.0;
Double y = 1E-6;

Double z = x*y;

The value of z comes out to:
0.0000049999999999999996

The difference between 5E-6 and the number above is much much much bigger
than Double::Epsilon. So I cannot even use that to compare my Doubles. What
do I do? I tried this in C++ .NET, C# .NET, VB .NET and they all give me the
same result. I am using v 1.0 of the Framework (if that matters).

Why am I losing precision here? I am not doing anything complicated.

If someone has seen this before and has solved it, can you please let me
know?

Thanks
MT



Wed, 12 Oct 2005 03:41:21 GMT  
 Double Precision bug??

Quote:

> I am trying to do something simple:
> Double x = 5.0;
> Double y = 1E-6;

> Double z = x*y;

> The value of z comes out to:
> 0.0000049999999999999996

Have you really a simple program like:

void main()
{
  System.Double x = 5.0;
  System.Double y = 1E-6;

  System.Double z = x*y;

  System.Console.WriteLine(z.ToString());

Quote:
}

?

Are you using some kind of DirectX in you program ?

Is anyone calling _controlfp or _control87 ?

Does anyone change the PC (Precision Control) from the FPU ? If the PC
is set to 0, then only 26 bits are used for precison... normally 56 are
used...

--
Greetings
  Jochen

   Do you need a memory-leak finder ?
   http://www.codeproject.com/useritems/leakfinder.asp



Wed, 12 Oct 2005 03:52:46 GMT  
 Double Precision bug??

Quote:

> Please help!!!!

> I am trying to do something simple:
> Double x = 5.0;
> Double y = 1E-6;

> Double z = x*y;

> The value of z comes out to:
> 0.0000049999999999999996

> The difference between 5E-6 and the number above is much much much
> bigger than Double::Epsilon. So I cannot even use that to compare my
> Doubles. What do I do? I tried this in C++ .NET, C# .NET, VB .NET and
> they all give me the same result. I am using v 1.0 of the Framework
> (if that matters).

> Why am I losing precision here? I am not doing anything complicated.

> If someone has seen this before and has solved it, can you please let
> me know?

You're not losing precision - you're misunderstanding the precision of
floating point math.

The error you have indicated here, is approximately 2e-53, which is the
normalized precision of double.  This is a natural consequence of the fact
that double has a 53-bit mantissa.

Double::Episilon is the smallest value that Double can represent
(4.94065645841247e-324).  This is not the same as the precision that a
calculation performed on double is guaranteed to provide.

-cd



Wed, 12 Oct 2005 04:05:14 GMT  
 Double Precision bug??



Quote:

> > Please help!!!!

> > I am trying to do something simple:
> > Double x = 5.0;
> > Double y = 1E-6;

> > Double z = x*y;

> > The value of z comes out to:
> > 0.0000049999999999999996

> > The difference between 5E-6 and the number above is much much much
> > bigger than Double::Epsilon. So I cannot even use that to compare my
> > Doubles. What do I do? I tried this in C++ .NET, C# .NET, VB .NET and
> > they all give me the same result. I am using v 1.0 of the Framework
> > (if that matters).

> > Why am I losing precision here? I am not doing anything complicated.

> > If someone has seen this before and has solved it, can you please let
> > me know?

> You're not losing precision - you're misunderstanding the precision of
> floating point math.

> The error you have indicated here, is approximately 2e-53, which is the
> normalized precision of double.  This is a natural consequence of the fact
> that double has a 53-bit mantissa.

> Double::Episilon is the smallest value that Double can represent
> (4.94065645841247e-324).  This is not the same as the precision that a
> calculation performed on double is guaranteed to provide.

> -cd

Alright, I am ok with this. Now, shouldn't the (==) and (!=) operator take
that into account? If I am building a new type and I know that I cannot
precisely represent a number beyond a certain number of bits, I should not
say that 2 numbers are not equal if I cannot guarantee that they are
different?
So, in my case, if I have a variable a = 5E-6 and I compare it to z, .NET
will tell me that they are different even though they are different in the
54th bit? I have written so many lines of code assuming (you know what they
say about this word) that if I compare 2 double precision numbers, this
stuff is taken care of by the libraries. Do I now have to go through all of
my code to see where I compare 2 double precision numbers and change it to
something like:
If (Math::Abs(a-z) < 1E-53)

That sounds absurd, right?

Also, why should 5*1E-6 cause an error. If I was writing a library and I saw
that the numbers that I need to represent have integral mantissas, I will
not do any special conversions. I would just multiply the mantissas and
shift the exponent accordingly.... Am I missing something or do I sound
really stupid here?

MT



Wed, 12 Oct 2005 05:02:28 GMT  
 Double Precision bug??

Quote:

> Please help!!!!

> I am trying to do something simple:
> Double x = 5.0;
> Double y = 1E-6;

> Double z = x*y;

> The value of z comes out to:
> 0.0000049999999999999996

Doubles should be precise to about 15 or 16 places, which is indeed
what you're seeing here.  Singles should be precise to about seven or
eight places.

--
Craig Powers
MVP - Visual C++



Wed, 12 Oct 2005 05:06:27 GMT  
 Double Precision bug??


Quote:

> > I am trying to do something simple:
> > Double x = 5.0;
> > Double y = 1E-6;

> > Double z = x*y;

> > The value of z comes out to:
> > 0.0000049999999999999996

> Have you really a simple program like:

> void main()
> {
>   System.Double x = 5.0;
>   System.Double y = 1E-6;

>   System.Double z = x*y;

>   System.Console.WriteLine(z.ToString());
> }

> ?

No, it is not as simple as that. I simplified it so that I could explain my
problem. Although, I did write a program in VC++ .NET, VB.NET and VC# .NET
that looks exactly like that. ToString() just rounds numbers, so you will
not see it in the console. However, if you go to a watch window, you will
see what I am talking about.

Quote:

> Are you using some kind of DirectX in you program ?

> Is anyone calling _controlfp or _control87 ?

Not that I am aware of. I believe a part of our application (maybe a control
in the GUI) might be using DirectX, but I cannot vouch for them. My stuff is
written in Managed C++ (VC++ .NET). I can guarantee that I absolutely don't
use any DirectX :)

Quote:

> Does anyone change the PC (Precision Control) from the FPU ? If the PC
> is set to 0, then only 26 bits are used for precison... normally 56 are
> used...

I don't know how to verify this. Can you please post a message showing me
how to check for this? Although, one of the MVPs said that my error was
within the 2E-53 range - I need to double check this....

Quote:

> --
> Greetings
>   Jochen

Thanks,
MT


Wed, 12 Oct 2005 05:10:23 GMT  
 Double Precision bug??


Quote:




> > > Please help!!!!

> > > I am trying to do something simple:
> > > Double x = 5.0;
> > > Double y = 1E-6;

> > > Double z = x*y;

> > > The value of z comes out to:
> > > 0.0000049999999999999996

> > > The difference between 5E-6 and the number above is much much much
> > > bigger than Double::Epsilon. So I cannot even use that to compare my
> > > Doubles. What do I do? I tried this in C++ .NET, C# .NET, VB .NET and
> > > they all give me the same result. I am using v 1.0 of the Framework
> > > (if that matters).

> > > Why am I losing precision here? I am not doing anything complicated.

> > > If someone has seen this before and has solved it, can you please let
> > > me know?

> > You're not losing precision - you're misunderstanding the precision of
> > floating point math.

> > The error you have indicated here, is approximately 2e-53, which is the
> > normalized precision of double.  This is a natural consequence of the fact
> > that double has a 53-bit mantissa.

> > Double::Episilon is the smallest value that Double can represent
> > (4.94065645841247e-324).  This is not the same as the precision that a
> > calculation performed on double is guaranteed to provide.

> > -cd

> Alright, I am ok with this. Now, shouldn't the (==) and (!=) operator take
> that into account?

No, they should not.  Any notion as to
how close should be considered "equal"
is entirely application specific.

Quote:
> If I am building a new type and I know that I cannot
> precisely represent a number beyond a certain number of bits, I should not
> say that 2 numbers are not equal if I cannot guarantee that they are
> different?

If your application uses the equality
operators with floating point operands,
then its logic needs rethinking.  In my
years of doing scientific/engineering
programming, I have rarely seen FP
equality comparisons that made sense.
They nearly always indicate a poorly
thought out solution to a problem not
quite understood.

Quote:
> So, in my case, if I have a variable a = 5E-6 and I compare it to z, .NET
> will tell me that they are different even though they are different in the
> 54th bit? I have written so many lines of code assuming (you know what they
> say about this word) that if I compare 2 double precision numbers, this
> stuff is taken care of by the libraries. Do I now have to go through all of
> my code to see where I compare 2 double precision numbers and change it to
> something like:
> If (Math::Abs(a-z) < 1E-53)

> That sounds absurd, right?

Not as absurd as assuming that the
library writers would somehow have
been able to guess how close two FP
values should be before they cannot
be distinguished from each other by
equality tests.

Quote:
> Also, why should 5*1E-6 cause an error. If I was writing a library and I saw
> that the numbers that I need to represent have integral mantissas, I will
> not do any special conversions. I would just multiply the mantissas and
> shift the exponent accordingly.... Am I missing something or do I sound
> really stupid here?

You are missing many things, but I
would call it naivety rather than
stupidity.  Performance considerations
(among others) would argue against
your approach.

You have also missed that what you
call an exponent shift is no such
thing.  1E-6 does not correspond to
any number that can be represented
precisely in FP format.  The only
numbers that can be represented are
rationals with power-of-2 denominators
and integer numerators, both falling
within limited ranges.

--
-Larry Brasfield
(address munged, s/sn/h/ to reply)



Wed, 12 Oct 2005 05:34:26 GMT  
 Double Precision bug??

Quote:



>> You're not losing precision - you're misunderstanding the precision
>> of floating point math.

>> The error you have indicated here, is approximately 2e-53, which is
>> the normalized precision of double.  This is a natural consequence
>> of the fact that double has a 53-bit mantissa.

>> Double::Episilon is the smallest value that Double can represent
>> (4.94065645841247e-324).  This is not the same as the precision that
>> a calculation performed on double is guaranteed to provide.

>> -cd

> Alright, I am ok with this. Now, shouldn't the (==) and (!=) operator
> take that into account? If I am building a new type and I know that I
> cannot precisely represent a number beyond a certain number of bits,
> I should not say that 2 numbers are not equal if I cannot guarantee
> that they are different?

The problem is that floating point is an inexact representation, unlike
integer, so saying that two floating point number are "equal" is rarely
useful.  Instead, what you typically need to know is whether two floating
point numbers are equivalent, within the precision requirements of your
problem domain.  Of course, only you know what those precision requirements
are - there's no way a library could provide one.  Equality, as implemented
by the hardware (the System.Double class is just a facade - all the real
work is implemented directly in silicon) simply compares the bit-patterns of
two numbers.  If all the bits are the same, the numbers are "equal".
(Before someone pipe up - yes, there are special cases where that's not
true - NaN == NaN is never evaluated as true even if the two NaNs have
identical bit patterns).

Quote:
> So, in my case, if I have a variable a = 5E-6 and I compare it to z,
> .NET will tell me that they are different even though they are
> different in the 54th bit? I have written so many lines of code
> assuming (you know what they say about this word) that if I compare 2
> double precision numbers, this stuff is taken care of by the
> libraries. Do I now have to go through all of my code to see where I
> compare 2 double precision numbers and change it to something like:
> If (Math::Abs(a-z) < 1E-53)

> That sounds absurd, right?

It may sound absurd, but that's exactly what you have to do, if "equality"
is important to your program.  Because floating point is an inexact
representation, you rarely want to compare for equality - you want to
compare for equivalence within the precision requirements of your program.

Quote:

> Also, why should 5*1E-6 cause an error. If I was writing a library
> and I saw that the numbers that I need to represent have integral
> mantissas, I will not do any special conversions. I would just
> multiply the mantissas and shift the exponent accordingly.... Am I
> missing something or do I sound really stupid here?

Floating point formats (at least on modern CPUs) are always based on binary
representations.  No negative power of 10 can be accurately represented in
binary, so anytime you convert a decimal number to a floating point format,
the result is inexact.  As Larry already replied, only numbers that can be
represented as a rational number with a power-of-two denominator can be
represented accurately in floating point.

You might want to find one of the classic works on computer programming,
such as Donald Knuth's "The Art of Computer Science" (volume 2, chapter 4)
and read up on the fundamentals and theory of floating point arithmetic.

-cd



Wed, 12 Oct 2005 06:10:28 GMT  
 Double Precision bug??

Quote:

>> Are you using some kind of DirectX in you program ?

> Not that I am aware of. I believe a part of our application (maybe a
> control in the GUI) might be using DirectX, but I cannot vouch for
> them. My stuff is written in Managed C++ (VC++ .NET). I can guarantee
> that I absolutely don't use any DirectX :)

If someone is using DirectX, then you have a problem!!!

If someone calls DirectX::CreateDevice then this method will change the
precison from 54 to 26 bits... this has process wide consequences...

All floating-point operations have now only an precision of 26 bits!!!!

There are two solutions:
1. While calling CreateDevice you have to specify that this method should
not change the floating-pointer control word

2. Every time you need more precision, change the PC (Precision Control) of
the FPU to 56 bits. And after that restore it...

--
Greetings
  Jochen

   Do you need a memory-leak finder ?
   http://www.codeproject.com/useritems/leakfinder.asp



Wed, 12 Oct 2005 20:35:35 GMT  
 Double Precision bug??
Larry, Carl, Craig and Jochen,

Thanks a lot for your responses. I think I have a better understanding of my
problem now. All my current experience has been with Fixed Point Embedded
Systems CPUs and some of these concepts were new to me. I don't believe a
lot of harm has been done currently, however I am going to change some of my
comparisons to do exactly what I had mentioned (use a threshold on the
difference)

I found a really good page that explains these concepts if anybody is
interested:
http://mindprod.com/jglossfloatingpoint.html

MT



Wed, 12 Oct 2005 22:05:30 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. printf precision bug wrt floats, doubles

2. precision of a double ?

3. long long integer and double precision number

4. Details of double precision type

5. double multiply precision question

6. double precision

7. Precision in float to double conversion

8. precision of a double ?

9. writing out double precision numbers?

10. Higher precision than double

11. Double-precision variable question

12. precision of float vs. double

 

 
Powered by phpBB® Forum Software