VB Floating Point Bugs ?? 
Author Message
 VB Floating Point Bugs ??

I have just encountered a problem with VB 4.0 16bit (Enterprise Edit) when
I am trying
to sum a series of non-integers, the result is quit strange and I am wonder
if this is my
fault or its the problem of VB4.0 ??

Here is the codes :

Dim a As Double
a = 0#
a = a + 62.3797
a = a + 27.6776
a = a + 4.2657
a = a + 2.4969
a = a + 1.8113
a = a + 1.0856
a = a + 0.2832

If a <> 100 Then
  MsgBox format(a)+" not equal 100"
End If

The result should be 100.0 however, the program keeps going into the IF
control, is it a bugs of VB4.0 ??
Any comments are welcome, pls reply to my email address as well.

--
eRic

http://www.*-*-*.com/ ~eeee



Sun, 07 Nov 1999 03:00:00 GMT  
 VB Floating Point Bugs ??

When using doubles, never compare with a long (100). Use 100.0. This may
still not solve your problem.

The best way to do this is to first convert the number to a string and then
convert back to a double

example: Lets say you want to compare a with 100.0 to 4 decimal places,
then do this:

if cdbl(format(a, "0.0000")) <> 0. then ...

Hope this helps

Andy



Quote:
> I have just encountered a problem with VB 4.0 16bit (Enterprise Edit)
when
> I am trying
> to sum a series of non-integers, the result is quit strange and I am
wonder
> if this is my
> fault or its the problem of VB4.0 ??

> Here is the codes :

> Dim a As Double
> a = 0#
> a = a + 62.3797
> a = a + 27.6776
> a = a + 4.2657
> a = a + 2.4969
> a = a + 1.8113
> a = a + 1.0856
> a = a + 0.2832

> If a <> 100 Then
>   MsgBox format(a)+" not equal 100"
> End If

> The result should be 100.0 however, the program keeps going into the IF
> control, is it a bugs of VB4.0 ??
> Any comments are welcome, pls reply to my email address as well.

> --
> eRic

> http://www.asiaonline.net/users/~eeee



Sun, 07 Nov 1999 03:00:00 GMT  
 VB Floating Point Bugs ??

I tried this code and inserted a "debug.print a-100" into the If statement.
I got:
-1.4210854715202E-14
Yikes.
--
Andy Hopper
Research Engineer
Georgia Institute of Technology
Remove leading "ath_" from email address to reply via email.



Sun, 07 Nov 1999 03:00:00 GMT  
 VB Floating Point Bugs ??

I ran into math problems before in 16 bit Windows (in VB, PB, and others)
try the same calculations in Calculator to determine if this is the case. I
can't remember the specific problem, but it occurred in very simple
calculations.

Since, I try to do all calculations in the backend stored procedures...



Mon, 08 Nov 1999 03:00:00 GMT  
 VB Floating Point Bugs ??

This is a common problem. As you already may have heard, computers have
trouble representing fractions. eg represent 1/3 as a binary floating point
number is never right on the money, even in base 10 it's
333333333333333333333 ad-infintum.

Not familiar with VB persay but in other languages, yes. I noticed all your
numbers you add are 4 decimal places. If you need 4 decimal accuracy, you
could just dim 'a' as an double integer and multiply everything you want to
add to 'a' by 10,000 (ten thousand). When done all the adding, take the
integer result, and divide it by 10,000 (a/10000). The answer will be an
integer devoid of any fractional amount. Your 'if a<>100 then' should pass.
This probably may or may not solve tour problem, but I thought I'd throw it
out at ya.

There are other ways to skin a cat. The problem is to find out why you skin
the cat in the first place ;-)



Quote:
> I have just encountered a problem with VB 4.0 16bit (Enterprise Edit)
when
> I am trying
> to sum a series of non-integers, the result is quit strange and I am
wonder
> if this is my
> fault or its the problem of VB4.0 ??

> Here is the codes :

> Dim a As Double
> a = 0#
> a = a + 62.3797
> a = a + 27.6776
> a = a + 4.2657
> a = a + 2.4969
> a = a + 1.8113
> a = a + 1.0856
> a = a + 0.2832

> If a <> 100 Then
>   MsgBox format(a)+" not equal 100"
> End If

> The result should be 100.0 however, the program keeps going into the IF
> control, is it a bugs of VB4.0 ??
> Any comments are welcome, pls reply to my email address as well.

> --
> eRic

> http://www.asiaonline.net/users/~eeee



Mon, 08 Nov 1999 03:00:00 GMT  
 VB Floating Point Bugs ??

On Wed, 21 May 1997 07:00:59 -0700, "Andrew Hopper"

Quote:

>I tried this code and inserted a "debug.print a-100" into the If statement.
>I got:
>-1.4210854715202E-14
>Yikes.

That's fine and expected.  If you add a bunch of 99.9999 numbers in
floating point, in any language, on any computer that uses IEEE
standard floating point, your comparison should be not "if sum = 100."
but "if abs(sum - 100.) < .00001".  That's as good as it gets.
1.4e-14 is much smaller than 1e-5, so you're fine.

Scaled integers can give you better results, I believe that's what the
currency datatype is under the covers.  You can also scale integers
but store them in floating point numbers, but that's an advanced topic
<g>.

Josh



Tue, 09 Nov 1999 03:00:00 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. VB Floating Point Bugs ??

2. Converting floating point number to IEEE 32-bit precision floating point format

3. Floating Point bug in VB ?

4. VB BUG in floating point handling !

5. Floating point property bug in form designer

6. Floating Point bug in VB4 ?

7. Converting floating number to the IEEE 32/40-bit precision floating point format

8. floating point precision - VB vs C

9. floating point precision - VB vs C

10. C code to VB - Floating point question

11. Serious problems with floating point calculations in VB

12. Floating Point Speed - Comparison of VB 5.0, Delphi 3.0, and VC 5.0

 

 
Powered by phpBB® Forum Software