Bug involving Double Values in SP5 
Author Message
 Bug involving Double Values in SP5

Hello,

I'm experiencing a situation which is simillar to the bug
in KB article Q257496. This bug is mentioned in KB Article
Q261144 that it was fixed in SP4.

What I am seeing is if I have two double values which in
this example have the value 5.72337356845286E+16 and I
compare the two in an If they do not compare as being
equal.

If I convert the doubles using FormatNumber$(dzCheckSum,
0, , , vbFalse), they do compare as being equal. I can
also wrap the doubles in a Val() statment and my
comparision evalutes them to be equal.

I am not using any optimizations, This is an ActiveX DLL
with the following settings

Retained in memory
Upgrade ActiveX Controls
Unattended execution
Apartment threaded.
Auto Increment Version
Default settings for DLL Address and Optimizations(I've
also tried turning off optimizations)

I am using VB6 SP5 on Win2k SP1. Can anyone else confirm
this or have I just lost my mind? Any suggestions besides
hacking around this and converting the values to a string
or using a Val()? I'd rather not take the extra
performance hits but will if I have no other choice.

Josh Berke



Tue, 27 Jan 2004 06:39:32 GMT  
 Bug involving Double Values in SP5

Quote:
> I'm experiencing a situation which is simillar to the bug
> in KB article Q257496. This bug is mentioned in KB Article
> Q261144 that it was fixed in SP4.

> What I am seeing is if I have two double values which in
> this example have the value 5.72337356845286E+16 and I
> compare the two in an If they do not compare as being
> equal.

This isn't a "bug" as such, it's normal for any operation using floating
point values.  Instead of comparing equality, test for the difference being
less than some small delta...

if abs(a-b)<1e-10 then...



Tue, 27 Jan 2004 06:55:31 GMT  
 Bug involving Double Values in SP5


Fri, 19 Jun 1992 00:00:00 GMT  
 Bug involving Double Values in SP5
Another developer here had an interesting idea. I
subtracted the two numbers from each other. I should get 0
yes? The following snippets are from my immediate window.

?dz3PCheckSum-dzCheckSum
-16

Here are some more interesting calculations

dzCheckSum = dzCheckSum / 100000
dz3PCheckSum = dz3PCheckSum / 100000
?dzCheckSum - dz3PCheckSum
 0.0001220703125

?dzCheckSum
 572337356845.286

?dz3PCheckSum
 572337356845.286

Something is very inconsitent either VB's IDE is reporting
the wrong values and the decimals are much larger(Although
the true values I started out with are not decimal values
and the decimals only exist because of the dividing, and I
tried to round the initial doubles, and tried an INT())

The values did become equal once I divided them twice by
100000 and then did an INT() on them. Perhaps it has
something do with how VB is comparing very large doubles?
I am trying to calculate a Check Sum here which means this
is somewhat crucial, that I am able to know for sure that
the two numbers are equall.

Any thoughts comments or ideas are definelty appreciated.



Tue, 27 Jan 2004 07:04:41 GMT  
 Bug involving Double Values in SP5


Fri, 19 Jun 1992 00:00:00 GMT  
 Bug involving Double Values in SP5
There is no room for error in this. If the two sums do not
match then I have to assume that the data I am using is
corrupt.

Here's what I don't like I tested the delta it was -16. I
divde both numbers by a 100000 the difference becomes  
0.0001220703125.

I don't understand how this can not be a bug, and should
be expected in floating point operations? How can the same
value generated by the same algorithim with the same seed
have a delta of -16 but the string values of the number be
identicle?

Quote:
>-----Original Message-----

message

>> I'm experiencing a situation which is simillar to the
bug
>> in KB article Q257496. This bug is mentioned in KB
Article
>> Q261144 that it was fixed in SP4.

>> What I am seeing is if I have two double values which in
>> this example have the value 5.72337356845286E+16 and I
>> compare the two in an If they do not compare as being
>> equal.

>This isn't a "bug" as such, it's normal for any operation
using floating
>point values.  Instead of comparing equality, test for

the difference being

- Show quoted text -

Quote:
>less than some small delta...

>if abs(a-b)<1e-10 then...

>.



Tue, 27 Jan 2004 07:46:36 GMT  
 Bug involving Double Values in SP5


Fri, 19 Jun 1992 00:00:00 GMT  
 Bug involving Double Values in SP5


Quote:
> Another developer here had an interesting idea. I
> subtracted the two numbers from each other. I should get 0
> yes? The following snippets are from my immediate window.

> ?dz3PCheckSum-dzCheckSum
> -16

> Here are some more interesting calculations

> dzCheckSum = dzCheckSum / 100000
> dz3PCheckSum = dz3PCheckSum / 100000
> ?dzCheckSum - dz3PCheckSum
>  0.0001220703125

> ?dzCheckSum
>  572337356845.286

> ?dz3PCheckSum
>  572337356845.286

> Something is very inconsitent either VB's IDE is reporting
> the wrong values and the decimals are much larger(Although
> the true values I started out with are not decimal values
> and the decimals only exist because of the dividing, and I
> tried to round the initial doubles, and tried an INT())

> The values did become equal once I divided them twice by
> 100000 and then did an INT() on them. Perhaps it has
> something do with how VB is comparing very large doubles?
> I am trying to calculate a Check Sum here which means this
> is somewhat crucial, that I am able to know for sure that
> the two numbers are equall.

> Any thoughts comments or ideas are definelty appreciated.

This is all a consequence of storing an infinite range of
numbers in a finite number of bytes.  No floating point system
is free of this, though there are different exhibitions of the
behavior.

When you print the numbers, they're being formatted for display
and will likely be less precise the the "real" number.  This can
make two number appear to be equal.

You can see this fairly easily with this comparison:

Is (5 / 3) * 3 = (5 * 3) / 3 true?

If you calculate 5 / 3, you get
1.666666666666666666666666666666666666666666666666666666666 (ad
naseum)

You cannot, in a finite space, write down this number.  To
approximate it, you can call it 1.66666 .... 66666667, but when
you multiply it by 3, you should expect a number slightly larger
than 5, because of the rounding.

5 * 3 can easily be written as 15.  Dividing by 3 gets us right
back to 5.



Tue, 27 Jan 2004 08:37:12 GMT  
 Bug involving Double Values in SP5


Fri, 19 Jun 1992 00:00:00 GMT  
 Bug involving Double Values in SP5
very simple. Since the DOUBLE type is using a floating point, the numbers
behind the dot will not be exact...
so If you try 103030304040.0304049494# - 393934943439393.102394020 you won't
get an exact answer.... Use the currency instead to get an exact answer up
to 4 after the dot.

use floating point only for graphs and math not for financial calculations.
--
Egbert Nierop

Session management for webfarms:
http://www.nieropwebconsult.nl/asp_session_manager.htm

Quote:
> There is no room for error in this. If the two sums do not
> match then I have to assume that the data I am using is
> corrupt.

> Here's what I don't like I tested the delta it was -16. I
> divde both numbers by a 100000 the difference becomes
> 0.0001220703125.

> I don't understand how this can not be a bug, and should
> be expected in floating point operations? How can the same
> value generated by the same algorithim with the same seed
> have a delta of -16 but the string values of the number be
> identicle?

> >-----Original Message-----

> message

> >> I'm experiencing a situation which is simillar to the
> bug
> >> in KB article Q257496. This bug is mentioned in KB
> Article
> >> Q261144 that it was fixed in SP4.

> >> What I am seeing is if I have two double values which in
> >> this example have the value 5.72337356845286E+16 and I
> >> compare the two in an If they do not compare as being
> >> equal.

> >This isn't a "bug" as such, it's normal for any operation
> using floating
> >point values.  Instead of comparing equality, test for
> the difference being
> >less than some small delta...

> >if abs(a-b)<1e-10 then...

> >.



Tue, 27 Jan 2004 17:58:21 GMT  
 Bug involving Double Values in SP5
Perhaps the following KB article will shed some light on this for you.

(Complete) Tutorial to Understand IEEE Floating-Point Errors
Article ID: Q42980
http://support.microsoft.com/support/kb/articles/Q42/9/80.ASP

Mike


Quote:
> I don't understand how this can not be a bug, and should
> be expected in floating point operations? How can the same
> value generated by the same algorithim with the same seed
> have a delta of -16 but the string values of the number be
> identicle?



Tue, 27 Jan 2004 19:40:46 GMT  
 Bug involving Double Values in SP5
Hi,

You cannot use a computer and floating-point arithmetic to do exact
calculation.  This has nothing to do with language or implementation.  In
fact, when you get down to facts, you cannot do exact arithmetic with
floating point numbers even with pencil and paper.  If you need exact
arithmetic, you must use a fixed-point notation.  This limits you dynamic
range, naturally.

--
Richard Grier (Microsoft Visual Basic MVP)
Hard & Software
12962 West Louisiana Avenue
Lakewood, CO  80228
303-986-2179 (voice)
303-986-3143 (fax)
Author of Visual Basic Programmer's Guide to Serial Communications, 2nd
Edition ISBN 1-890422-25-8 (355 pages).
For information look on my homepage at http://www.hardandsoftware.net.
Use the Books link to order.  For faster service contact the publisher at
http://www.mabry.com/vbpgser.



Tue, 27 Jan 2004 21:50:34 GMT  
 Bug involving Double Values in SP5
Ok so then the problem I see is how to accomplish
mathmatic functions with very large whole numbers. I don't
need floating points however a long value was not large
enough and I keept causing buffer over flows.

Quote:
>-----Original Message-----
>very simple. Since the DOUBLE type is using a floating
point, the numbers
>behind the dot will not be exact...
>so If you try 103030304040.0304049494# -

393934943439393.102394020 you won't
Quote:
>get an exact answer.... Use the currency instead to get
an exact answer up
>to 4 after the dot.

>use floating point only for graphs and math not for

financial calculations.
Quote:
>--
>Egbert Nierop

>Session management for webfarms:
>http://www.nieropwebconsult.nl/asp_session_manager.htm

message

>> There is no room for error in this. If the two sums do
not
>> match then I have to assume that the data I am using is
>> corrupt.

>> Here's what I don't like I tested the delta it was -16.
I
>> divde both numbers by a 100000 the difference becomes
>> 0.0001220703125.

>> I don't understand how this can not be a bug, and should
>> be expected in floating point operations? How can the
same
>> value generated by the same algorithim with the same
seed
>> have a delta of -16 but the string values of the number
be
>> identicle?

>> >-----Original Message-----

>> message

>> >> I'm experiencing a situation which is simillar to the
>> bug
>> >> in KB article Q257496. This bug is mentioned in KB
>> Article
>> >> Q261144 that it was fixed in SP4.

>> >> What I am seeing is if I have two double values
which in
>> >> this example have the value 5.72337356845286E+16 and
I
>> >> compare the two in an If they do not compare as being
>> >> equal.

>> >This isn't a "bug" as such, it's normal for any
operation
>> using floating
>> >point values.  Instead of comparing equality, test for
>> the difference being
>> >less than some small delta...

>> >if abs(a-b)<1e-10 then...

>> >.

>.



Tue, 27 Jan 2004 22:08:39 GMT  
 Bug involving Double Values in SP5

Quote:
> Ok so then the problem I see is how to accomplish
> mathmatic functions with very large whole numbers. I don't
> need floating points however a long value was not large
> enough and I keept causing buffer over flows.

The Decimal data type may be what you need.  It maintains 28 (27? something
like that) digits accurately whether they are left or right of the decimal
point.  The only catch is that you have to use Variants to used them:

dim a,b
a=cdec(100)
b=cdec(3)
a=a/b



Tue, 27 Jan 2004 22:57:43 GMT  
 Bug involving Double Values in SP5


Fri, 19 Jun 1992 00:00:00 GMT  
 
 [ 19 post ]  Go to page: [1] [2]

 Relevant Pages 

1. VBScript bug involving variable scope

2. VBScript bug involving variable scope

3. Bug - Workaround Needed VB 6 SP5

4. SERIOUS BUG IN MSVBVM60 (SP5)

5. Duplicate definition bug VB6pro SP5

6. BUG in VB6 SP5

7. VB sp5 Bug cannot load control mnutools

8. VB6 SP5 bug - Dataprojects failing to load

9. SP5 bug fixes

10. Connection Point Bug in VB6, SP5?

11. Bug in VB6 (SP5) with MAPI/OE Control/Api

12. Connection Point Bug in VB6, SP5?

 

 
Powered by phpBB® Forum Software