Bug involving Double Values in SP5

Quote:

> Another developer here had an interesting idea. I

> subtracted the two numbers from each other. I should get 0

> yes? The following snippets are from my immediate window.

> ?dz3PCheckSum-dzCheckSum

> -16

> Here are some more interesting calculations

> dzCheckSum = dzCheckSum / 100000

> dz3PCheckSum = dz3PCheckSum / 100000

> ?dzCheckSum - dz3PCheckSum

> 0.0001220703125

> ?dzCheckSum

> 572337356845.286

> ?dz3PCheckSum

> 572337356845.286

> Something is very inconsitent either VB's IDE is reporting

> the wrong values and the decimals are much larger(Although

> the true values I started out with are not decimal values

> and the decimals only exist because of the dividing, and I

> tried to round the initial doubles, and tried an INT())

> The values did become equal once I divided them twice by

> 100000 and then did an INT() on them. Perhaps it has

> something do with how VB is comparing very large doubles?

> I am trying to calculate a Check Sum here which means this

> is somewhat crucial, that I am able to know for sure that

> the two numbers are equall.

> Any thoughts comments or ideas are definelty appreciated.

This is all a consequence of storing an infinite range of

numbers in a finite number of bytes. No floating point system

is free of this, though there are different exhibitions of the

behavior.

When you print the numbers, they're being formatted for display

and will likely be less precise the the "real" number. This can

make two number appear to be equal.

You can see this fairly easily with this comparison:

Is (5 / 3) * 3 = (5 * 3) / 3 true?

If you calculate 5 / 3, you get

1.666666666666666666666666666666666666666666666666666666666 (ad

naseum)

You cannot, in a finite space, write down this number. To

approximate it, you can call it 1.66666 .... 66666667, but when

you multiply it by 3, you should expect a number slightly larger

than 5, because of the rounding.

5 * 3 can easily be written as 15. Dividing by 3 gets us right

back to 5.