arithmatic precision problems

This posting is provided "AS IS" with no warranties, and confers no rights.

Quote:

> Does vb.net suffer from the arithmetic precision problems that vb 6

suffered

> from (i.e. because IEEE standards were followed not all decimal numbers

> could be represented in binary so the closest approximation to the decimal

> number in binary was chosen to represent it).

The short answer is "Yes."

The longer answer is there is no practical way to avoid this. And that

virtually

*all* computer languages "suffer" in this same way.

It's in the nature of the need to approximate answers in one base when

working in a different base.

To pick an example, try doing some *decimal* arithmetic, by hand.

Here's your task:

Write out the answer to 1 divided by 3 to the number of digits needed

to achieve absolute accuracy.

You can't do it.

You can write it to as many digits of accuracy as you want, but wherever

you leave off, you will have "rounded" the answer:

0.333333333333333333333333333333333... forever

Now, *IF* you could use "trinary" (base 3) arithmetic, this would *not*

be true. 1 divided by 3 in trinary arithmetic is

0.1

So the answer to the problem we gave you is simple:

Use trinary arithmetic if you want the exact answer to 1 divided by 3.

And if you want the answer to 1 divided by 10 exactly, use decimal

arithmetic.

But the hardware of the computer you are running on doesn't *have* any

decimal arithmetic capabilities. All it has is binary arithmetic. And the

closest we can get to 1 divided by 10 in binary arithmetic is

0.0001100110011001100110011001100110011.... forever

And that is the root of the "problems" that you perceive with IEEE

floating point: They are rooted in the need to *approximate* decimal

answers using binary numbers.

To change this, you'd need to find a computer that did decimal arithmetic.

Once upon a time (in the 1960s and early 1970s) there were machines

that did so. They were simply too slow and cumbersome. Today, all

computers do IEEE floating point.

Having said all that...

There *are* partial solutions. For example, if you are working with

money, work with pennies. Instead of $1.98, store 198 pennies. Do

all your math in pennies (using integer numbers, only). And then adjust

your answer to put the decimal point in the right place. This is, for

example,

what the "Currency" data type in Access uses. (Except it always uses

1/100 of a penny as the unit, no choice. So that the decimal point is

always implicitly 4 digits to the left. But the concept is the same.)

This posting is provided "AS IS" with no warranties, and confers no rights.

MS VB.Net Team