arithmatic precision problems 
Author Message
 arithmatic precision problems

Does vb.net suffer from the arithmetic precision problems that vb 6 suffered
from ? (i.e. because IEEE standards were followed not all decimal numbers
could be represented in binary so the closest approximation to the decimal
number in binary was chosen to represent it).

Thanks,

David



Sat, 29 Jan 2005 02:45:35 GMT  
 arithmatic precision problems
That's got nothing to do with IEEE or VB 6 or VB .NET.
Any representation of real/single/double floating point numbers has the same
issues.
IEEE is one means of representing the numbers internally.
--
http://www.standards.com/; Programming and support for  Word macros,
including converting from WordBasic to VBA; Technical reviewing; Standards;
Product functional/design/specifications
------------------------------------------------

Quote:
> Does vb.net suffer from the arithmetic precision problems that vb 6
suffered
> from ? (i.e. because IEEE standards were followed not all decimal numbers
> could be represented in binary so the closest approximation to the decimal
> number in binary was chosen to represent it).

> Thanks,

> David



Sat, 29 Jan 2005 04:32:25 GMT  
 arithmatic precision problems

This posting is provided "AS IS" with no warranties, and confers no rights.

Quote:
> Does vb.net suffer from the arithmetic precision problems that vb 6
suffered
> from (i.e. because IEEE standards were followed not all decimal numbers
> could be represented in binary so the closest approximation to the decimal
> number in binary was chosen to represent it).

The short answer is "Yes."

The longer answer is there is no practical way to avoid this.  And that
virtually
*all* computer languages "suffer" in this same way.

It's in the nature of the need to approximate answers in one base when
working in a different base.

To pick an example, try doing some *decimal* arithmetic, by hand.
Here's your task:

Write out the answer to 1 divided by 3 to the number of digits needed
to achieve absolute accuracy.

You can't do it.

You can write it to as many digits of accuracy as you want, but wherever
you leave off, you will have "rounded" the answer:
                0.333333333333333333333333333333333... forever

Now, *IF* you could use "trinary" (base 3) arithmetic, this would *not*
be true.  1 divided by 3 in trinary arithmetic is
                0.1

So the answer to the problem we gave you is simple:
Use trinary arithmetic if you want the exact answer to 1 divided by 3.

And if you want the answer to 1 divided by 10 exactly, use decimal
arithmetic.

But the hardware of the computer you are running on doesn't *have* any
decimal arithmetic capabilities.  All it has is binary arithmetic.  And the
closest we can get to 1 divided by 10 in binary arithmetic is
               0.0001100110011001100110011001100110011.... forever

And that is the root of the "problems" that you perceive with IEEE
floating point:  They are rooted in the need to *approximate* decimal
answers using binary numbers.

To change this, you'd need to find a computer that did decimal arithmetic.
Once upon a time (in the 1960s and early 1970s) there were machines
that did so.  They were simply too slow and cumbersome.  Today, all
computers do IEEE floating point.

Having said all that...

There *are* partial solutions.  For example, if you are working with
money, work with pennies.  Instead of $1.98, store 198 pennies.  Do
all your math in pennies (using integer numbers, only).  And then adjust
your answer to put the decimal point in the right place.  This is, for
example,
what the "Currency" data type in Access uses.  (Except it always uses
1/100 of a penny as the unit, no choice.  So that the decimal point is
always implicitly 4 digits to the left.  But the concept is the same.)

This posting is provided "AS IS" with no warranties, and confers no rights.

MS VB.Net Team



Sat, 29 Jan 2005 05:13:18 GMT  
 arithmatic precision problems
So what your saying is that all computer languages suffer from this problem,
which has to do with representing decimal numbers in binary ?


Quote:
> That's got nothing to do with IEEE or VB 6 or VB .NET.
> Any representation of real/single/double floating point numbers has the
same
> issues.
> IEEE is one means of representing the numbers internally.
> --
> http://www.standards.com/; Programming and support for  Word macros,
> including converting from WordBasic to VBA; Technical reviewing;
Standards;
> Product functional/design/specifications
> ------------------------------------------------


> > Does vb.net suffer from the arithmetic precision problems that vb 6
> suffered
> > from ? (i.e. because IEEE standards were followed not all decimal
numbers
> > could be represented in binary so the closest approximation to the
decimal
> > number in binary was chosen to represent it).

> > Thanks,

> > David



Sat, 29 Jan 2005 05:16:28 GMT  
 arithmatic precision problems
Yes.

FYI: I was one of the original/early participants on the committee that came
up with the IEEE standard. Started back in 1978 or so.

--
http://www.standards.com/; Programming and support for  Word macros,
including converting from WordBasic to VBA; Technical reviewing; Standards;
Product functional/design/specifications
------------------------------------------------

Quote:
> So what your saying is that all computer languages suffer from this
problem,
> which has to do with representing decimal numbers in binary ?



> > That's got nothing to do with IEEE or VB 6 or VB .NET.
> > Any representation of real/single/double floating point numbers has the
> same
> > issues.
> > IEEE is one means of representing the numbers internally.
> > --
> > http://www.standards.com/; Programming and support for  Word macros,
> > including converting from WordBasic to VBA; Technical reviewing;
> Standards;
> > Product functional/design/specifications
> > ------------------------------------------------


> > > Does vb.net suffer from the arithmetic precision problems that vb 6
> > suffered
> > > from ? (i.e. because IEEE standards were followed not all decimal
> numbers
> > > could be represented in binary so the closest approximation to the
> decimal
> > > number in binary was chosen to represent it).

> > > Thanks,

> > > David



Sat, 29 Jan 2005 05:36:34 GMT  
 arithmatic precision problems
There's just too much new stuff, and quite frankly I haven't learned it all!

I'm the one who gave you the VBDotNet team answer.

And it's correct, as far as it goes.  (Basically, the same answer that
Howard Kaikow gave you.)

But it doesn't go far enough!

Imagine my chagrin to return to my office and be looking for something else
and to stumble upon...

The DECIMAL Data Type in VB.NET!

This is, for all intents and purposes, a much cleaner implementation of the
CURRENCY type of Access.  The advantage here is that you aren't limited to
only 4 digits after the decimal point.

Note that the Decimal data type in VB.Net is actually an implementation of
the System.Decimal class from the .Net framework, so see the framework docs
for all the methods available on this class.

But in addition to the available methods, VB.NET really *does* do the right
thing for basic operations:
          Dim d As Decimal = 7777.3333D
          Dim e As Decimal = 111.222D
          Dim f As Decimal = d * e
Yes, that really works.  And the answer is decimal accurate.  (And the D on
the end of numbers tells VB that the number is a Decimal constant.  You
*really* need to use that for all but integer constants.)

I would like to note that the Decimal type, implemented as a class using a
bunch of behind-the-scenes software, is going to be a *lot* slower than
using floating point.  But if you are working on, say, money-related issues
where you need to-the-penny accuracy...  Well, here it is.

So...  Thanks for educating me!   And maybe many more besides.

*************

This posting is provided "AS IS" with no warranties, and confers no rights.



Sat, 29 Jan 2005 06:56:01 GMT  
 arithmatic precision problems
One last comment...

I should note that even DECIMAL isn't going to help you if you try to get an
answer that is an infinitely repeating decimal number.

       Dim one as Decimal = 1.0D
       Dim three as Decimal = 3.0D
       Dim oneThird as Decimal = one / three

Ooops...  You can't get more than 28 digits after the decimal point with the
DECIMAL type, so...

This was probably obvious, but just in case somebody reading this thread go
her or his hopes up too high...

Bill

 This posting is provided "AS IS" with no warranties, and confers no rights.



Sat, 29 Jan 2005 07:03:25 GMT  
 arithmatic precision problems
The Decimal data type does not alter what I previously stated.
Decimal data has been used on computers since way back.
But Decimal data type is not a native hardware data type, nor is an octal
value nor a hexadecimal value.

Indeed, one of the first computers I ever programmed was the IBM 1620.
The internal code name within  IBM for the IBM 1620 was CADET.

Can't Add Don't Even Try

All arithmetic was decimal using a table lookup to do the arithmetic.

Later computers, such as the IBM 360 and IBM 370 used decimal arithmetic
based on representing each decimal digit using 4 bits.
Some computers have/had a commercial instruction set included in their
assembly languages to facilitate the hardware manipulation of such
representations.

--
http://www.standards.com/; Programming and support for  Word macros,
including converting from WordBasic to VBA; Technical reviewing; Standards;
Product functional/design/specifications
------------------------------------------------

Quote:
> There's just too much new stuff, and quite frankly I haven't learned it
all!

> I'm the one who gave you the VBDotNet team answer.

> And it's correct, as far as it goes.  (Basically, the same answer that
> Howard Kaikow gave you.)

> But it doesn't go far enough!

> Imagine my chagrin to return to my office and be looking for something
else
> and to stumble upon...

> The DECIMAL Data Type in VB.NET!

> This is, for all intents and purposes, a much cleaner implementation of
the
> CURRENCY type of Access.  The advantage here is that you aren't limited to
> only 4 digits after the decimal point.

> Note that the Decimal data type in VB.Net is actually an implementation of
> the System.Decimal class from the .Net framework, so see the framework
docs
> for all the methods available on this class.

> But in addition to the available methods, VB.NET really *does* do the
right
> thing for basic operations:
>           Dim d As Decimal = 7777.3333D
>           Dim e As Decimal = 111.222D
>           Dim f As Decimal = d * e
> Yes, that really works.  And the answer is decimal accurate.  (And the D
on
> the end of numbers tells VB that the number is a Decimal constant.  You
> *really* need to use that for all but integer constants.)

> I would like to note that the Decimal type, implemented as a class using a
> bunch of behind-the-scenes software, is going to be a *lot* slower than
> using floating point.  But if you are working on, say, money-related
issues
> where you need to-the-penny accuracy...  Well, here it is.

> So...  Thanks for educating me!   And maybe many more besides.

> *************

> This posting is provided "AS IS" with no warranties, and confers no
rights.



Sat, 29 Jan 2005 10:21:36 GMT  
 arithmatic precision problems
Take a look at MSFT KB article Q42980.
I've not read the article, so I cannot vouch for its correctness, but it
gives a useful introduction to this problem.

--
http://www.standards.com/; Programming and support for  Word macros,
including converting from WordBasic to VBA; Technical reviewing; Standards;
Product functional/design/specifications
------------------------------------------------

Quote:
> Yes.

> FYI: I was one of the original/early participants on the committee that
came
> up with the IEEE standard. Started back in 1978 or so.

> --
> http://www.standards.com/; Programming and support for  Word macros,
> including converting from WordBasic to VBA; Technical reviewing;
Standards;
> Product functional/design/specifications
> ------------------------------------------------


> > So what your saying is that all computer languages suffer from this
> problem,
> > which has to do with representing decimal numbers in binary ?



> > > That's got nothing to do with IEEE or VB 6 or VB .NET.
> > > Any representation of real/single/double floating point numbers has
the
> > same
> > > issues.
> > > IEEE is one means of representing the numbers internally.
> > > --
> > > http://www.standards.com/; Programming and support for  Word macros,
> > > including converting from WordBasic to VBA; Technical reviewing;
> > Standards;
> > > Product functional/design/specifications
> > > ------------------------------------------------


> > > > Does vb.net suffer from the arithmetic precision problems that vb 6
> > > suffered
> > > > from ? (i.e. because IEEE standards were followed not all decimal
> > numbers
> > > > could be represented in binary so the closest approximation to the
> > decimal
> > > > number in binary was chosen to represent it).

> > > > Thanks,

> > > > David



Sat, 29 Jan 2005 10:24:14 GMT  
 arithmatic precision problems
Right.

Decimal data type also has the issue of a finite number of digits in the
representation of the number.

--
http://www.standards.com/; Programming and support for  Word macros,
including converting from WordBasic to VBA; Technical reviewing; Standards;
Product functional/design/specifications
------------------------------------------------

Quote:
> One last comment...

> I should note that even DECIMAL isn't going to help you if you try to get
an
> answer that is an infinitely repeating decimal number.

>        Dim one as Decimal = 1.0D
>        Dim three as Decimal = 3.0D
>        Dim oneThird as Decimal = one / three

> Ooops...  You can't get more than 28 digits after the decimal point with
the
> DECIMAL type, so...

> This was probably obvious, but just in case somebody reading this thread
go
> her or his hopes up too high...

> Bill

>  This posting is provided "AS IS" with no warranties, and confers no
rights.



Sat, 29 Jan 2005 10:30:29 GMT  
 arithmatic precision problems

Quote:
> Does vb.net suffer from the arithmetic precision problems that
> vb 6 suffered from ? (i.e. because IEEE standards were followed
> not all decimal numbers could be represented in binary so the
> closest approximation to the decimal number in binary was chosen
> to represent it).

"This behavior is by design". Floating point numbers (Single, Double...) can
never be 100% accurate.

There's the decimal data type now - not only as a Variant sub type - that
can handle non-integer values without loss of accuracy (within a certain
range, i.e. 28 decimal places)

Armin



Sat, 29 Jan 2005 18:16:43 GMT  
 
 [ 20 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Precision problem

2. Arithmatic in a Database

3. help with date arithmatic

4. Precision problems / bug in Visual Basic

5. Arithmatic on TIME data?

6. MS SQL and ADC - Decimal Precision Problems

7. VB5 Precision problem

8. VB5 Precision problem

9. Decimal precision and square root problem - VB 2008 Express

10. VB and VC++ (Single-to-Float type) precision problem

11. VB5 Precision problem

12. VB5 Precision problem

 

 
Powered by phpBB® Forum Software