Hi there

I found an error in a function using int(). See below.

?(3.28-3)*100
 28
It's correct!

?int(28)
 28
Correct !

?int((3.28-3)*100)
 27
Funny isn't it?

On Wed, 25 Apr 2001 14:31:57 +0200, "CHEVRIER JOHANN"

It is not an error.  ((3.28-3)*100) calculates out to be about
27.99999999999998 internaally which is just rounded up to 28 when
converting from the internal binary to decimal for the display.

This means that  
    int((3.28-3)*100) = 27        
is acceptible.

When you are using floating point multiplications and divisions, you
always have to accept rounding errors.  If you do not want errors, use
an integer-based type like Currency of Decimal.

Try

?int((cdec(3.28)-cdec(3))*cdec(100))

Not exactly an error with the int() function.  More a  problem with the way
computers store numbers internally.  Binary representation cannot
accurately store some decimal numbers (like .28).  Here is a general
algorithm for how to convert decimal base-10 numbers to base-2.

OLD base 10 number: .28000       NEW base 2 number:

Multiply the OLD number by the NEW BASE.
     - the digits to the left of the decimal point become a digit in the
new number.
     - the digits to the right of the decimal point become become the OLD
on the next line.
     - continue until OLD is .0000.

  OLD                                     NEW
.2800 * 2 = 0.5600                .0
.5600 * 2 = 1.1200                .01
.1200 * 2 = 0.2400                .010  
.2400 * 2 = 0.4800                .0100
.4800 * 2 = 0.9600                .01000
.9600 * 2 = 1.9200                .010001
.9200 * 2 = 1.8400                .0100011
.8400 * 2 = 1.6800                .01000111         (I believe this is the
point where the precision is truncated)
.6800 * 2 = 1.3600                .010001111
.3600 * 2 = 0.7200                .0100011110
.7200 * 2 = 1.4400                .01000111101
.4400 * 2 = 0.8800                .010001111010
.8800 * 2 = 1.7600                .0100011110101

If you carried this out farther I believe you would find a point where it
would start repeating and never get to 0.  Thus, the internal
representation for your sample is almost .28, but not quite exact.  Thus
when you multiply by 100 and take the int() you get 27.9999999 something.

Not that this helps correct any problems that may arise due to this, but at
least you know what is going on now :).

Kevin

--------------------

| Subject: int function doesn't work correctly.
| Date: Wed, 25 Apr 2001 14:31:57 +0200
| Lines: 18
| Organization: IG3m
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 5.50.4522.1200
| X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200

| Newsgroups: microsoft.public.vb.bugs
| NNTP-Posting-Host: 195.25.74.236
| Path: cppssbbsa01.microsoft.com!tkmsftngp01!tkmsftngp05
| Xref: cppssbbsa01.microsoft.com microsoft.public.vb.bugs:44540
| X-Tomcat-NG: microsoft.public.vb.bugs
|
| Hi there
|
| I found an error in a function using int(). See below.
|
| ?(3.28-3)*100
|  28
| It's correct!
|
| ?int(28)
|  28
| Correct !
|
| ?int((3.28-3)*100)
|  27
| Funny isn't it?
|
|
|
|

However, try the following example :

Dim d As Double
Dim s As Single
d = 64# / 12.8
s = 64# / 12.8

Debug.Print Int(d)                    -> 5
Debug.Print Int(s)                    -> 5
Debug.Print Int(64# / 12.8)     ->  4???

Andy

You are correct, you would expect it to return a 5.  However, the fact that
it is really not an error with the Int function can be found in KB article
Q242933.  It goes on to explain basically the same thing I showed.  It also
shows a workaround: to use the CDec function around anything that might
produce a fraction.  So the code you have would be:

Debug.Print Int(64# / CDec(12.8)           -> 5

Another workaround is to always assign a calculation performed in an Int()
function to a variable and pass the variable into the Int function.

Kevin

--------------------

| Newsgroups: microsoft.public.vb.bugs
| Subject: Re: int function doesn't work correctly.
| Date: Wed, 2 May 2001 15:18:44 -0400
| Organization: Posted via Supernews, http://www.supernews.com



| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 5.00.2615.200
| X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200

| Lines: 97
| Path:
cppssbbsa01.microsoft.com!tkmsftngp01!newsfeed00.sul.t-online.de!t-online.de
!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!sn-xit-03!sn-post-01!supern
ews.com!corp.supernews.com!not-for-mail
| Xref: cppssbbsa01.microsoft.com microsoft.public.vb.bugs:44653
| X-Tomcat-NG: microsoft.public.vb.bugs
|
| However, try the following example :
|
| Dim d As Double
| Dim s As Single
| d = 64# / 12.8
| s = 64# / 12.8
|
| Debug.Print Int(d)                    -> 5
| Debug.Print Int(s)                    -> 5
| Debug.Print Int(64# / 12.8)     ->  4???
|
| Andy
|
|


| > Not exactly an error with the int() function.  More a  problem with the
| way
| > computers store numbers internally.  Binary representation cannot
| > accurately store some decimal numbers (like .28).  Here is a general
| > algorithm for how to convert decimal base-10 numbers to base-2.
| >
| > OLD base 10 number: .28000       NEW base 2 number:
| >
| > Multiply the OLD number by the NEW BASE.
| >      - the digits to the left of the decimal point become a digit in the
| > new number.
| >      - the digits to the right of the decimal point become become the
OLD
| > on the next line.
| >      - continue until OLD is .0000.
| >
| >   OLD                                     NEW
| > 2800 * 2 = 0.5600                .0
| > 5600 * 2 = 1.1200                .01
| > 1200 * 2 = 0.2400                .010
| > 2400 * 2 = 0.4800                .0100
| > 4800 * 2 = 0.9600                .01000
| > 9600 * 2 = 1.9200                .010001
| > 9200 * 2 = 1.8400                .0100011
| > 8400 * 2 = 1.6800                .01000111         (I believe this is
the
| > point where the precision is truncated)
| > 6800 * 2 = 1.3600                .010001111
| > 3600 * 2 = 0.7200                .0100011110
| > 7200 * 2 = 1.4400                .01000111101
| > 4400 * 2 = 0.8800                .010001111010
| > 8800 * 2 = 1.7600                .0100011110101
| >
| > If you carried this out farther I believe you would find a point where
it
| > would start repeating and never get to 0.  Thus, the internal
| > representation for your sample is almost .28, but not quite exact.  Thus
| > when you multiply by 100 and take the int() you get 27.9999999
something.
| >
| > Not that this helps correct any problems that may arise due to this, but
| at
| > least you know what is going on now :).
| >
| > Kevin
| >
| > --------------------

| > | Subject: int function doesn't work correctly.
| > | Date: Wed, 25 Apr 2001 14:31:57 +0200
| > | Lines: 18
| > | Organization: IG3m
| > | X-Priority: 3
| > | X-MSMail-Priority: Normal
| > | X-Newsreader: Microsoft Outlook Express 5.50.4522.1200
| > | X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200

| > | Newsgroups: microsoft.public.vb.bugs
| > | NNTP-Posting-Host: 195.25.74.236
| > | Path: cppssbbsa01.microsoft.com!tkmsftngp01!tkmsftngp05
| > | Xref: cppssbbsa01.microsoft.com microsoft.public.vb.bugs:44540
| > | X-Tomcat-NG: microsoft.public.vb.bugs
| > |
| > | Hi there
| > |
| > | I found an error in a function using int(). See below.
| > |
| > | ?(3.28-3)*100
| > |  28
| > | It's correct!
| > |
| > | ?int(28)
| > |  28
| > | Correct !
| > |
| > | ?int((3.28-3)*100)
| > |  27
| > | Funny isn't it?
| > |
| > |
| > |
| > |
| >
|
|
|

Your explanation is certainly true, but why VB doesn't display 27.9999999...
when it calculate (3.27-3)*100?

On Thu, 3 May 2001 14:33:27 +0200, "CHEVRIER JOHANN"

Because then it would be wrong by almost 1.  It does'nt show
26.99999999... because when it converted from internal binary to
decimal, it had to round a small bit of the error away, and that made
it 27.

VB functions, however always work with the internal binary, so they
are not affected by binary-> decimal conversion rounding.

I cannot help thinking there is a point being missed here. VB doesn't
have to be consistant about how it rounds floating point errors -
rounding errors are a fact of life that EVERY good programmer takes
into account.

If your program depends on (3.27-3)*100 = 27, or (3.27-3)*100 < 27,
then congratulations - you have written a very unreliable program.