TERRIBLE BUG IN INT FUNCION??????????????????? 
Author Message
 TERRIBLE BUG IN INT FUNCION???????????????????

try this with Visual Basic 6 SP5

Private Sub Test()
    Dim s1 As String
    Dim d As Double
    Dim l1 As Long

    s1 = "309,65" ' --- , is the decimal separator
    d = CDbl(s1) ' --- now d = 309.65
    d = d * 100 ' --- now d = 30965.00
    l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    Debug.Print (l1)
End Sub

is this a VB bug????



Mon, 21 Jun 2004 21:51:43 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:
>try this with Visual Basic 6 SP5

>Private Sub Test()
>    Dim s1 As String
>    Dim d As Double
>    Dim l1 As Long

>    s1 = "309,65" ' --- , is the decimal separator
>    d = CDbl(s1) ' --- now d = 309.65
>    d = d * 100 ' --- now d = 30965.00
>    l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>    Debug.Print (l1)
>End Sub

>is this a VB bug????

I think it's a bug.  
Cast d to a long first and she'll be fine.
l1 = Int(clng(d))

Regards, Frank



Mon, 21 Jun 2004 22:39:25 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
Na, my first thought would be that it is the usual rounding
errors associated with computers

Someone will correct me if I'm wrong but I think 30965.00
stored in a double would be stored as something like .30965

J


3 Jan 2002 05:51:43 -0800 bearing the following fruit:

Quote:
>try this with Visual Basic 6 SP5

>Private Sub Test()
>    Dim s1 As String
>    Dim d As Double
>    Dim l1 As Long

>    s1 = "309,65" ' --- , is the decimal separator
>    d = CDbl(s1) ' --- now d = 309.65
>    d = d * 100 ' --- now d = 30965.00
>    l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>    Debug.Print (l1)
>End Sub

>is this a VB bug????



Mon, 21 Jun 2004 23:28:28 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
Hi,

This is not a bug.

The internal floating point representation of a Double is always a very
close approximation to the actual decimal value, but by its very nature it
usually differs by a very small amount.  (Similarly, 0.333333333333333 is
a very close decimal approximation to 1/3, but it still differs by a small
amount.)  It would appear that the floating point representation of
30965.00 on your computer is in fact a wee bit less that the exact decimal
value, i.e. it is equivalent to to something like 30964.99999999999.  This
is why Int() gives you 30964.  On my computer, however, I get 30965 for
the same code.  This is presumably because of a slight difference in the
way the processor handles floating point numbers.

Given the very nature of floating point numbers you have to take
precautions to ensure that Int() will give you the result you want.  In
the present case, you could simply avoid the problem by using the Currency
type instead of a Double:

    Dim s1 As String
    Dim c As Currency
    Dim l1 As Long

    s1 = "309,65" ' --- , is the decimal separator
    c = CCur(s1) ' --- now c = 309.65
    c = c * 100 ' --- now c = 30965.00
    l1 = Int(c) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    Debug.Print (l1)

The Currency type gives an exact representation of decimal numbers, but is
limited to four decimal places.

John..........

Quote:

> try this with Visual Basic 6 SP5

> Private Sub Test()
>     Dim s1 As String
>     Dim d As Double
>     Dim l1 As Long

>     s1 = "309,65" ' --- , is the decimal separator
>     d = CDbl(s1) ' --- now d = 309.65
>     d = d * 100 ' --- now d = 30965.00
>     l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>     Debug.Print (l1)
> End Sub

> is this a VB bug????



Mon, 21 Jun 2004 23:33:08 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:
> try this with Visual Basic 6 SP5

> Private Sub Test()
>     Dim s1 As String
>     Dim d As Double
>     Dim l1 As Long

>     s1 = "309,65" ' --- , is the decimal separator
>     d = CDbl(s1) ' --- now d = 309.65
>     d = d * 100 ' --- now d = 30965.00
>     l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>     Debug.Print (l1)
> End Sub

> is this a VB bug????

At least it is strange.
But it is also strange what you do, placing the integer portion of a double
in a long.
You should use the clng function to place a value in a long variable.

Greets,
Rik



Mon, 21 Jun 2004 23:29:59 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
The problem is that floating point numbers are not exactly what they seem. In the same way
that 1/3 has no exact representation in the decimal number system (the best we can do is
write 0.33333 putting down 3's until we get tired), most fractional numbers have no exact
representation in the binary system (the underlying storage mechanism for computers).
Rather, they are close approximations. When VB shows you a floating point number, it only
shows you 15 of the possible 16 digits it has for that number. The hidden number is known
(at least in calculator terms) as a guard digit and is meant to absorb some of the
inaccuracies associated when calculating with floating point numbers. When VB displays a
floating point number, it "rounds" the internal 16 digits to 15 digits and displays them,
rounding as necessary.

As it turns out, the 0.65 (the dot is the decimal point for me) of 309.65 does not have an
exact binary representation -- the closest binary number is slightly smaller than that
value within VB, so all of your calculations are being done on a number that is slightly
smaller than you think they are when you look at them (remember, VB was nice to you and
rounded it for display purposes). When you got to the Int(d) statement, d wasn't really
30965, but something ever so slightly less.

You can see where all of this is coming from by executing the following in your Immediate
window

     Print 309.65 - 309

You won't get 0.65, but rather 0.649999999999977 (the internal guard digit rounding no
longer produces a string of trailing zeroes with the displayable 15 digits) which means
that within VB, 309.65 is really 308.649999999999977 and if you run your calculations
manually on that value, you'll see why you are getting the result you get.

By the way, VB does seem to treat this calculation a little different than I would expect
(probably an attempt to maximize accuracy for us). Since 309.65 can only be composed of 16
full digits, I would have expected the internal number to be 308.6499999999999 and when
subtracting 309 from it, only 0.6499999999999 would result. But it seems VB has handled
this problem, internally, this way

     309 + 0.649999999999977 - 309

keeping the integer and decimal portions separated during the calculation; hence the extra
0.000000000000077 accuracy.

Rick


Quote:
> try this with Visual Basic 6 SP5

> Private Sub Test()
>     Dim s1 As String
>     Dim d As Double
>     Dim l1 As Long

>     s1 = "309,65" ' --- , is the decimal separator
>     d = CDbl(s1) ' --- now d = 309.65
>     d = d * 100 ' --- now d = 30965.00
>     l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>     Debug.Print (l1)
> End Sub

> is this a VB bug????



Tue, 22 Jun 2004 00:36:41 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:

> amount.)  It would appear that the floating point representation of
> 30965.00 on your computer is in fact a wee bit less that the exact decimal
> value, i.e. it is equivalent to to something like 30964.99999999999.  This
> is why Int() gives you 30964.  On my computer, however, I get 30965 for
> the same code.  This is presumably because of a slight difference in the
> way the processor handles floating point numbers.

This is correct.

One way of avoiding this is using CInt() and CLng() instead of Int()
and Fix(). Int() and Fix() truncate doubles, which will occasionally
give you errors like we just saw. CInt() and CLng(), however, round
the values to the nearest even number.

Replacing the Int() with CLng() in the posted code solves the rounding
error problem.



Tue, 22 Jun 2004 02:47:20 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
On Thu, 03 Jan 2002 11:33:08 -0400, John Keyston

Quote:

>value, i.e. it is equivalent to to something like 30964.99999999999.  This
>is why Int() gives you 30964.  On my computer, however, I get 30965 for
>the same code.  This is presumably because of a slight difference in the

Can i ask what sort of CPU you have  ? Cyrix and Celeron here, both
produced the same result. :-/
Does this mean that yours is better, or it just has a rounding problem
elsewhere ? :)

Regards, Frank



Tue, 22 Jun 2004 04:03:40 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:
> try this with Visual Basic 6 SP5

Interestingly, VB5 Pro returns 3096500 instead of your 30964...


Tue, 22 Jun 2004 05:08:51 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
Hi,

Sorry, I goofed.  I actually get 30964 (Duron 800).  I intended to change my
message before sending it, but then I got distracted, the phone rang etc. etc.
and I clicked Send forgetting to make the change.

I must admit that I don't know whether one can in fact get different results
with different processors today.  I seem to recall that this happened to me in
the past, and I was ready to believe it to be possible.  Although a certain
level of accuracy is always guaranteed for a Double, I don't believe that there
is an implicit guarantee that the error will always the same sign and
magnitutude for the same calculation.  Someone with detailed inside knowledge
of floating point operations may be able to help here.

John............

Quote:

> On Thu, 03 Jan 2002 11:33:08 -0400, John Keyston

> >value, i.e. it is equivalent to to something like 30964.99999999999.  This
> >is why Int() gives you 30964.  On my computer, however, I get 30965 for
> >the same code.  This is presumably because of a slight difference in the

> Can i ask what sort of CPU you have  ? Cyrix and Celeron here, both
> produced the same result. :-/
> Does this mean that yours is better, or it just has a rounding problem
> elsewhere ? :)

> Regards, Frank



Tue, 22 Jun 2004 05:07:55 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:

> try this with Visual Basic 6 SP5

> Private Sub Test()
>     Dim s1 As String
>     Dim d As Double
>     Dim l1 As Long

>     s1 = "309,65" ' --- , is the decimal separator
>     d = CDbl(s1) ' --- now d = 309.65
>     d = d * 100 ' --- now d = 30965.00
>     l1 = Int(d) ' --- l1 = 30964 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>     Debug.Print (l1)
> End Sub

> is this a VB bug????

This is business as usual for floating-point, which uses binary fractions.
Unfortunately, quite a few "simple-looking" decimal fractions, like 0.1,
just cannot be represented exactly in a finite number of bits in a binary
fraction, just as 1/3 cannot be represented exactly in a finite number of
digits in a decimal fraction:  0.33333333...  You need to decide what's
"close enough" for your purposes:

global const epsilon = 1e-10 ' "close enough"

l1 = int(epsilon + d)  ' round to next lowest integer value
l2 = -int(epsilon - d) ' round to next highest integer value

There's also a magic setting you might be able to twiddle in the floating
point processor to squeeze out a little more precision.  In Visual C++,
you can use controlfp(_MCW_PC, _PC_53) and controlfp(_MCW_PC, _PC_64) to
switch between "unleaded" and "nitrous".  There may be a VB equivalent,
but there may also exist pitfalls unique to VB.  Your mileage (and engine
repair bills!) may vary.

--
Joe Foster <mailto:jlfoster%40znet.com>  DC8s in Spaace: <http://www.xenu.net/>
WARNING: I cannot be held responsible for the above        They're   coming  to
because  my cats have  apparently  learned to type.        take me away, ha ha!



Tue, 22 Jun 2004 00:37:27 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:


> > try this with Visual Basic 6 SP5

> Interestingly, VB5 Pro returns 3096500 instead of your 30964...

Sounds like a locale issue to me!

--
Joe Foster <mailto:jlfoster%40znet.com>   Space Cooties! <http://www.xenu.net/>
WARNING: I cannot be held responsible for the above        They're   coming  to
because  my cats have  apparently  learned to type.        take me away, ha ha!



Tue, 22 Jun 2004 08:47:25 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????

Quote:

> [snippage] CInt() and CLng(), however, round
> the values to the nearest even number.

?clng(1.4999);cint(2.5001)

--
Joe Foster <mailto:jlfoster%40znet.com>     Got Thetans? <http://www.xenu.net/>
WARNING: I cannot be held responsible for the above        They're   coming  to
because  my cats have  apparently  learned to type.        take me away, ha ha!



Tue, 22 Jun 2004 08:51:11 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????
On Thu, 3 Jan 2002 16:47:25 -0800, "Joe \"Nuke Me Xemu\" Foster"

Quote:



>> > try this with Visual Basic 6 SP5

>> Interestingly, VB5 Pro returns 3096500 instead of your 30964...

>Sounds like a locale issue to me!

What's his locale compared to mine  ? I'm getting 964 on both
machines, even in BC4.02, where there's a pretty well respected maths
library working on this stuff.

Maybe he has an older Pentium that makes up for the loss ? ;-)

Regards, Frank



Tue, 22 Jun 2004 09:21:58 GMT  
 TERRIBLE BUG IN INT FUNCION???????????????????



Quote:

> > [snippage] CInt() and CLng(), however, round
> > the values to the nearest even number.

> ?clng(1.4999);cint(2.5001)

He most probably meant whole number  :)

jb



Tue, 22 Jun 2004 16:39:11 GMT  
 
 [ 24 post ]  Go to page: [1] [2]

 Relevant Pages 

1. ???Bug in Int()-function of ACC95 and ACC97 ???

2. Bug when ising INT function

3. BUG: VB4(16 bit) - Int and Fix functions don't work correctly

4. Bug in Int() and Numbers

5. BUG?? Weird Int rounding

6. Converting BitArray to Int and Int to BitArray

7. How to display int, short int range values

8. Int to Str or Str to Int ??

9. Converting ULONG (4 byte binary) int VB Long / SQL INT for DTS Import

10. Convert Int to Str & Str to Int

11. int main( int argc, char *argv[]) in VB??

12. Does anyone know how to use new Bitmap(int,int,int,PixelFormat, IntPtr) ?

 

 
Powered by phpBB® Forum Software