Sounds like a simple rounding error to me, your only out by .0000000000001, what's the big deal?
(I'm really tired right now, so my math could be wrong, but I don't think so)
Dale Atkin
Tue, 09 Jan 2001 03:00:00 GMT
Atkin GeoMacK Consultancy Ltd #3 / 6
Single data type BUG ???
Wait a sec, my math was wrong, I thought it said 9.9999999999999e-2 (are you sure it didn't, 99,999999999e-2 isn't proper sci-notation, really wierd if it did)
Tue, 09 Jan 2001 03:00:00 GMT
Bob Marti #4 / 6
Single data type BUG ???
Quote:
> Hi, > I encounted a Single type BUG.. :
> dim a as single > a=0.0001 > debug.print a*1000
> instead of 0.1 the result is 99.99999999e-2
> any ideas ? > Thanks > Ariel.
Not a bug, it's due to how floating point numbers are stored. fractional part is stored as a series of the form 1/2^n. check microsoft site articles on IEEE floating point format.
Tue, 09 Jan 2001 03:00:00 GMT
Jeff Weckstei #5 / 6
Single data type BUG ???
Quote:
> Hi, > I encounted a Single type BUG.. :
> dim a as single > a=0.0001 > debug.print a*1000
> instead of 0.1 the result is 99.99999999e-2
> any ideas ? > Thanks > Ariel.
No bug. A single is 16-bit number, VB can't totally convert everthing from binary to a 10 base number. You encounter this in assembly programming a lot.
Single data type BUG ???