an if expression comparing a computed floating point value with a floating point constant fails when it shouldn't 
Author Message
 an if expression comparing a computed floating point value with a floating point constant fails when it shouldn't

Tcl 8.3.2 Bug:  Generated by Ajuba's bug entry form at
        http://www.*-*-*.com/
Responses to this post are encouraged.
------

Submitted by:  Jon Hilbert
OperatingSystem:  All Windows
OperatingSystemVersion:  Windows NT 4.0
Machine:  Dell Dimension P200
Synopsis:  an if expression comparing a computed floating point value with a floating point constant fails when it shouldn't

ReproducibleScript:
for {set i 0} {$i < 10} {incr i} {set a($i) "0.10"}
set total 0
for {set i 0} {$i < 10} {incr i} {set total [expr {$total + $a($i)}]}
if {$total != 1.0} {puts "what? why doesn't $total == 1.0 ?"}
if {[expr $total] == 1.0} {puts "but \[expr $total] == 1.0 ?"}
info patchlevel

ObservedBehavior:
as the script shows, the results of "if {$total != 1.0}" are not the same as "if {[expr $total] != 1.0}". I think they should be.



Mon, 10 Mar 2003 03:00:00 GMT  
 an if expression comparing a computed floating point value with a floating point constant fails when it shouldn't

Quote:

> Tcl 8.3.2 Bug:  Generated by Ajuba's bug entry form at
>    http://www.ajubasolutions.com/support/bugForm.html
> Responses to this post are encouraged.
> ------

> Submitted by:  Jon Hilbert
> OperatingSystem:  All Windows
> OperatingSystemVersion:  Windows NT 4.0
> Machine:  Dell Dimension P200
> Synopsis:  an if expression comparing a computed floating point value

with a floating point constant fails when it shouldn't
Quote:

> ReproducibleScript:
> for {set i 0} {$i < 10} {incr i} {set a($i) "0.10"}
> set total 0
> for {set i 0} {$i < 10} {incr i} {set total [expr {$total + $a($i)}]}
> if {$total != 1.0} {puts "what? why doesn't $total == 1.0 ?"}
> if {[expr $total] == 1.0} {puts "but \[expr $total] == 1.0 ?"}
> info patchlevel

> ObservedBehavior:
> as the script shows, the results of "if {$total != 1.0}" are not the

same as "if {[expr $total] != 1.0}". I think they should be.

Equality of floats is always tricky, in any language!!

% for {set i 0} {$i < 10} {incr i} {set a($i) "0.10"}
% set total 0
0
% for {set i 0} {$i < 10} {incr i} {set total [expr {$total + $a($i)}]}
% if {$total != 1.0} {puts "what? why doesn't $total == 1.0 ?"}
what? why doesn't 1.0 == 1.0 ?
% if ![string compare $total 1.0] {puts "string expressions are OK!"}
string expressions are OK!
% expr {$total - 1.0}
-1.11022302463e-16
%

Sent via Deja.com http://www.deja.com/
Before you buy.



Mon, 10 Mar 2003 03:00:00 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. An alternative to floating point (was Re: Floating point non-exactness)

2. IBM 370 Floating point to IEEE floating point

3. Comparing numbers, floating point error ?

4. fixed point vs floating point

5. code with fixed-point or floating-point

6. fixed point vs floating point

7. fixed point vs floating point

8. Floating Point vs Fixed Point

9. comparing floating point numbers

10. [Q] comparing floating point numbers

11. floating point compares

12. Comparing floating point numbers...kudos

 

 
Powered by phpBB® Forum Software