Skip to content

Commit

Permalink
Strange results in floating point
Browse files Browse the repository at this point in the history
6.0 / 3.0 != 2.0 is an incorrect example because floats can represent all three numbers perfectly, so the result is equal. 9.0 / 0.3 != 30.0 is a better example, it produces the correct result in double precision but not in single precision.
  • Loading branch information
oscaaaaaaar authored Jan 10, 2022
1 parent 4ca291e commit d74e42b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion Language/Variables/Data Types/float.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ If doing math with floats, you need to add a decimal point, otherwise it will be

The float data type has only 6-7 decimal digits of precision. That means the total number of digits, not the number to the right of the decimal point. Unlike other platforms, where you can get more precision by using a double (e.g. up to 15 digits), on the Arduino, double is the same size as float.

Floating point numbers are not exact, and may yield strange results when compared. For example 6.0 / 3.0 may not equal 2.0. You should instead check that the absolute value of the difference between the numbers is less than some small number.
Floating point numbers are not exact, and may yield strange results when compared. For example 9.0 / 0.3 may not quite equal 30.0. You should instead check that the absolute value of the difference between the numbers is less than some small number.

Conversion from floating point to integer math results in truncation:
[source,arduino]
Expand Down

0 comments on commit d74e42b

Please sign in to comment.