[samigo-team] A note on floating point equality
David Horwitz
david.horwitz at uct.ac.za
Wed Apr 25 01:53:58 PDT 2012
Hi Gang,
Just a note on a common mistake made with floating point precision I saw
while fixing SAM-1562. Basically it is a misunderstanding of the
difference between rounding and limiting the precision of the float.
In this case a float was cast to a double:
Double.valueOf(Float)
rather than rounded and cast to a Double:
Double.valueOf(MathUtils.round(Float, 2))
in the former case a float that was something in the region of
99.99999999999*% was being sent to the gradebook as 99.99% rather than 100%
regards
David
More information about the samigo-team
mailing list