Millisecond Forums

Floting point problem

https://forums.millisecond.com/Topic13567.aspx

By mrg4 - 6/15/2014

Hello,

we have a task where the participant earns points according to which key he presses.
In one  case the participant receives 1 point, in the other case he gets 0.1 points. After each trial a feedback is given, saying:

"You received 0.1 / 1 points. Your total score is: XY.XY"

We therefore defined one value element for the gain in the recent trial and one counting the total score.
In case the participant receives 1 point, everything works perfecty fine, but due to the fact that the computer works with floating point, after the first time 0.1 points are added to the score, our feedback looks like e.g.:

"You received 0.10000000000000001 points. Your total score is: 5.10000000000000001"

We already tried to solve the problem by using rounding, but none of our attempts worked. We now use 0.5 / 5 points instead of 0.1 / 1, and it works perfectly.

This is thought to be a temporary solution, so we're hoping anyone is able to help us.


Thank you very much in advance.

Best regards




By Dave - 6/15/2014

This is a completely natural consequence of how computers represent real numbers (cf. http://www.millisecond.com/forums/FindPost7984.aspx ). Your best bet is to use the format() function to limit the output of digits after the dot.

-> format("%.2f", values.myvalue)