I'm just starting out with the Ruby language, and appear to either be
doinf something wrong, or have run into a rather basic math bug. When
running "irb", simple math additions seem to sometimes have weird
results. for instance.
1.9.3-p125 :022 > 120.02+82
=> 202.01999999999998
but then
1.9.3-p125 :023 > 120.03+82
=> 202.03
This anomoly seems to be repeatable on a friends box as well. At first,
I was onyl able to trigger this with a 3 digit number starting with a 1,
with .08 after, plus a 2 digit number. However, checked a few other
combinations have also had this kind of issue.
Now since I'm quite new to this, I realize that this could of course be
something I'm doing wrong, but obviously, since I'm tyring to learn, I'd
like to know why I'm getting these results. Thanks in advance guys.
I'm not saying you should read all this, but this issue is a well
known fact about floating point math in programing languages, and it's
present in lots of languages.
On Fri, Mar 30, 2012 at 10:04, Steven Owens <lists@ruby-forum.com> wrote:
Hello guys,
I'm just starting out with the Ruby language, and appear to either be
doinf something wrong, or have run into a rather basic math bug. When
running "irb", simple math additions seem to sometimes have weird
results. for instance.
1.9.3-p125 :022 > 120.02+82
=> 202.01999999999998
but then
1.9.3-p125 :023 > 120.03+82
=> 202.03
This anomoly seems to be repeatable on a friends box as well. At first,
I was onyl able to trigger this with a 3 digit number starting with a 1,
with .08 after, plus a 2 digit number. However, checked a few other
combinations have also had this kind of issue.
Now since I'm quite new to this, I realize that this could of course be
something I'm doing wrong, but obviously, since I'm tyring to learn, I'd
like to know why I'm getting these results. Thanks in advance guys.
I'm not saying you should read all this, but this issue is a well
known fact about floating point math in programing languages, and it's
present in lots of languages.
If you need precision, you can look into Ruby's BigDecimal standard library. But often times float precision works OK for things that aren't money or similarly important values.
-greg
···
On 3/30/12 9:16 AM, Shawn Carron wrote:
Gustavo Campanelli wrote in post #1054186:
Thanks for the link. We are both new to the language and were unsure of
what we were seeing.