Hi sir Guy (aka ts [mailto:decoux@moulon.inra.fr]):
You expounded last Tuesday, December 10, 2002 8:12 PM:
Float#to_i is the same than Float#truncate
pigeon% ruby -e ‘p ((2.4/0.2).truncate)’
11
pigeon%
pigeon% ruby -e ‘p ((2.4/0.2).floor)’
11
pigeon%
pigeon% ruby -e ‘p ((2.4/0.2).ceil)’
12
pigeon%
pigeon% ruby -e ‘p ((2.4/0.2).round)’
12
pigeon%
but
irb(main):002:0> p “test #{2.4/0.2}”
“test 12”
nil
irb(main):003:0> p 2.4/0.2
12
nil
irb(main):004:0> 2.4/0.2
12
C:\family\ruby>type a1.rb
def t
2.4/0.2
end
x = t
p x
p x.truncate
C:\family\ruby>ruby a1.rb
12
11
Isn’t it that #to_i and #truncate exhibit odd and therefore dangerous
behaviour?
As a nuby, how do I avoid such “errors”?
Guy Decoux
kind regards,
-botp
“Peña, Botp” botp@delmonte-phil.com writes:
Isn’t it that #to_i and #truncate exhibit odd and therefore dangerous
behaviour?
As a nuby, how do I avoid such “errors”?
There really is no easy way to escape such errors. The difficulties
lies in how the computer represents the number 0.2. As other posts in
this threads shows, 0.2 is one of the numbers that can’t be
represented correctly.
The best representation computer can come up with for 0.2 is:
0.200000000000000011102230246251565404236316680908203125
Thus, when dividing 12.4 with that representation, you actually do not
get a whole number.
The problems of numerical computation using computer are widely
known. In fact, to graduate from Applied Math major in my university,
a student has to take two 400-level numerical computation classes
just so they know how to do numerical computation correctly.
Using spesialised mathematic library like the GNU MP library may help
alleviate this problem, but no solution can solve this completely.
YS.
Yohanes Santoso wrote:
…
Using spesialised mathematic library like the GNU MP library may help
alleviate this problem, but no solution can solve this completely.
There is also BigFloat (haven’t used it myself):
http://www.ruby-lang.org/raa/list.rhtml?name=bigfloat