Hi,
I am new to ruby and found this while experimenting with ruby.
irb(main):010:0> 5.1%0.5
=> 0.0999999999999996
irb(main):011:0>
Is this similar to the float related operations that MS Excel has?
- Karthick S.
Karthick S. wrote:
I am new to ruby and found this while experimenting with ruby.
irb(main):010:0> 5.1%0.5
=> 0.0999999999999996
Ruby is no different to any other language which uses binary floating
point arithmetic:
$ perl -e 'print 5.1 - 10*0.5, "\n"'
0.0999999999999996
The problem is that 1/10 does not have an exact representation in binary
floating point, in the same way that 1/3 does not have an exact
representation in decimal floating point (0.33333...)
To learn more see http://docs.sun.com/source/806-3568/ncg_goldberg.html
For some applications, you might use the Rational or BigDecimal classes
instead. If dealing with currency, you can work in Integers for
cents/pence etc.
Or maybe you just want to use fewer significant digits when printing:
v = 5.1 % 0.5
=> 0.0999999999999996
"%.8f" % v
=> "0.10000000"
although you're still likely to be tripped up by this:
v == 0.1
=> false
HTH,
Brian.
···
--
Posted via http://www.ruby-forum.com/\.
It isn't relevant to the OPs example, but not all languages produce
the results for modulo when presented with negative arguments:
Ruby, Perl, and MS Excel all return a result with the sign of the
divisor, while Java returns a result with the sign of the dividend.
···
On Mon, Jul 12, 2010 at 6:15 AM, Brian Candler <b.candler@pobox.com> wrote:
Karthick S. wrote:
I am new to ruby and found this while experimenting with ruby.
irb(main):010:0> 5.1%0.5
=> 0.0999999999999996Ruby is no different to any other language which uses binary floating
point arithmetic
--
Rick DeNatale
Blog: http://talklikeaduck.denhaven2.com/
Github: rubyredrick (Rick DeNatale) · GitHub
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale