Newbie question

ruby-talk
1

Greetings all, I think this is generally more a programming question than a
ruby explicit one, however, I am wondering why this code yields 23,
0.999999999999979 instead of 23, 1.

Any help would be great!

i = 0
while i <= 50
  a = 37*i / 50
  c = a + 0.0
  b = (37*i/50.0 - c ) *50
  if b < 1.5 && b > 0.5
    puts i
    puts b
  end
  i = i+1
end

thanks much
Will

2

Rounding errors in floating point arithmetic.

Farrel

···

On 08/10/2008, Will <wrsh07@gmail.com> wrote:

Greetings all, I think this is generally more a programming question than a
ruby explicit one, however, I am wondering why this code yields 23,
0.999999999999979 instead of 23, 1.

Any help would be great!

i = 0
while i <= 50
  a = 37*i / 50
  c = a + 0.0
  b = (37*i/50.0 - c ) *50
  if b < 1.5 && b > 0.5
    puts i
    puts b
  end
  i = i+1
end

thanks much
Will

--
Aimred - Ruby Development and Consulting

3

Will wrote:

Greetings all, I think this is generally more a programming question than a
ruby explicit one, however, I am wondering why this code yields 23,
0.999999999999979 instead of 23, 1.

A common question. Check out http://en.wikipedia.org/wiki/Floating_point\.

···

--
RMagick: http://rmagick.rubyforge.org/

4

require 'bigdecimal'

#put a bigdecimal in the numerator or denominator...
((BigDecimal("37") * 23 / 50 - 17) * 50).to_f
#=> 1.0

Todd

···

On Tue, Oct 7, 2008 at 5:27 PM, Will <wrsh07@gmail.com> wrote:

Greetings all, I think this is generally more a programming question than a
ruby explicit one, however, I am wondering why this code yields 23,
0.999999999999979 instead of 23, 1.

Any help would be great!

i = 0
while i <= 50
a = 37*i / 50
c = a + 0.0
b = (37*i/50.0 - c ) *50
if b < 1.5 && b > 0.5
   puts i
   puts b
end
i = i+1
end