Floating point problem... floating indeed :(

ruby-talk
1

I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

···

#####################
#!/usr/bin/ruby1.9

a = 1.0

a.to_f

while(a < 15.0)
  a = a + 0.1
  print a,"\t",a * a,"\n"
end
#####################

and the output is...(as following) quite odd. Anyone can explain it?
or a way to avoid it (while
keeping the precision)

1.1 1.21
1.2 1.44
1.3 1.69
1.4 1.96
1.5 2.25
1.6 2.56
1.7 2.89
1.8 3.24
1.9 3.61
2.0 4.0
2.1 4.41
2.2 4.84
2.3 5.29000000000001
2.4 5.76000000000001
2.5 6.25000000000001
2.6 6.76000000000001
2.7 7.29000000000001
2.8 7.84000000000001
2.9 8.41000000000001
3.0 9.00000000000001
3.1 9.61000000000001
3.2 10.24
3.3 10.89
3.4 11.56
3.5 12.25
3.6 12.96
3.7 13.69
3.8 14.44
3.9 15.21
4.0 16.0
4.1 16.81
4.2 17.64
4.3 18.49
4.4 19.36
4.5 20.25
4.6 21.16
4.7 22.09
4.8 23.04
4.9 24.01
5.0 25.0
5.1 26.01
5.2 27.04
5.3 28.09
5.4 29.16
5.5 30.25
5.6 31.36
5.7 32.49
5.8 33.64
5.9 34.81
6.0 35.9999999999999
6.1 37.2099999999999
6.19999999999999 38.4399999999999
6.29999999999999 39.6899999999999
6.39999999999999 40.9599999999999
6.49999999999999 42.2499999999999
6.59999999999999 43.5599999999999

2

See this writeup for some details on why this happens.

···

On 11/30/06, teeshift <teeshift@gmail.com> wrote:

I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

3

Someone else might wish to give a more concrete explanation, but as a
rule, floating-point calculations - using the usual Float datatype -
are not accurate, and prone to rounding errors such as the above.

If you're dealing with limited precision (i.e. X digits after the
decimal point) you should consider using the Decimal datatype instead.

···

On 12/1/06, teeshift <teeshift@gmail.com> wrote:

I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

#####################
#!/usr/bin/ruby1.9

a = 1.0

a.to_f

while(a < 15.0)
  a = a + 0.1
  print a,"\t",a * a,"\n"
end
#####################

and the output is...(as following) quite odd. Anyone can explain it?
or a way to avoid it (while
keeping the precision)

1.1 1.21
1.2 1.44
1.3 1.69
1.4 1.96
1.5 2.25
1.6 2.56
1.7 2.89
1.8 3.24
1.9 3.61
2.0 4.0
2.1 4.41
2.2 4.84
2.3 5.29000000000001
2.4 5.76000000000001
2.5 6.25000000000001
2.6 6.76000000000001
2.7 7.29000000000001
2.8 7.84000000000001
2.9 8.41000000000001
3.0 9.00000000000001
3.1 9.61000000000001
3.2 10.24
3.3 10.89
3.4 11.56
3.5 12.25
3.6 12.96
3.7 13.69
3.8 14.44
3.9 15.21
4.0 16.0
4.1 16.81
4.2 17.64
4.3 18.49
4.4 19.36
4.5 20.25
4.6 21.16
4.7 22.09
4.8 23.04
4.9 24.01
5.0 25.0
5.1 26.01
5.2 27.04
5.3 28.09
5.4 29.16
5.5 30.25
5.6 31.36
5.7 32.49
5.8 33.64
5.9 34.81
6.0 35.9999999999999
6.1 37.2099999999999
6.19999999999999 38.4399999999999
6.29999999999999 39.6899999999999
6.39999999999999 40.9599999999999
6.49999999999999 42.2499999999999
6.59999999999999 43.5599999999999

4

teeshift wrote:

/ ...

and the output is...(as following) quite odd. Anyone can explain it?

Floating-point numbers are almost always represented internally in binary
(base 2). For display, they are converted to base ten, but numbers with
fractional parts cannot be displayed perfectly after conversion to base
ten.

For example, the base-ten number 0.1 has no finite representation in base 2
-- it is a continuing, nonterminating fraction, sort of like the decimal
equivalent of 1/3 in base ten.

0.1 in base ten = 0.00011001100110011001101... (forever) in base 2

This is a rite of passage. Everyone goes through this while learning about
computers.

···

--
Paul Lutus
http://www.arachnoid.com