Rounding errors?

ruby-talk
1

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

:~$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Cheers,
Pete

···

--
Posted via http://www.ruby-forum.com/.

2

Pete Hodgson <ruby-forum@thepete.net> writes:

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

You may try a true Lisp instead of a Matzacred Lisp:

$ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
29

···

--
__Pascal Bourguignon__

3

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

>> sprintf "%0.30f", 0.29
=> "0.289999999999999980015985556747"
>> sprintf "%0.30f", 0.29*100
=> "28.999999999999996447286321199499"
>> sprintf "%0.30f", (0.29*100).to_i
=> "28.000000000000000000000000000000"

If you want to really understand what is going on:
<http://docs.sun.com/source/806-3568/ncg_goldberg.html&gt;

Gary Wright

···

On Jun 5, 2009, at 10:07 PM, Pete Hodgson wrote:

4

Pascal J. Bourguignon wrote:

Pete Hodgson <ruby-forum@thepete.net> writes:

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

You may try a true Lisp instead of a Matzacred Lisp:

$ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
29

There's nothing strange here, just IEEE floats, which don't represent every decimal float:

irb(main):001:0> (0.29)*100
=> 29.0
irb(main):002:0> ((0.29)*100).to_i
=> 28
irb(main):003:0> ((0.29)*100).round
=> 29
irb(main):004:0> ((0.29)*100).floor
=> 28
irb(main):006:0> "%20.18f" % ((0.29)*100)
=> "28.999999999999996447"
irb(main):007:0> "%20.18f" % 0.29
=> "0.289999999999999980"

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

5

The result of the calculation is undefined in pretty much any language. The loose definition of the expression is "return a value 100 times a value approximating 29 and truncated to an integer using the algorithm of your choice." So, you claim the *only* right answer must be the one yielded by Lisp. However, the correct answer is: The result of evaluating this expression is implementation-specific.

In C:

#include <stdio.h>

int main(void) {
   double d = 0.29;
   float f = 0.29;

   printf("double truncated to integer is: %i\n", (int)(d * 100));
   printf("float truncated to integer is: %i\n", (int)(f * 100));

   return 0;
}

Would ANSI[1] C results be correct enough to be used as a gold standard? Well, if so, the result of the program above is:

$ ./foo
double truncated to integer is: 28
float truncated to integer is: 29

The moral of the story is not "use Lisp". The moral is use floating point numbers with great care. Oh, geez, I totally forgot, this is a Ruby list, not a C list.

[1] gcc version 4.0.1 (Apple Inc. build 5488)

···

On Jun 5, 2009, at 7:15 PM, Pascal J. Bourguignon wrote:

Pete Hodgson <ruby-forum@thepete.net> writes:

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

You may try a true Lisp instead of a Matzacred Lisp:

$ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
29

--
__Pascal Bourguignon__

6

"s.ross" <cwdinfo@gmail.com> writes:

Pete Hodgson <ruby-forum@thepete.net> writes:

:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can
lead
to things like this, but this one seems a little extreme to me.

You may try a true Lisp instead of a Matzacred Lisp:

$ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
29

--
__Pascal Bourguignon__

The result of the calculation is undefined in pretty much any
language. The loose definition of the expression is "return a value
100 times a value approximating 29 and truncated to an integer using
the algorithm of your choice." So, you claim the *only* right answer
must be the one yielded by Lisp. However, the correct answer is: The
result of evaluating this expression is implementation-specific.

My point being that there are some implementations that are more
useful and helpful than others.

···

On Jun 5, 2009, at 7:15 PM, Pascal J. Bourguignon wrote:

The moral of the story is not "use Lisp". The moral is use floating
point numbers with great care. Oh, geez, I totally forgot, this is a
Ruby list, not a C list.

--
__Pascal Bourguignon__

7

The result of the calculation is undefined in pretty much any language. The loose definition of the expression is "return a value 100 times a value approximating 29 and truncated to an integer using the algorithm of your choice." So, you claim the *only* right answer must be the one yielded by Lisp.

Agreed. And, btw, with BigDecimal even Ruby comes up with more "natural" results:

require 'bigdecimal'
x = BigDecimal.new "0.29"
x *= 100
puts x
printf "%f %10.8f\n", x, x

->

0.29E2
29.000000 29.00000000

However, the correct answer is: The result of evaluating this expression is implementation-specific.

Actually, this is only half true: if the implementation is IEEE-754 standards conform then it must yield certain results - regardless of implementation. I believe nowadays most if not all math libraries conform to the standard so you should expect to see the same results from all implementations. I would assume that differences are mostly due to numeric data type (see your C example) and conversion for printing (e.g. how many places are output by default).

http://grouper.ieee.org/groups/754/

The moral of the story is not "use Lisp". The moral is use floating point numbers with great care.

Exactly.

Kind regards

  robert

···

On 06.06.2009 05:46, s.ross wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/