Rounding error, (100.0 * 9.95).to_i == 994

I can understand floating point errors, but this one seems too small for that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994

irb(main):016:0> (100.0*9.95).to_i
=> 994
irb(main):017:0> (100*9.95).to_i
=> 994
irb(main):018:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):019:0> RUBY_VERSION
=> "1.8.1"
irb(main):020:0> RUBY_PLATFORM
=> "sparc-solaris2.8"

irb(main):020:0> (100.0*9.95).to_i
=> 994
irb(main):021:0> (100*9.95).to_i
=> 994
irb(main):022:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):023:0> RUBY_VERSION
=> "1.6.8"
irb(main):024:0> RUBY_PLATFORM
=> "powerpc-darwin7.0"

Dan

Dan Janowski wrote:

I can understand floating point errors, but this one seems too small for that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994

a = 100 * 9.95
"%.16f" % a
=> "994.9999999999998863"

It's not a problem with Ruby, it's a problem with floating point numbers in general. :frowning:

- Jamis

···

--
Jamis Buck
jgb3@email.byu.edu
http://www.jamisbuck.org/jamis

Dan Janowski wrote:

I can understand floating point errors, but this one seems too small for that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994

irb(main):016:0> (100.0*9.95).to_i
=> 994
irb(main):017:0> (100*9.95).to_i
=> 994
irb(main):018:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):019:0> RUBY_VERSION
=> "1.8.1"
irb(main):020:0> RUBY_PLATFORM
=> "sparc-solaris2.8"

irb(main):020:0> (100.0*9.95).to_i
=> 994
irb(main):021:0> (100*9.95).to_i
=> 994
irb(main):022:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):023:0> RUBY_VERSION
=> "1.6.8"
irb(main):024:0> RUBY_PLATFORM
=> "powerpc-darwin7.0"

Dan

>ri Float#to_i
------------------------------------------------------------- Float#to_i
      flt.to_i => integer
      flt.to_int => integer
      flt.truncate => integer

···

------------------------------------------------------------------------
      Returns _flt_ truncated to an +Integer+.

you may want to try

>ri Float#round
------------------------------------------------------------ Float#round
      flt.round => integer
------------------------------------------------------------------------
      Rounds _flt_ to the nearest integer. Equivalent to:

         def round

           return floor(self+0.5) if self > 0.0

           return ceil(self-0.5) if self < 0.0

           return 0.0

         end

         1.5.round #=> 2

         (-1.5).round #=> -2

Hope that helps,

Walt

--
Walter Szewelanczyk
IS Director
M.W. Sewall & CO. email : walter@mwsewall.com
259 Front St. Phone : (207) 442-7994 x 128
Bath, ME 04530 Fax : (207) 443-6284

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

Bill

···

On Thu, 28 Oct 2004 03:27:33 +0900, Walter Szewelanczyk <walter@mwsewall.com> wrote:

you may want to try

>ri Float#round

eep:~> perl
$f = 100.0 * 9.95;
printf("%d\n", $f);
994

if it really bothers you (and you don't want to use round),
there's an ugly solution:

require 'bigdecimal'
v = BigDecimal.new("9.95")
vone = v * 100
vone.to_i
=> 995

(but really, you just want to use round. :wink:

  -dave

···

On Thu, Oct 28, 2004 at 03:25:40AM +0900, Jamis Buck scribed: > Dan Janowski wrote:

>I can understand floating point errors, but this one seems too small for
>that.
>Especially given this:
>
>irb(main):023:0> v=100.0*9.95
>=> 995
>irb(main):024:0> v.to_i
>=> 994

a = 100 * 9.95
"%.16f" % a
=> "994.9999999999998863"

It's not a problem with Ruby, it's a problem with floating point numbers
in general. :frowning:

--
work: dga@lcs.mit.edu me: dga@pobox.com
      MIT Laboratory for Computer Science http://www.angio.net/

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show the exact value. Why shouldn't to_i benefit?

It seems to me that this type of intolerance would effectively make to_i not only worthless but dangerous.

Dan

···

On Oct 27, 2004, at 14:25, Jamis Buck wrote:

Dan Janowski wrote:

I can understand floating point errors, but this one seems too small for that.
Especially given this:
irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994

a = 100 * 9.95
"%.16f" % a
=> "994.9999999999998863"

It's not a problem with Ruby, it's a problem with floating point numbers in general. :frowning:

- Jamis

--
Jamis Buck
jgb3@email.byu.edu
http://www.jamisbuck.org/jamis

Bill Atkins wrote:

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it as 995.0, rounding it.

- Jamis

···

Bill

On Thu, 28 Oct 2004 03:27:33 +0900, Walter Szewelanczyk > <walter@mwsewall.com> wrote:

you may want to try

>ri Float#round

.

--
Jamis Buck
jgb3@email.byu.edu
http://www.jamisbuck.org/jamis

Dan Janowski wrote:

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show the exact value. Why shouldn't to_i benefit?

The value is even less exact. .to_s automatically rounds to 15 digits. Internally a higher precision is used. (Though it is still not perfect.)

Basically Ruby 'blurs' the values on output so that you need not bother about manually rounding them all the time.

Your problem is that you're using .to_i. That will give you 5.0 even for values like 5.99999999999999.

I hope my explanation was of some help.

Dan Janowski wrote:

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show the exact value. Why shouldn't to_i benefit?

It seems to me that this type of intolerance would effectively make to_i not only worthless but dangerous.

I sympathize, but I think the point is this:
   - to_i doesn't round, it truncates;
   - you have #round if you want to use it;
   - to_s has to either round or truncate; rounding
     seems logical
   - therefore expr.to_s and expr.to_i.to_s may differ

It's a matter of opinion, but I think it's working properly.
Just another of hundreds of design issues Matz has faced.

Hal

Hello!

···

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck <jgb3@email.byu.edu> wrote:

Bill Atkins wrote:

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the issue with floating point numbers is making this come out to 994.999999999983 (or wahtever ;))? If every language is plagued by this problem, then I'd be curious to know the history behind it..

Thanks!
-JD-

On Thu, Oct 28, 2004 at 04:49:28AM +0900, Jason DiCioccio scribed:

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever ;))? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

You can't represent all floating point numbers accurately in
binary. The problem in this case isn't actually in the storage
of 9.95 or 995 in floating point; it comes in when you do the
multiplication. You might take a peek at

http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

It has a nice overview of some of the pitfalls of floating point,
and the links go on from there...

  -Dave

···

--
work: dga@lcs.mit.edu me: dga@pobox.com
      MIT Laboratory for Computer Science http://www.angio.net/

Jason DiCioccio wrote:

Hello!

Bill Atkins wrote:

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the issue with floating point numbers is making this come out to 994.999999999983 (or wahtever ;))? If every language is plagued by this problem, then I'd be curious to know the history behind it..

It's a problem caused by the fact that floating point numbers cannot be represented exactly, and when you perform operations on those inexact numbers, the different between the approximation and the true becomes greater and greater.

- Jamis

···

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck > <jgb3@email.byu.edu> wrote:

--
Jamis Buck
jgb3@email.byu.edu
http://www.jamisbuck.org/jamis

Jason DiCioccio wrote:

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the issue with floating point numbers is making this come out to 994.999999999983 (or wahtever ;))? If every language is plagued by this problem, then I'd be curious to know the history behind it..

Floating point numbers are expressed in base 2. 9.95 has an infinite number of digits in base 2 representation just like the result of 1/3 has in base 10.

However floating point numbers clearly can't be stored with an infinite amount of bits which is why the above gets slightly imprecise.

You can have a look at all the digits of a float like this:

irb(main):172:0> "%.32f" % 9.95
=> "9.94999999999999930000000000000000"

Hello Jason,

Wednesday, October 27, 2004, 9:49:28 PM, you wrote:

Hello!

Bill Atkins wrote:

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever ;))? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

9.95 can not exactly put into a floeating point number with base 2.
with base 10 it's no problem.
9.95==9*10^0+9*10^-1+5*10^-2
but with bas 2 this will become a periaodic dual fraction.
so 9.95 is the problem. its internally in the computer slightly lower
than the real 9.95
multiplying by 100.0 (which can be expresses exactly) does not repair
the problem of 9.95

every language is plagued by this problem, if it uses the floating
point arithmetic given by the processor (and doing this is a good idea
because the fine speed). we as programmers have to know about this
problem and live with it. mostly its a good idea to ignore it. :slight_smile:
sometimes we don't calc 100.0 Dollars * 9.95 Dollars but do 10000
Pence * 995 Pence to avoid the problem (because irregular
pence-vanishing with doubles is not amusing, but regular
pence-vanishing with intagers can be managed).

···

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck > <jgb3@email.byu.edu> wrote:

--
Best regards,
Volkard mailto:volkard@normannia.de

Jason DiCioccio wrote:

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the issue with floating point numbers is making this come out to 994.999999999983 (or wahtever ;))? If every language is plagued by this problem, then I'd be curious to know the history behind it..

:slight_smile: It's not history, it's math.

Here's the short explanation.

Remember repeating decimals, which you learned about in elementary school?
For example, 1/3 = 0.33333... can't be expressed in a finite number of
digits.

Likewise 0.95 can't be expressed in a finite number of (binary) digits.
When you convert it to binary, it repeats infinitely.

Hal

Thanks for your responses. I actually did hear about this a long time ago I think, but I had forgotten the specifics. It's really not often that I end up dealing with floating point numbers. Most things are done (for me) with Integers it seems. I'd imagine if I were designing a game engine or doing scientific work, that would not be the case :).

Thanks again!
-JD-

···

--On Thursday, October 28, 2004 05:09:48 +0900 Volkard Henkel <volkard@normannia.de> wrote:

Hello Jason,

Wednesday, October 27, 2004, 9:49:28 PM, you wrote:

Hello!

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck >> <jgb3@email.byu.edu> wrote:

Bill Atkins wrote:

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever ;))? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

9.95 can not exactly put into a floeating point number with base 2.
with base 10 it's no problem.
9.95==9*10^0+9*10^-1+5*10^-2
but with bas 2 this will become a periaodic dual fraction.
so 9.95 is the problem. its internally in the computer slightly lower
than the real 9.95
multiplying by 100.0 (which can be expresses exactly) does not repair
the problem of 9.95

every language is plagued by this problem, if it uses the floating
point arithmetic given by the processor (and doing this is a good idea
because the fine speed). we as programmers have to know about this
problem and live with it. mostly its a good idea to ignore it. :slight_smile:
sometimes we don't calc 100.0 Dollars * 9.95 Dollars but do 10000
Pence * 995 Pence to avoid the problem (because irregular
pence-vanishing with doubles is not amusing, but regular
pence-vanishing with intagers can be managed).

--
Best regards,
Volkard mailto:volkard@normannia.de

David G. Andersen wrote:

The problem in this case isn't actually in the storage
of 9.95 or 995 in floating point; it comes in when you do the
multiplication.

I'm not sure it's *just* in the multiplication... 0.95 can't
be stored exactly in binary, can it?

Hal

Not exactly, '1/3' is finite. Rational numbers can be expressed. Irrationals
cannot. But current standards do not attempt to deal with repeating decimals,
even though they could. The reason for this seems to be a matter of history
related to an ability to test inequalities quickly.

IEEE754 is now 20 years old, and is showing its age. I have actually been
thinking of working on a improved version myself. But there's many details to
deal with, so who knows... maybe.

T.

···

On Wednesday 27 October 2004 04:36 pm, Hal Fulton wrote:

Jason DiCioccio wrote:
> I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
> issue with floating point numbers is making this come out to
> 994.999999999983 (or wahtever ;))? If every language is plagued by this
> problem, then I'd be curious to know the history behind it..
>
:slight_smile: It's not history, it's math.

Here's the short explanation.

Remember repeating decimals, which you learned about in elementary school?
For example, 1/3 = 0.33333... can't be expressed in a finite number of
digits.

trans. (T. Onoma) wrote:

> Jason DiCioccio wrote:
> > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
> > issue with floating point numbers is making this come out to
> > 994.999999999983 (or wahtever ;))? If every language is plagued by this
> > problem, then I'd be curious to know the history behind it..
> >
> :slight_smile: It's not history, it's math.
>
> Here's the short explanation.
>
> Remember repeating decimals, which you learned about in elementary school?
> For example, 1/3 = 0.33333... can't be expressed in a finite number of
> digits.

Not exactly, '1/3' is finite. Rational numbers can be expressed. Irrationals cannot. But current standards do not attempt to deal with repeating decimals, even though they could. The reason for this seems to be a matter of history related to an ability to test inequalities quickly.

Expressed as a single number in either base 10 or 2, 0.333... requires an
infinite number of digits (which is all I said).

Knowing that it is rational, you could separately store the two numbers expressing
that ratio. But that is completely different from traditional floating-point storage,
and the simple traditional methods of (e.g.) addition and subtraction would not
work.

IEEE754 is now 20 years old, and is showing its age. I have actually been thinking of working on a improved version myself. But there's many details to deal with, so who knows... maybe.

Good idea, go for it.

Hal

···

On Wednesday 27 October 2004 04:36 pm, Hal Fulton wrote:

trans. (T. Onoma) wrote:

···

On Wednesday 27 October 2004 04:36 pm, Hal Fulton wrote:
> Jason DiCioccio wrote:
> > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
> > issue with floating point numbers is making this come out to
> > 994.999999999983 (or wahtever ;))? If every language is plagued by this
> > problem, then I'd be curious to know the history behind it..
> >
> :slight_smile: It's not history, it's math.
>
> Here's the short explanation.
>
> Remember repeating decimals, which you learned about in elementary school?
> For example, 1/3 = 0.33333... can't be expressed in a finite number of
> digits.

Not exactly, '1/3' is finite.

... That's not what I learned in school. How many decimal digits does it take to express 1/3, then?

- Jamis

--
Jamis Buck
jgb3@email.byu.edu
http://www.jamisbuck.org/jamis