Bug Rounding Floats? (9.245 * 100).round => 924? S/B 925!

ruby-talk
1

All,

Would someone try the below unit test and explain why ruby does not
pass. It came up while I was trying to write a generalized rounding
method for floats that takes the number of place to round to as a
parameter. It gave wrong answers, but only on rare inputs.

For example, (9.245 * 100).round should be 925, but my ruby gives 924.

···

=================================
#! /usr/bin/env ruby
#
require 'test/unit'

class TestFloatRounding < Test::Unit::TestCase
  def test_round
    assert_equal(925, 924.5.round, "Rounding Error: 924.5.round")
    assert_equal(9.25, (9.245 * 10.0**2).round / 10.0**2, "Rounding
Error: (9.245 * 10.0**2).round / 10.0**2")
    assert_equal(925, (9.245 * 10.0**2).round, "Rounding Error:(9.245
* 10.0**2).round")
    assert_equal(924.5, 9.245 * 10.0**2, "Rounding Error: 9.245 *
10.0**2")
    assert_equal(925, 924.5.round, "Rounding Error: 924.5.round")
    assert_equal(924.5, 9.245 * 10 * 10, "Rounding Error: 9.245 * 10 *
10")
    assert_equal(925, (9.245 * 10 * 10).round, "Rounding Error: (9.245
* 10 * 10).round")
    assert_equal(925.0, 9.245 * 10 * 10 + 0.5, "Rounding Error: 9.245
* 10 * 10 + 0.5")
    assert_equal(925, (9.245 * 10 * 10 + 0.5).floor, "Rounding Error:
(9.245 * 10 * 10 + 0.5).floor")
  end
end

2

try what your term really is:

printf("%.50f",9.245 * 100)

-Thomas

···

2009/7/22 ddoherty03 <ddoherty03@gmail.com>

For example, (9.245 * 100).round should be 925, but my ruby gives 924.

--
Thomas Preymesser
thopre@gmail.com

Mike Ditka <http://www.brainyquote.com/quotes/authors/m/mike_ditka.html&gt; -
"If God had wanted man to play soccer, he wouldn't have given us arms."

3

Welcome to IEEE floating-point arithmetic. Any programming language
that uses it will be subject to the same results.

···

At 2009-07-22 01:37PM, "Thomas Preymesser" wrote:

[Note: parts of this message were removed to make it a legal post.]

2009/7/22 ddoherty03 <ddoherty03@gmail.com>

> For example, (9.245 * 100).round should be 925, but my ruby gives 924.
>

try what your term really is:

  printf("%.50f",9.245 * 100)

--
Glenn Jackman
    Write a wise saying and your name will live forever. -- Anonymous

4

A workaround, convert the expression to a string and back to a number:
    
    num = Float("%.1f" % (9.245 * 100)).round

    puts "yippee" if num == 925

···

At 2009-07-22 02:08PM, "Glenn Jackman" wrote:

At 2009-07-22 01:37PM, "Thomas Preymesser" wrote:
> [Note: parts of this message were removed to make it a legal post.]
>
> 2009/7/22 ddoherty03 <ddoherty03@gmail.com>
>
>
> > For example, (9.245 * 100).round should be 925, but my ruby gives 924.
> >
>
> try what your term really is:
>
> printf("%.50f",9.245 * 100)

Welcome to IEEE floating-point arithmetic. Any programming language
that uses it will be subject to the same results.

--
Glenn Jackman
    Write a wise saying and your name will live forever. -- Anonymous

5

Glenn,

Thanks for the explanation. If I could pick your brain for a second
on how to generalize your workaround, I would greatly appreciate it.

Here is how I tried to implement the "nround" method.

···

On Jul 22, 1:13 pm, Glenn Jackman <gle...@ncf.ca> wrote:

A workaround, convert the expression to a string and back to a number:

num = Float\(&quot;%\.1f&quot; % \(9\.245 \* 100\)\)\.round

puts &quot;yippee&quot; if num == 925

######################################
class Float
  def nround(n = 0)
    (self * 10.0 ** n).round / 10.0 ** n
  end
end
######################################

My issue is getting the workaround to deal with the parameter n
properly.

Regards,

6

ddoherty03 wrote:

Thanks for the explanation. If I could pick your brain for a second
on how to generalize your workaround, I would greatly appreciate it.

If performance isn't a problem, you could use 'bigdecimal':

  http://ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/

See for example,

  Ruby Quiz - NDiff (#46)

Regards,

···

--
Bil Kleb
http://fun3d.larc.nasa.gov
http://twitter.com/bil_kleb

7

Taking Bil Kleb's advice,

    require 'bigdecimal'
    require 'bigdecimal/util'

    class Float
      def nround(n=0)
        (self.to_d * (10.0**n).to_d).round.to_f / 10.0**n
      end
    end

    p 9.245.nround(2) # => 9.25

···

At 2009-07-22 03:22PM, "ddoherty03" wrote:

On Jul 22, 1:13 pm, Glenn Jackman <gle...@ncf.ca> wrote:
> A workaround, convert the expression to a string and back to a number:
> num = Float("%.1f" % (9.245 * 100)).round
> puts "yippee" if num == 925

Glenn,

Thanks for the explanation. If I could pick your brain for a second
on how to generalize your workaround, I would greatly appreciate it.

Here is how I tried to implement the "nround" method.

######################################
class Float
   def nround(n = 0)
     (self * 10.0 ** n).round / 10.0 ** n
   end
end
######################################

My issue is getting the workaround to deal with the parameter n
properly.

--
Glenn Jackman
    Write a wise saying and your name will live forever. -- Anonymous

8

Bil & Glenn & Thomas,

Thanks. Performance is not a problem, so this works for me.

Much appreciated.

···

On Jul 23, 9:37 am, Glenn Jackman <gle...@ncf.ca> wrote:

At 2009-07-22 03:22PM, "ddoherty03" wrote:

> On Jul 22, 1:13 pm, Glenn Jackman <gle...@ncf.ca> wrote:
> > A workaround, convert the expression to a string and back to a number:
> > num = Float("%.1f" % (9.245 * 100)).round
> > puts "yippee" if num == 925

> Glenn,

> Thanks for the explanation. If I could pick your brain for a second
> on how to generalize your workaround, I would greatly appreciate it.

> Here is how I tried to implement the "nround" method.

> ######################################
> class Float
> def nround(n = 0)
> (self * 10.0 ** n).round / 10.0 ** n
> end
> end
> ######################################

> My issue is getting the workaround to deal with the parameter n
> properly.

Taking Bil Kleb's advice,

require &#39;bigdecimal&#39;
require &#39;bigdecimal/util&#39;

class Float
  def nround\(n=0\)
    \(self\.to\_d \* \(10\.0\*\*n\)\.to\_d\)\.round\.to\_f / 10\.0\*\*n
  end
end

p 9\.245\.nround\(2\)  \# =&gt; 9\.25

--
Glenn Jackman
Write a wise saying and your name will live forever. -- Anonymous