Float#==. Legacy?

[Guillaume_Marcais1 (Guillaume Marcais)]

As far as my experience goes, using Float#== is always an error. Float should be compared within epsilon because of errors, etc. etc. But the question is: why keep Float#== if it is basically useless? Why not have Float#== be defined in the core by:

class Float; def ==(o); ((o - self).abs < 0.0000001); end; end

(Most likely with 0.0000001 be a parameter of the Float class so it can be changed at runtime.) Is the current implementation of Float#== legacy from C (and pretty much every other language I can think of)?

It would make the behavior of Float more natural to humans and should not break much, given the uselessness of today's Float#==.

As an example, some unit tests crashed today with this message:

   1) Failure:
test_read_coords(TestScaffoldReorder) [./test/test_scaffold_reorder.rb:50]:
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}> expected but was
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}>.

It looks stupid, doesn't it? It failed because the computed float where not exactly equal to the constant entered in the testing code, but they display the same. I could drill down the hashes and compare with assert_within_epsilon (or whatever it is), but it makes the code much uglier and more complicated than a simple assert_equal. Or wrap the structure in a class and have a proper equality operator. Or...
But all this seems too complicated for the quick program at hand. And redefining Float#== as shown above made the test pass with much less pain.

So any good reason to keep Float#== the way it is? Or is there any real danger of breaking existing libraries if I redefine Float#== this way?

Guillaume.

[Tim_Pease]

If you are using Test::Unit, the assert_in_delta method should be used
for comparing floats.

I prefer the idea of an epsilon_eq? method for Float that takes
another float and a small epsilon. The epsilon would default to
Float::EPSILON which ruby alread defines.

Blessings,
TwP

···

On 7/6/06, Guillaume Marcais <guslist@free.fr> wrote:

As far as my experience goes, using Float#== is always an error. Float
should be compared within epsilon because of errors, etc. etc. But the
question is: why keep Float#== if it is basically useless? Why not have
Float#== be defined in the core by:

class Float; def ==(o); ((o - self).abs < 0.0000001); end; end

(Most likely with 0.0000001 be a parameter of the Float class so it can
be changed at runtime.) Is the current implementation of Float#==
legacy from C (and pretty much every other language I can think of)?

It would make the behavior of Float more natural to humans and should
not break much, given the uselessness of today's Float#==.

As an example, some unit tests crashed today with this message:

   1) Failure:
test_read_coords(TestScaffoldReorder)
[./test/test_scaffold_reorder.rb:50]:
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}>
expected but was
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}>.

It looks stupid, doesn't it? It failed because the computed float where
not exactly equal to the constant entered in the testing code, but they
display the same. I could drill down the hashes and compare with
assert_within_epsilon (or whatever it is), but it makes the code much
uglier and more complicated than a simple assert_equal. Or wrap the
structure in a class and have a proper equality operator. Or...
But all this seems too complicated for the quick program at hand. And
redefining Float#== as shown above made the test pass with much less
pain.

So any good reason to keep Float#== the way it is? Or is there any real
danger of breaking existing libraries if I redefine Float#== this way?

Guillaume.

[Matthew_Smillie]

As far as my experience goes, using Float#== is always an error. Float should be compared within epsilon because of errors, etc. etc.

This is something that I'd refer to as a code smell rather than an error. Floating point numbers can represent and operate on lots of actual numbers with no loss of precision, for a trivial example: 2.0 + 2.0 will always equal exactly 4.0, and any other result from that operation, no matter how small the difference, would be an error.

Floating point errors arise in specific cases: when a calculation goes outside the precision of the notation. These errors lead to problems with equality only when the two numbers being compared have been calculated in a series of steps which go outside the range of precision in different ways. For example:

2.0**53 == 2.0**53+1.0+(-1.0) # => false
2.0**53 == 2.0**53+(-1.0)+1.0 # => true

2.0**53 is the boundary case for whole integers in floating point notation. You can subtract one and get the expected result, but adding one runs into the absorption problem where large_number + 1 == large_number.

But the question is: why keep Float#== if it is basically useless? Why not have Float#== be defined in the core by:

class Float; def ==(o); ((o - self).abs < 0.0000001); end; end

Interestingly, for small values of epsilon, this could magnify the problem because subtracting two nearly-equal values is a problematic operation. In other words, the difference used for this comparison wouldn't actually represent the actual difference between the numbers in all cases.

This seems at least as counter-intuitive to me as the current problem with ==. You could have two numbers which differ by less than epsilon in mathematical terms, but computing the difference may result in more than epsilon (or vice-versa). I'm not sure that this would be an actual improvement.

Also, you'll note that it fails to solve the example given above for any value of epsilon less than 1.0.

As an example, some unit tests crashed today with this message:

  1) Failure:
test_read_coords(TestScaffoldReorder) [./test/test_scaffold_reorder.rb:50]:
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}> expected but was
<{"9876"=>{"8153"=>[19.1, [63.7, 580.0]], "8154"=>[15.0, [1612.5]]}}>.

It looks stupid, doesn't it? It failed because the computed float where not exactly equal to the constant entered in the testing code, but they display the same. I could drill down the hashes and compare with assert_within_epsilon (or whatever it is), but it makes the code much uglier and more complicated than a simple assert_equal. Or wrap the structure in a class and have a proper equality operator. Or...
But all this seems too complicated for the quick program at hand. And redefining Float#== as shown above made the test pass with much less pain.

So any good reason to keep Float#== the way it is? Or is there any real danger of breaking existing libraries if I redefine Float#== this way?

As I mentioned before, Float#== isn't necessarily an error, it's just an indication that there may be errors - a code smell.

Another code smell is an overuse of literals and constants, and this is equally the case for testing code as for production code. This seems to me to be where the error really lies: you're using a constant in the testing code where you should be using a computed value based on the input.

Knowing (as we do), that floating point calculations are not 100% accurate, a more reliable approach would be to make the test results using a parallel calculation to the code being tested, not simply a constant. In a trivial example:

def foo(a, b) a/b end

# test setup
a = 22.0
b = 7.0
const_result = 3.14285714285714
calc_result = 22.0/7.0

foo(a, b) == const_result # => false
foo(a, b) == calc_result # => true

matthew smillie.

···

On Jul 6, 2006, at 16:39, Guillaume Marcais wrote:

[Martin_DeMello]

This is ugly, because Float#== is a two-argument method that you're
faking with one argument and one global constant. An array argument
might be a nice piece of syntactic sugar, though I'd still prefer to
overload =~ instead, e.g. a =~ [b, epsilon] with a =~ b defaulting to
Float::EPSILON.

Also, note that your method definition needs to be

def =~ (o); ((o - self)/o).abs <= EPSILON; end

you want relative, not absolute, error margins.

martin

···

On 7/6/06, Guillaume Marcais <guslist@free.fr> wrote:

As far as my experience goes, using Float#== is always an error. Float
should be compared within epsilon because of errors, etc. etc. But the
question is: why keep Float#== if it is basically useless? Why not have
Float#== be defined in the core by:

class Float; def ==(o); ((o - self).abs < 0.0000001); end; end

(Most likely with 0.0000001 be a parameter of the Float class so it can
be changed at runtime.) Is the current implementation of Float#==
legacy from C (and pretty much every other language I can think of)?

It would make the behavior of Float more natural to humans and should
not break much, given the uselessness of today's Float#==.

[Jacob_Fugal]

While I agree (mostly) with the rest of your post, I think this is an
oversimplification of the problem, and produces a "solution" more
painful than the "problem" iteslf.

You are right, that choosing the wrong delta for assert_in_delta can
be just as problematic as using Float#==. But I don't think that
throwing away assert_in_delta and using assert_equal with a parallel
calculation is the answer. In the trivial example you gave, it's fine,
since you've got one operation. But what if the method under test
performs many operations, due to complicated business rules?
(Hopefully those rules are factored out into their own tested methods,
but that doesn't stop them from being part of the behavior of that
method). Should we duplicate the entire process in the test? That
seems wasteful and error prone. I'll stick with assert_in_delta and an
expected value.

The important thing when dealing with floating point values is to
*always* know the domain and range (speaking mathematically) of your
function, then test those functions with appropriate deltas.

Jacob Fugal

···

On 7/6/06, Matthew Smillie <M.B.Smillie@sms.ed.ac.uk> wrote:

Another code smell is an overuse of literals and constants, and this
is equally the case for testing code as for production code. This
seems to me to be where the error really lies: you're using a
constant in the testing code where you should be using a computed
value based on the input.

Knowing (as we do), that floating point calculations are not 100%
accurate, a more reliable approach would be to make the test results
using a parallel calculation to the code being tested, not simply a
constant.

[Pit]

Matthew Smillie schrieb:

As far as my experience goes, using Float#== is always an error. Float should be compared within epsilon because of errors, etc. etc.

This is something that I'd refer to as a code smell rather than an error. Floating point numbers can represent and operate on lots of actual numbers with no loss of precision, for a trivial example: 2.0 + 2.0 will always equal exactly 4.0, and any other result from that operation, no matter how small the difference, would be an error.

You have to know a lot about the implementation of floating point numbers in order to be able to predict the result of seemingly trivial examples:

   0.1 + 0.1 == 0.2 # => true
   0.1 + 0.2 == 0.3 # => false

The only really trivial examples I can think of use small integers. If I'm working with those numbers, I wouldn't use floats, though.

Regards,
Pit

···

On Jul 6, 2006, at 16:39, Guillaume Marcais wrote:

[Tim_Pease]

Martin, I like the syntax. To make it more like Float#==

class Float
  def =~( other )
    epsilon_eql?(coerce(other), EPSILON)
  end

  def epsilon_eql?( other, epsilon )
    return false unless other.instance_of? self.class
    ((other-self)/other).abs <= epsilon
  end
end

I think this gives the best of both worlds. A simple =~ syntax for
using the default epsilon, and the slightly less elegant epsilon_eql?
syntax when you want to sepcify your own.

Blessings,
TwP

···

On 7/7/06, Martin DeMello <martindemello@gmail.com> wrote:

This is ugly, because Float#== is a two-argument method that you're
faking with one argument and one global constant. An array argument
might be a nice piece of syntactic sugar, though I'd still prefer to
overload =~ instead, e.g. a =~ [b, epsilon] with a =~ b defaulting to
Float::EPSILON.

Also, note that your method definition needs to be

def =~ (o); ((o - self)/o).abs <= EPSILON; end

you want relative, not absolute, error margins.

martin

[Matthew_Smillie]

I didn't (and wouldn't) suggest abandoning assert_in_delta. What I was discussing was the idea to push that functionality (which is specific to certain situations) into the general Float#== method, where, for a number of reasons, it's inappropriate.

As for the example of "what if the method uses other really complicated methods?" though, aren't those other methods unit tested, likely on the same general input domain? If so, and they pass to your satisfaction, then why not simply use them in the calculation of the test value for the encompassing method - errors which arise from the subsidiary methods should be caught in the unit tests for those methods, so there's no loss of generality in the testing. To extend the trivial example from before:

def foo(a, b)
   # arbitrary floating point math
   a * (some_method(a) + other_method(a, b))
end

# test setup - these should all be values for which
# you're confident the implementation is adequate.
a = 22.0
b = SomeCompany.new("test")
c = some_method(a)
d = other_method(a, b)

# replicate the fundamental logic of the method being tested here.
calc_result = a * (c + d)

The test, after all, is on the logic of the method being tested, not on the behaviour of all associated methods - those have their own tests to validate them.

I don't know if this is any more error prone than using an expected value, since you have to make the complicated calculations *somewhere* in order to come up with the expected value in the first case. Making them in the tests themselves definitely has the advantage of self-documentation, and self-adjustment if the subsidiary methods are changed (such changes, presumably, tested in their own unit tests).

As for more wasteful, it is (like so many other things) a trade-off. What if the variation you're seeing isn't a floating point error, but something incorrect in your logic somewhere? Say, a rounding error in a database field, or a problem with serialisation to/from SOAP. What's the risk of that? What's the impact? If you're only concerned about accuracy within some delta, then you only need to test to that delta. It's just that sometimes that delta might be 0.

So, there are lots of times where assert_in_delta is the best thing to do (ensuring two different methods or computation agree to some degree, developer time, simplicity), but that doesn't meant that Float#== should take on that functionality.

matthew smillie.

···

On Jul 6, 2006, at 21:49, Jacob Fugal wrote:

On 7/6/06, Matthew Smillie <M.B.Smillie@sms.ed.ac.uk> wrote:

Another code smell is an overuse of literals and constants, and this
is equally the case for testing code as for production code. This
seems to me to be where the error really lies: you're using a
constant in the testing code where you should be using a computed
value based on the input.

Knowing (as we do), that floating point calculations are not 100%
accurate, a more reliable approach would be to make the test results
using a parallel calculation to the code being tested, not simply a
constant.

While I agree (mostly) with the rest of your post, I think this is an
oversimplification of the problem, and produces a "solution" more
painful than the "problem" iteslf.

You are right, that choosing the wrong delta for assert_in_delta can
be just as problematic as using Float#==. But I don't think that
throwing away assert_in_delta and using assert_equal with a parallel
calculation is the answer. In the trivial example you gave, it's fine,
since you've got one operation. But what if the method under test
performs many operations, due to complicated business rules?
(Hopefully those rules are factored out into their own tested methods,
but that doesn't stop them from being part of the behavior of that
method). Should we duplicate the entire process in the test? That
seems wasteful and error prone. I'll stick with assert_in_delta and an
expected value.

[Tim_Pease]

Bug in my own method

def =~( other )
  epsilon_eql?(coerce(other)[0], EPSILON)
end

<sigh> it's definitely a Friday

TwP

···

On 7/7/06, Tim Pease <tim.pease@gmail.com> wrote:

> On 7/7/06, Martin DeMello <martindemello@gmail.com> wrote:
>
> This is ugly, because Float#== is a two-argument method that you're
> faking with one argument and one global constant. An array argument
> might be a nice piece of syntactic sugar, though I'd still prefer to
> overload =~ instead, e.g. a =~ [b, epsilon] with a =~ b defaulting to
> Float::EPSILON.
>
> Also, note that your method definition needs to be
>
> def =~ (o); ((o - self)/o).abs <= EPSILON; end
>
> you want relative, not absolute, error margins.
>
> martin
>

Martin, I like the syntax. To make it more like Float#==

class Float
  def =~( other )
    epsilon_eql?(coerce(other), EPSILON)
  end

  def epsilon_eql?( other, epsilon )
    return false unless other.instance_of? self.class
    ((other-self)/other).abs <= epsilon
  end
end

I think this gives the best of both worlds. A simple =~ syntax for
using the default epsilon, and the slightly less elegant epsilon_eql?
syntax when you want to sepcify your own.

Blessings,
TwP

[Ara.T.Howard6 (Ara.T.Howard)]

hi guys-

i've written this at least 10 times - how bout an rcr?

-a

···

On Sat, 8 Jul 2006, Tim Pease wrote:

Martin, I like the syntax. To make it more like Float#==

class Float
def =~( other )
  epsilon_eql?(coerce(other), EPSILON)
end

def epsilon_eql?( other, epsilon )
  return false unless other.instance_of? self.class
  ((other-self)/other).abs <= epsilon
end

I think this gives the best of both worlds. A simple =~ syntax for
using the default epsilon, and the slightly less elegant epsilon_eql?
syntax when you want to sepcify your own.

Blessings,
TwP

--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

[Jacob_Fugal]

> You are right, that choosing the wrong delta for assert_in_delta can
> be just as problematic as using Float#==. But I don't think that
> throwing away assert_in_delta and using assert_equal with a parallel
> calculation is the answer. In the trivial example you gave, it's fine,
> since you've got one operation. But what if the method under test
> performs many operations, due to complicated business rules?
> (Hopefully those rules are factored out into their own tested methods,
> but that doesn't stop them from being part of the behavior of that
> method). Should we duplicate the entire process in the test? That
> seems wasteful and error prone. I'll stick with assert_in_delta and an
> expected value.

I didn't (and wouldn't) suggest abandoning assert_in_delta. What I
was discussing was the idea to push that functionality (which is
specific to certain situations) into the general Float#== method,
where, for a number of reasons, it's inappropriate.

Ah, ok, we're on the same page for that discussion then. I was
initially intrigued by the idea, but your post made it obvious that it
doesn't belong there, primarily -- in my mind -- because of the lack
of a "universal" delta. And the syntax of == doesn't make it very
convenient to specify the delta.

As for the example of "what if the method uses other really
complicated methods?" though, aren't those other methods unit tested,
likely on the same general input domain? If so, and they pass to
your satisfaction, then why not simply use them in the calculation of
the test value for the encompassing method - errors which arise from
the subsidiary methods should be caught in the unit tests for those
methods, so there's no loss of generality in the testing. To extend
the trivial example from before:

def foo(a, b)
   # arbitrary floating point math
   a * (some_method(a) + other_method(a, b))
end

# test setup - these should all be values for which
# you're confident the implementation is adequate.
a = 22.0
b = SomeCompany.new("test")
c = some_method(a)
d = other_method(a, b)

# replicate the fundamental logic of the method being tested here.
calc_result = a * (c + d)

This is exactly what I was getting at though. You've basically
replicated the entire function in the test itself! I don't think
that's the point of having the unit test.

What if the variation you're seeing isn't a floating point error, but
something incorrect in your logic somewhere? Say, a rounding error
in a database field, or a problem with serialisation to/from SOAP.
What's the risk of that? What's the impact? If you're only
concerned about accuracy within some delta, then you only need to
test to that delta. It's just that sometimes that delta might be 0.

All good points. That's why I stress the importance of using the
appropriate delta. For instance, if you're working with currency
represented as floats (bad monkey!) then a delta of 0.001 should be
plenty sufficient. If I'm actually doing floating point math however,
I probably want to use a very small delta to make sure my calculations
aren't actually off.

So, there are lots of times where assert_in_delta is the best thing
to do (ensuring two different methods or computation agree to some
degree, developer time, simplicity), but that doesn't meant that
Float#== should take on that functionality.

Agreed.

Jacob Fugal

···

On 7/6/06, Matthew Smillie <M.B.Smillie@sms.ed.ac.uk> wrote:

On Jul 6, 2006, at 21:49, Jacob Fugal wrote:

[Tim_Pease]

Done.

  RCR 340: Approximate comparison of floats

···

On 7/7/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Sat, 8 Jul 2006, Tim Pease wrote:

> Martin, I like the syntax. To make it more like Float#==
>
> class Float
> def =~( other )
> epsilon_eql?(coerce(other), EPSILON)
> end
>
> def epsilon_eql?( other, epsilon )
> return false unless other.instance_of? self.class
> ((other-self)/other).abs <= epsilon
> end
>
> I think this gives the best of both worlds. A simple =~ syntax for
> using the default epsilon, and the slightly less elegant epsilon_eql?
> syntax when you want to sepcify your own.
>
> Blessings,
> TwP

hi guys-

i've written this at least 10 times - how bout an rcr?

-a
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

[Joel_VanderWerf1 (Joel VanderWerf)]

The Float#=~ that Martin suggested is nice, but it still doesn't help with one of the problems which (I think) the OP wanted:

x = 1.2
y = 1.2000000000000001
p(x =~ y) # => true
p([x] =~ [y]) # => false

Array#=~ is not defined now, and Object#=~ always returns false (see rb_obj_pattern_match()).

Should Array#=~ attempt to propagate the #=~ call through its members? Or is there a good reason why this is left unspecified by ruby.

The same question arises for hashes...

Any ideas?

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

[Guillaume_Marcais1 (Guillaume Marcais)]

Yes, this is what I was referring to.

I came about a little differently. Mainly that calling Float#within_delta or equivalent is easy when I have the float itself, but if is embedded in an Array or Hash, it renders #== useless on the structure:

[1+1, 3] == [2, 3] # => true
[0.1+0.2, 3] == [0.3, 3] # => false

For the second case, one would needs to loop on the elements of the array.

And my second realization is if Float#== is (almost) always a mistake, why not change it to something useful most of the time. Changing Float#== in my code the way I did make me a little nervous as it is changing a pretty fundamental behavior, but on the other hand it is hardly ever used and it made sense in my case.

Guillaume.

···

Le 7 juil. 06, à 15:30, Joel VanderWerf a écrit :

The Float#=~ that Martin suggested is nice, but it still doesn't help with one of the problems which (I think) the OP wanted:

x = 1.2
y = 1.2000000000000001
p(x =~ y) # => true
p( =~ [y]) # => false

[Juergen_Strobel]

But the current #== is the best you can come up with in general. There
is no universally correct epsilon, it depends on the operations and
actual numbers used so far, and small errors in successive floating
point operations do accumulate, sometimes drastically.

A whole subfield of mathematics is concerned with this. Depending on
your code, you might have to apply different epsilons even to floats
in the same array.

If it makes sense in your code go for it, but this depends on your
code and would do more harm than good in general.

-Jürgen

···

On Sat, Jul 08, 2006 at 11:22:37AM +0900, Guillaume Marcais wrote:

Le 7 juil. 06, à 15:30, Joel VanderWerf a écrit :

>
>The Float#=~ that Martin suggested is nice, but it still doesn't help
>with one of the problems which (I think) the OP wanted:
>
>x = 1.2
>y = 1.2000000000000001
>p(x =~ y) # => true
>p( =~ [y]) # => false
>

Yes, this is what I was referring to.

I came about a little differently. Mainly that calling
Float#within_delta or equivalent is easy when I have the float itself,
but if is embedded in an Array or Hash, it renders #== useless on the
structure:

[1+1, 3] == [2, 3] # => true
[0.1+0.2, 3] == [0.3, 3] # => false

For the second case, one would needs to loop on the elements of the
array.

And my second realization is if Float#== is (almost) always a mistake,
why not change it to something useful most of the time. Changing
Float#== in my code the way I did make me a little nervous as it is
changing a pretty fundamental behavior, but on the other hand it is
hardly ever used and it made sense in my case.

Guillaume.

--
The box said it requires Windows 95 or better so I installed Linux