Major Addition Bug?

ruby-talk
21

In article 10a4scd81rq1le3@corp.supernews.com,

···

Michael Geary Mike@DeleteThis.Geary.com wrote:

Phil Tomson wrote:

[Floating point imprecision seems] to present a major
problem if you want to deal with financial figures, for
example. Are there any workarounds?

Financial calculations aren’t done in floating point. There are various
number formats that are used, but they all boil down to some form of integer
arithmetic. For example, fixed point decimal may be used, where the value
$625.91 is represented as the integer 62591 with a scaling factor of two
decimal places. In other words, the arithmetic is actually done in pennies,
not dollars.

-Mike

This seems to be the 100,000th post to Ruby-talk (at least the English
version).

What has our player won?

BTW: Yes, doing financial calculations in pennies seems to make a lot of
sense.

Phil

22

Lloyd Zusman wrote:

In 1969 I took a course called Introduction to Computer Programming. In
one of the early chapters of the textbook, we learned about a concept
called “floating point arithmetic”. One aspect of this topic that was
drilled into us is the way that floating point arithmetic does not yield
exact results.

I want to add that this concept, which is so basic to the understanding
of programming that it was taught in an introductory course before we
even wrote a line of code, was old information even in 1969.

This basic concept of floating point arithmetic is every bit as
pertinent today as it was in 1969 and the years prior to that. Did you
actually not learn this concept before starting to program in Ruby?

I don’t know Sean’s situation or background, but let me point out that
many people on this list are not of an actual computer science
background. Best know example: David Alan Black, though very
knowledgeable in computing, actually comes from a humanities background
if I’m not mistaken.

Of course, overall I agree with you. I’d be disappointed if I found that
someone with a degree in computer science (or even 2-3 semesters) didn’t
understand this fact.

But some people on this list are self-taught programmers. They may never
have had a single class in computer science. No discrete structures, no
digital systems, no operating systems, no analysis of algorithms.

And while I agree that this fact of floating-point life is an essential
concept for any serious programmer, it’s not the sort of thing you learn
when you’re self-taught. Hmm, maybe there should be a book of that stuff
for the self-taught programmer. (/me thinks and scribbles)

Anyhow, we have a wide diversity of ages and experience levels here.
Pretty cool, I think.

Some people on this list have not even finished high school yet, I
am guessing…

Cheers,
Hal

23

It’s great to get some validation on some of my ancient rounding problems. It
never occurred to me that the problem rested with a fundamental issue in the
floating point processor. I had always thought I was doing something wrong
or the compiler was funky.

I’ve done a lot of financial stuff using long integers and pennies. It’s nice
to hear that’s a method others use, and recommend.

All: Don’t worry about busting my chops. It was earned. I am just as likely
to nail someone for their ignorance. Live and learn.

Anyway, upside: this didn’t cost me a dime in tuition and now I know. =)
Suckers! j/k

Sean O'Dell
···

On Wednesday 12 May 2004 12:25, Aredridel wrote:

I’m not sure how well known this is. Is it well known as being a problem
or well known as in “you should never do this sort of thing”.

Seems to present a major problem if you want to deal with financial
figures, for example. Are there any workarounds?

Short answer: use rational numbers – slower, but works.

Long answer: For finance, specifically, use integer numbers of pennies,
at least for US currency, or some other fixed-point format. It has
error, but it’s an accepted, standard error. According to most
accountants, 0.005 + 0.044 is 0.05.

24

Lloyd Zusman wrote:

This basic concept of floating point arithmetic is every bit as
pertinent today as it was in 1969 and the years prior to that.

Careful. Decimal floating point is not, per se, a contradiction in
terms. Many machines had it in the 1950s, and some systems still
support it in software to this day.

···

–
John W. Kennedy
“You can, if you wish, class all science-fiction together; but it is
about as perceptive as classing the works of Ballantyne, Conrad and W.
W. Jacobs together as the ‘sea-story’ and then criticizing that.”
– C. S. Lewis. “An Experiment in Criticism”

25

Hal Fulton wrote:

Seriously, I think it’s a matter of trying to store an infinite
number of digits in a finite number of bits.

For a daily life example: How many digits does it take to store
“1/3” exactly? 0.333333…

SCNR, but:

irb(main):001:0> 1/3
=> 0

That result fits into a single bit. :wink:

But in general there’s not much to add to this thread, but to summarize it:

  • For financial calculations use specialised classes (which should
    handle different currencies, BTW)
  • If you need a higher number of digits use BigDecim (which is
    documented in the ‘Ruby Standard Library Documentation’ available
    at Index of /downloads/stdlib )
  • Never, repeat never compare floats with == or other operators like
    that. Define some kind of accuracy, say epsilon and use a usual metric
    like this:
    ( a_value - something_else ).abs < epsilon
  • Don’t use == to compare floats and last but not least
  • Comparing floats with == is a Bad Thing.

Well, that’s it - more or less.

Happy rubying,

Stephan

26

Exactly. Has anyone ever tried to write 1.1 in binary? Tell me when you’ve
finished writing it :wink:
Use BigNums which are probably an implementation (I haven’t looked into
Ruby’s) of BCD or similar. BCD (Binary Coded Decimal) has been in use for
ages. It uses one byte per decimal digit. A less wasteful form is called
packed BCD and uses a nibble (4 bits) to store a decimal digit.
BCD is older than punch cards btw.

Another way out using floats is to redefine Float::==

$ irb
irb(main):001:0> class Float
irb(main):002:1> def ==(other)
irb(main):003:2> (self-other).abs<1E-7
irb(main):004:2> end
irb(main):005:1> end
=> nil
irb(main):006:0> (625.91 + 900.00 + 22.00) == 1547.91
=> true

Hth

···

On Thu, 13 May 2004 04:15:59 +0900, Hal Fulton wrote:

Sean O’Dell wrote:

On Wednesday 12 May 2004 09:53, ts wrote:

“S” == Sean O’Dell sean@celsoft.com writes:

raise “false” if ((625.91 + 900.00 + 22.00) != 1547.91)

Well, this is well known

Well known? I had no idea this bug existed until I wrote a line of code that
double-checked the arithmetic from a different app. I have code all over
doing simple arithmetic like this. Ruby can’t add 3 simple numbers? I don’t
mean to be disproportionately negative, but that shakes my faith more than a
little. It’s just simple arithmetic. How old is this bug? When is it
getting fixed?

Offhand I’d say this bug dates back to the 1940s, and isn’t
getting fixed any time soon.

Seriously, I think it’s a matter of trying to store an infinite
number of digits in a finite number of bits.

For a daily life example: How many digits does it take to store
“1/3” exactly? 0.333333…

Hal

27

Sean O’Dell wrote:

“S” == Sean O’Dell sean@celsoft.com writes:

raise “false” if ((625.91 + 900.00 + 22.00) != 1547.91)

Well, this is well known

Offhand I’d say this bug dates back to the 1940s, and isn’t
getting fixed any time soon.

Seriously, I think it’s a matter of trying to store an infinite
number of digits in a finite number of bits.

No. I’d rather think this is not the problem. The problem is
that you should not compare floating-point numbers with == or != unless you
have a good reason to do it.

In this case, you should just do this:

raise “false” if ((625.91 + 900.00 + 22.00) - 1547.91).abs>1e-12

which just succeeds fine. You don’t usually need an infinite precision, and
in more complicated cases, you can’t even acheive one. (You could use
rationals here, because these are only simple arithmetic operations, and
you’d get an exact result.

···

On Thu, May 13, 2004 at 04:15:59AM +0900, Hal Fulton wrote:

On Wednesday 12 May 2004 09:53, ts wrote:

For a daily life example: How many digits does it take to store
“1/3” exactly? 0.333333…

Hal

28

“Sean O’Dell” sean@celsoft.com writes:

raise “false” if ((625.91 + 900.00 + 22.00) != 1547.91)

How old is this bug? When is it
getting fixed?

It’s about 40 years old, and unlikely to be fixed. Floating point
numbers are not represented exactly inside computers, and so floating
point comparisons are routinely deprecated in books on programming.
Certain values cannot ever be expressed in floating point
representation.

If you want exact, fractional, math, you should probably use the
‘rational’ library and investigate ‘mathn’.

Funny though, if I set a variable to 1547.91 exactly, it is
represented just fine. A value during the arithmetic must be a value
which can’t be represented.

Even the value 1547.91 might not be stored exactly as that value in its
internal floating point representation. Doing the arithmentic isn’t the
only way that “exact” values to become inexact. Even the mere storing
the number internally in a floating point value might cause it it lose
accuracy. Doing arithmetic simply causes these errors to accumulate,
and therefore be more likely to occur in a result.

For example, this number looks exact, but actually, it’s an infinitely
repeating fraction in binary: 0.1

This is true for many “exact” values.

Therefore, you will lose accuracy simply by referencing many decimal
numbers in floating point.

I had to massage my floating point math in C/C++ a lot many years ago,
but I always assumed it was a flaw in the MS compiler, and once I got
in the habit of doing it, I forgot about it.

You have a basic misunderstanding of floating point math if you consider
its rounding and truncation characteristics to be a compiler bug.

As other people have mentioned, there is a BigDecimal class in Ruby
that behaves closer to the way that you want.

Also, consider this: if you always want the computer to give you exact
results, what should these exact results be for each of the following
arithmetic evaluations? …

a = 1/3
b = Math.atan2(1.0, 1.0)
c = Math.sqrt(5)

How many decimal points should they go to. Where should they round?

···

On Wednesday 12 May 2004 10:18, Dave Thomas wrote:

On May 12, 2004, at 12:07, Sean O’Dell wrote:

On Wednesday 12 May 2004 09:53, ts wrote:

Thanks for the info. I’m far less bitter now about having to round the
numbers for comparison. =)

Sean O’Dell

–
Lloyd Zusman
ljz@asfast.com
God bless you.

29

If you want exact, fractional, math, you should probably use the
‘rational’ library and investigate ‘mathn’.

Funny though, if I set a variable to 1547.91 exactly, it is
represented just
fine. A value during the arithmetic must be a value which can’t be
represented.

I had to massage my floating point math in C/C++ a lot many years ago,
but I
always assumed it was a flaw in the MS compiler, and once I got in the
habit
of doing it, I forgot about it.

just a point here… floats are just one binary integer times two to
the power of another binary integer:

num = 0b11000001011111010001111010111000010100011110101110001 * 0b10
** -0b101010
==>1547.91
“%.43f”%num
==>“1547.9100000000000818545231595635414123535156250”
“%.43f”%1547.91
==>“1547.9100000000000818545231595635414123535156250”

The BigDecimal class uses actual decimal numbers, IIRC. It might be
more appropriate for a calculator app, or something like that. I
suppose it is much slower, however.

Thanks for the info. I’m far less bitter now about having to round the
numbers for comparison. =)

rather than rounding, you might consider:

(625.91 + 900.00 + 22.00) - 1547.91 > 0.0000000001
==>false

or even, for the scope of your app:

class Float
def ==(other)
self - other < 0.000000000001
end
end
==>nil
(625.91 + 900.00 + 22.00) == 1547.91
==>true

or, if you aren’t into redefing #==, you might just make a method, say,
#approx_eql?(other) for Float.

cheers,
–Mark

···

On May 12, 2004, at 10:27 AM, Sean O’Dell wrote:

30

Mohammad Khan wrote:

I guess, you guys are missing the point,

in real life you can’t have exact answer of 1/3 + 2.3 + 1.3
but you can have exact answer of 625.91 + 900.00 + 22.00

Only if you build a computer that works in decimal. In binary,
625.91 is as inexact as 1/3 is in decimal.

Hal

31

Sean O’Dell wrote:

Funny though, if I set a variable to 1547.91 exactly, it is represented just
fine.

No it isn’t.

ruby -e “puts sprintf(‘%20.13f’, 1547.91)”

I had to massage my floating point math in C/C++ a lot many years ago, but I
always assumed it was a flaw in the MS compiler,

Kindly get in touch with your C/C++ teacher and shoot him. What we are
talking about here is a basic characteristic of nearly all computers
made since the mid-1960’s, and a large fraction of the computers made
before then.

Only a handful of languages support exact decimal arithmetic (COBOL,
PL/I, Ada, Java, and REXX come to mind), and IBM zSeries and iSeries
mainframes are about the only modern machines capable of doing decimal
arithmetic in hardware.

···

–
John W. Kennedy
“Compact is becoming contract,
Man only earns and pays.”
– Charles Williams. “Bors to Elayne: On the King’s Coins”

32

damn, I got distracted by my brother or I would have been the one!

Now I have to wait for 1_000_000 to put my name in the history :confused:

···

il 12 May 2004 19:00:24 GMT, ptkwt@aracnet.com (Phil Tomson) ha scritto::

This seems to be the 100,000th post to Ruby-talk (at least the English
version).

33
  • For financial calculations use specialised classes (which should
    handle different currencies, BTW)

This got me thinking. It would be really cool to have a currency class
that allowed for conversion between currencies using some web-retrieved
table of current conversion rates. Rates would be cached after first
being retrieved and would be updated once a day or so.

Carl

34

I guess, you guys are missing the point,

in real life you can’t have exact answer of 1/3 + 2.3 + 1.3

Oh, but you can! you just need to choose an appropriate radix. Radix 30
will do:

1/3 + 2.3 + 1.3 becomes:

0.a + 2.9 + 1.9 = 3.s

Your answer is 3.s radix 30. This translates to approximately 3.93333
radix ten (decimal); or about 11.11101 radix 2 (binary).

In real life, number stay fractions. Decimal notation is only precise
for numbers whose denominators factor out to twos and fives. Binary
(and hence floating point arithmetic) is only precise for fractions
whose denominator is factorable into twos. So unless your decimal
number ends in .25, .125, and so on, you’re out of luck.

but you can have exact answer of 625.91 + 900.00 + 22.00

Only when using base ten, which computers usually don’t.

cheers,
–Mark

···

On May 12, 2004, at 12:23 PM, Mohammad Khan wrote:

35

Financial calculations aren’t done in floating point. There
are various number formats that are used, but they all boil
down to some form of integer arithmetic. For example,
fixed point decimal may be used, where the value $625.91
is represented as the integer 62591 with a scaling factor of
two decimal places. In other words, the arithmetic is
actually done in pennies, not dollars.

Phil Tomson wrote:

This seems to be the 100,000th post to Ruby-talk (at least
the English version).

Cool! And it was my first post to the group.

This Must Mean Something.

What has our player won?

I know! I know! Can I get a free copy of Ruby?

(I was up till 4AM this morning reading the pickaxe book. Had a bad cold,
couldn’t sleep and couldn’t think well enough to write any code, and I’d
been meaning to investigate Ruby a bit. Very interesting! I like dynamic
languages such as Python and JavaScript, and I especially like languages
with expressive power. Ruby looks like it has a lot going for it.)

-Mike

···

Michael Geary Mike@DeleteThis.Geary.com wrote:

36

Zsban Ambrus wrote:

No. I’d rather think this is not the problem. The problem is
that you should not compare floating-point numbers with == or != unless you
have a good reason to do it.

Should ‘ruby -W2’ warn on use of Float#== ?

37

“Sean O’Dell” sean@celsoft.com writes:

I had to massage my floating point math in C/C++ a lot many years ago,
but I always assumed it was a flaw in the MS compiler, and once I got
in the habit of doing it, I forgot about it.

You have a basic misunderstanding of floating point math if you consider
its rounding and truncation characteristics to be a compiler bug.

This is absolutely true; I’ve never taken any computer class, so I completely
missed that basic information.

As other people have mentioned, there is a BigDecimal class in Ruby
that behaves closer to the way that you want.

Now that I know what it is, I’m fine with returning to old behavior and just
massaging my comparisons.

Also, consider this: if you always want the computer to give you exact
results, what should these exact results be for each of the following
arithmetic evaluations? …

a = 1/3

Well, this has an infinite decimal value even in plain math. Everyone should
expect his sort of rounding problem. But the simple arithmetic rounding
error is not something you would anticipate without, say, having a professor
point your nose right at it.

Anyway, thanks for the info everyone,

Sean O'Dell
···

On Wednesday 12 May 2004 10:50, Lloyd Zusman wrote:

38

Well, my HP48 calculator’s processor is able to do math in binary or
decimal. The real numbers are stored in memory in decimal and the
operations by default made in decimal. It doesn’t make my calculations
any more precise, but it made (some of) the rounding errors less
suprising.

But Ruby doesn’t compile on my HP48 :frowning:

Guillaume.

···

On Wed, 2004-05-12 at 15:35, Hal Fulton wrote:

Mohammad Khan wrote:

I guess, you guys are missing the point,

in real life you can’t have exact answer of 1/3 + 2.3 + 1.3
but you can have exact answer of 625.91 + 900.00 + 22.00

Only if you build a computer that works in decimal. In binary,
625.91 is as inexact as 1/3 is in decimal.

39

I will go so far as to have a stern talk with Herbert C. Schildt for not
mentioning the floating point issue in any of the many books I bought which
were written by him, but I refuse to shoot him.

Sean O'Dell
···

On Wednesday 12 May 2004 12:58, John W. Kennedy wrote:

Kindly get in touch with your C/C++ teacher and shoot him.

40

gabriele renzi wrote:

il 12 May 2004 19:00:24 GMT, ptkwt@aracnet.com (Phil Tomson) ha
scritto::

This seems to be the 100,000th post to Ruby-talk (at least the English
version).

damn, I got distracted by my brother or I would have been the one!

Now I have to wait for 1_000_000 to put my name in the history :confused:

It’s ironic that the 100_000th post was in a discussion of decimal
arithmetic.

Shouldn’t we find powers of 2 more significant? The next one is 2**17,
or 131072, so we all have a while to prepare a discussion on some binary
topic like, oh, say, bitwise ops.