Floating point conversion error?

ruby-talk
1

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

···

--
Posted via http://www.ruby-forum.com/\.

2

Brandon Hoult wrote:

>> (2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn't entirely accurate.

"%.30f" % (2241.18 * 100)

=> "224117.999999999970896169543266296387"

And since 224117.999999999970896169543266296387.to_i is 224117, that's what
you get.

HTH,
Sebastian

···

--
NP: Immortal - Withstand The Fall Of Time
Jabber: sepp2k@jabber.org
ICQ: 205544826

3

Hello!

···

On Jan 23, 2008 4:25 PM, Brandon Hoult <bhoult@gmail.com> wrote:

>> (2241.18 * 100).to_i
=> 224117

Anyone know what is up with that?

Floating point numbers are never 100% accurate. See
Floating-point arithmetic - Wikipedia for a detailed
explanation.

4

What Every Computer Scientist Should Know About Floating-Point Arithmetic

<http://docs.sun.com/source/806-3568/ncg_goldberg.html&gt;

···

On Jan 23, 2008, at 10:25 AM, Brandon Hoult wrote:

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

5

Sebastian Hungerecker wrote:

Brandon Hoult wrote:

>> (2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn't entirely accurate.

"%.30f" % (2241.18 * 100)

=> "224117.999999999970896169543266296387"

And since 224117.999999999970896169543266296387.to_i is 224117, that's
what
you get.

HTH,
Sebastian

Thanks... found this after poking around... guess I should have spent
more time looking.

···

--
Posted via http://www.ruby-forum.com/\.