I'm reading Java class files using ruby and found some problems when
handling float values. The numbers are stored using IEEE 754 format.
After several hours trying and thinking I might be using unpack the
wrong way, I've wrote the following code which packs a Float and then
immediately unpacks it. And the result is different from the original
value.
···
------------
a = 0.1234567
b = [a].pack('g').unpack('g')[0]
puts "#{a} =? #{b} => #{a == b}"
------------
outputs:
0.1234567 =? 0.123456701636314 => false
I came accross this doc, which states that in Perl this happens too
(http://perldoc.perl.org/functions/pack.html) "Note that Perl uses
doubles internally for all numeric calculation, and converting from
double into float and thence back to double again will lose precision
(i.e., unpack("f", pack("f", $foo)) will not in general equal $foo)."
Maybe Ruby is doing the same Perl does?
Hmm I am not sure that this is exactly related but I would rather find
this behavior normal, given my old memories of floating number
representation, look at this e.g.
On 8/9/07, Luis Parravicini <lparravi@gmail.com> wrote:
Hello
I'm reading Java class files using ruby and found some problems when
handling float values. The numbers are stored using IEEE 754 format.
After several hours trying and thinking I might be using unpack the
wrong way, I've wrote the following code which packs a Float and then
immediately unpacks it. And the result is different from the original
value.
------------
a = 0.1234567
b = [a].pack('g').unpack('g')[0]
--
[...] as simple as possible, but no simpler.
-- Attributed to Albert Einstein
I'm reading Java class files using ruby and found some problems when
handling float values. The numbers are stored using IEEE 754 format.
After several hours trying and thinking I might be using unpack the
wrong way, I've wrote the following code which packs a Float and then
immediately unpacks it. And the result is different from the original
value.
------------
a = 0.1234567
b = [a].pack('g').unpack('g')[0]
puts "#{a} =? #{b} => #{a == b}"
------------
outputs:
0.1234567 =? 0.123456701636314 => false
I came accross this doc, which states that in Perl this happens too
(http://perldoc.perl.org/functions/pack.html\) "Note that Perl uses
doubles internally for all numeric calculation, and converting from
double into float and thence back to double again will lose precision
(i.e., unpack("f", pack("f", $foo)) will not in general equal $foo)."
Maybe Ruby is doing the same Perl does?
Ruby uses doubles internally for floats as well. Loss of precision is
common, but the above link should work well for you.
Any help will be appreciated
Thanks
No problem, glad to (try to) help.
···
On Thursday 09 August 2007 12:06:31 pm Luis Parravicini wrote:
And the result is different from the original
value.
------------
a = 0.1234567
b = [a].pack('g').unpack('g')[0]
puts "#{a} =? #{b} => #{a == b}"
------------
outputs:
0.1234567 =? 0.123456701636314 => false
It's quite common with float or double representation. Because float/
double can not represent ALL value within its range so some loss
precision happens. It may happen right after assignment:
eg: On my machine:
a = a= 100.1001001003333
a
100.100100100333 (the last digit gone?)
···
On 10 Tha ng Tám, 02:06, "Luis Parravicini" <lparr...@gmail.com> wrote: