I'm having an issue with a float being converted to an int,
The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.
For what ever reason the first instance gives an error, and every
instance there after works as it should
This cant be right?
ruby 1.8.7
[irb]
# This works the way I would think
1.0.o_i
=> 1
# This doesn't
x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
# this is where its werid
n_x.to_i
=> 0
[/irb]
any further 1.0.to_i works
···
--
Posted via http://www.ruby-forum.com/\.
Phil Cooper-king wrote:
any further 1.0.to_i works
You're experiencing IEEE Floating point aritmetic. Look at this example:
···
---------------------------------------
ruby -v: ruby 1.9.1p378 (2010-01-10 revision 26273) [x86_64-linux]
irb(main):001:0> x = -1
=> -1
irb(main):002:0> y = -1
=> -1
irb(main):003:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):004:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):005:0> sprintf("%.50f", n_x)
=> "0.99999999999999988897769753748434595763683319091797"
irb(main):006:0> n_x.to_i
=> 0
irb(main):007:0> n_x.round
=> 1
irb(main):008:0> sprintf("%.50f", 1.0)
=> "1.00000000000000000000000000000000000000000000000000"
irb(main):009:0>
---------------------------------------
Your result isn't that exactly 1 as you think.
Marvin
--
Posted via http://www.ruby-forum.com/\.
For what ever reason the first instance gives an error, and every
instance there after works as it should
What You Think Is Happening Is Not What Is Actually Happening 
ruby 1.8.7
[irb]
# This works the way I would think
1.0.o_i
=> 1
Yep, and every time you say 1.0.to_i you will get 1.
# This doesn't
x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb is showing you an approximation. It's confusing because irb is
showing you n_x to be 1.0, when in fact:
sprintf "%0.25f", n_x
=> "0.9999999999999998889776975"
# this is where its werid
n_x.to_i
=> 0
Yep:
0.9.to_i
=> 0
any further 1.0.to_i works
1.0.to_i and n_x.to_i (from your example) are two different things.
Search the group archives for "floating point", you're sure to find a
huge pile of other people with similar problems, and likely good
solutions.
Ben
···
On Tue, Feb 16, 2010 at 10:16 AM, Phil Cooper-king <phil@cooperking.net> wrote:
You are facing a problem about the inaccuracy inherent to floating
point numbers:
irb(main):001:0> 0.9999.to_i
=> 0
irb(main):002:0> x = -1
=> -1
irb(main):003:0> y = -1
=> -1
irb(main):004:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):005:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):009:0> "%.20f" % n_x
=> "0.99999999999999988898"
irb(main):010:0>
irb(main):010:0> n_x
=> 1.0
n_x is very close to 1.0 but it's not. to_i rounds the number down, so
you get 0. You can check the archives for a more thorough explanation
about this, there's been plenty of discussion in the past.
Jesus.
···
On Tue, Feb 16, 2010 at 7:16 PM, Phil Cooper-king <phil@cooperking.net> wrote:
I'm having an issue with a float being converted to an int,
The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.
For what ever reason the first instance gives an error, and every
instance there after works as it should
This cant be right?
ruby 1.8.7
[irb]
# This works the way I would think
1.0.o_i
=> 1
# This doesn't
x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
# this is where its werid
n_x.to_i
=> 0
[/irb]
any further 1.0.to_i works
thanks guys, this makes it clear
···
--
Posted via http://www.ruby-forum.com/.