This is nice:
[65] pry(main)> b.power(4.0001,5).to_s("F")
=> "1.1452"
But this one yields gajilion digits, despite the low precision
parameter:
[64] pry(main)> b.power(4.0,5).to_s("F")
=>
"1.145230820564952260841164968378904565512151464552278373093579496692262339862090976570839595577995167
408710258016262277652022322103944542550980444830272550796764510852122423760853182822668783694175299491
998229840756361333048675137604431608257538375836478005205851705333523733848357300705094580513261348742578974976"
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don't need to use #round?
Checking the C source (I'm not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don't know the precision) if the value is X.0
I would not worry about the number of digits in the BigDecimal instance.
Internally you should use most precise figures. If you want to do output
then you can apply proper formatting, e.g. with printf.
Cheers
robert
···
On Thu, Aug 22, 2013 at 10:55 PM, Földes László <lists@ruby-forum.com>wrote:
I'm experimenting with arbitrary precision numbers but I ran into a
precision problem with #power method.
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don't need to use #round?
Checking the C source (I'm not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don't know the precision) if the value is X.0
In the documentation to 'BigDecimal#power', the optional `prec' argument
is not even mentioned.
You can reduce the number of significant digits by class method limit
b=BigDecimal.new("1.034482758620689655172413793103448275862068965517241379310344827586206896551724")
BigDecimal::limit(5)
b.power(4, 123456789) # "1.1453", prec argument is ignored
It seems there is a bug in this method implementation. Too busy to
investigate myself.
Using #round solves the problem, but there is clearly a problem. Any
idea how to properly parameter the method so I don't need to use #round?
Checking the C source (I'm not a C ninja) it looks that a FLOAT value
gets converted if it equals with its rounded self which is certainly
true (although I don't know the precision) if the value is X.0
I would not worry about the number of digits in the BigDecimal instance.
Internally you should use most precise figures. If you want to do
output
then you can apply proper formatting, e.g. with printf.
Cheers
robert
My problem is that if calculation continues with so many digits, it will
be horribly slow. If I repeat the #power method with the long result,
after ten iteration it yields 5.000 digits, I assume a few ten thousand
iteration will result in an OutOfMemory error.
···
On Thu, Aug 22, 2013 at 10:55 PM, Fldes Lszl > <lists@ruby-forum.com>wrote:
In the documentation to 'BigDecimal#power', the optional `prec' argument
is not even mentioned.
You can reduce the number of significant digits by class method limit:
It even states that "Note that n must be an Integer." which is not true,
it can be Float, Rational, Bigdecimal. The C source might tell you more,
there are a bunch of type checks in there.
Then there's probably no way around invoking #round when appropriate.
Btw., what kind of calculations do you do?
Cheers
robert
···
On Fri, Aug 23, 2013 at 12:24 PM, Földes László <lists@ruby-forum.com>wrote:
My problem is that if calculation continues with so many digits, it will
be horribly slow. If I repeat the #power method with the long result,
after ten iteration it yields 5.000 digits, I assume a few ten thousand
iteration will result in an OutOfMemory error.