BigDecimal#sqrt

ruby-talk
1

I'm trying to understand the argument to BigDecimal#sqrt. I figured it was for precision and it does seem to affect it, but I can't understand the exactly relationship. Can anyone tell me what the argument controls and how?

Thanks.

James Edward Gray II

2

I'm trying to understand the argument to BigDecimal#sqrt. I figured it was for precision and it does seem to affect it, but I can't understand the exactly relationship. Can anyone tell me what the argument controls and how?

Let me try asking me question a different way... Is this the correct way to get a printable sqrt() with 16 digits precision?

>> sprintf "%.16f", BigDecimal.new("66543").sqrt(16)
=> "257.9592991151890260"

James Edward Gray II

···

On Dec 22, 2005, at 4:04 PM, James Edward Gray II wrote:

3

I'm trying to understand the argument to BigDecimal#sqrt. I figured it was for precision and it does seem to affect it, but I can't understand the exactly relationship. Can anyone tell me what the argument controls and how?

Let me try asking my question a different way... Is this the correct way to get a printable square root with N digits precision (N = 16 in this example)?

>> sprintf "%.16f", BigDecimal.new("66543").sqrt(16)
=> "257.9592991151890260"

James Edward Gray II

···

On Dec 22, 2005, at 4:04 PM, James Edward Gray II wrote:

4

This is the C source, if you haven't looked at it yet.

BigDecimal_sqrt(VALUE self, VALUE nFig)
{
    ENTER(5);
    Real *c, *a;
    S_INT mx, n;

    GUARD_OBJ(a,GetVpValue(self,1));
    mx = a->Prec *(VpBaseFig() + 1);

    n = GetPositiveInt(nFig) + VpDblFig() + 1;
    if(mx <= n) mx = n;
    GUARD_OBJ(c,VpCreateRbObject(mx, "0"));
    VpSqrt(c, a);
    return ToValue(c);
}

I don't fully understand what's going on with it, because my C is
pretty weak, but hopefully this will provide some insight from
someone.

···

On 12/22/05, James Edward Gray II <james@grayproductions.net> wrote:

I'm trying to understand the argument to BigDecimal#sqrt. I figured
it was for precision and it does seem to affect it, but I can't
understand the exactly relationship. Can anyone tell me what the
argument controls and how?

5

yeah, it looks right.
interesting format variation:

minfieldwidth = 22
precision = 16
sprintf "%*.*f", minfieldwidth, precision, BigDecimal.new("66543").sqrt(precision)
=> " 257.9592991151898786"

James Edward Gray II wrote:

···

On Dec 22, 2005, at 4:04 PM, James Edward Gray II wrote:

I'm trying to understand the argument to BigDecimal#sqrt. I figured it was for precision and it does seem to affect it, but I can't understand the exactly relationship. Can anyone tell me what the argument controls and how?

Let me try asking my question a different way... Is this the correct way to get a printable square root with N digits precision (N = 16 in this example)?

>> sprintf "%.16f", BigDecimal.new("66543").sqrt(16)
=> "257.9592991151890260"

James Edward Gray II

6

James Edward Gray II wrote:

Let me try asking me question a different way... Is this the correct way to get a printable sqrt() with 16 digits precision?

>> sprintf "%.16f", BigDecimal.new("66543").sqrt(16)
=> "257.9592991151890260"

Yes.

I've written some documentation for BigDecimal which will hopefully be checked in to Ruby some time.

mathew

···

--
      <URL:http://www.pobox.com/~meta/&gt;
My parents went to the lost kingdom of Hyrule
     and all I got was this lousy triforce.

7

Is it available online anywhere before then or could I trouble you to send it to me privately?

James Edward Gray II

···

On Dec 27, 2005, at 10:27 PM, mathew wrote:

James Edward Gray II wrote:

Let me try asking me question a different way... Is this the correct way to get a printable sqrt() with 16 digits precision?
>> sprintf "%.16f", BigDecimal.new("66543").sqrt(16)
=> "257.9592991151890260"

Yes.

I've written some documentation for BigDecimal which will hopefully be checked in to Ruby some time.