Array#size empties the Array?

ruby-talk
1

Hello,

I've encountered the following strang behaviour of Array#size.
The example is in a context of ActiveRecord, so I don't know
if it is an ActiveRecord or Ruby issue (or something even more
bizarre...)

I have taken it out of context, but: Can you think of _ANY_ context,
where the following behaviour makes any sense whatsoever or is
at least explainable?

First, consider this code piece:

    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    p [:rks_class, rks.class]
    p [:rks_item, rks[0]]
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]

It produces this output:

[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]

Now, imagine the above code piece is replaced
by the following, everything else remaining _exacly_ the same:

    p [:rks_size, rks.size]
    p [:rks_length, rks.length]
    p [:rks_class, rks.class]
    p [:rks_item, rks[0]]
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]

The output will then be:

[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]

And this is not a singular incidence, but it occurs repeatedly
always in the same way.
I have also tried variations on this, and observed the following:
Every time when #size was the first method
that was sent to the object "rks",
the object would be empty.
For example, sending #class first and then #size
would already save rks from destruction.

I've tried to read if there was a difference between Array#size
and Array#length, but I only found that #size is an alias for #length.
Then I looked if it might be due to some redefinition in ActiveSupport
but I have not found anything there.

The context of the example also includes Apollo and Test::Unit,
maybe they do something strange to Array#size??

At this point I wish to have some way of telling where (in which file)
a particular method was defined - is there already a means of getting
this information?

Any enlightenment appreciated

Sven

···

--
Posted via http://www.ruby-forum.com/.

2

Sorry, I had forgotten to give enough information:

1. rks is a local variable!

The statement sequence is actually:

    rks = vt.rks
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    ...

And this is inside a Test::Unit class, where I have not defined a
rks= writer method.

2. Versions

Ruby version is 1.8.4 on WindowsXP (one-click-installer).
ActiveRecord 1.14.1
Apollo 0.841a_vcl60

Regards

Sven

···

--
Posted via http://www.ruby-forum.com/.

3

Sven
try not to replace the code, which is equivalent, and run your test twice
with the same code.
Cheers
Robert

···

On 8/3/06, Sven Suska <software617rf@suska.org> wrote:

Sorry, I had forgotten to give enough information:

1. rks is a local variable!

The statement sequence is actually:

    rks = vt.rks
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    ...

And this is inside a Test::Unit class, where I have not defined a
rks= writer method.

2. Versions

Ruby version is 1.8.4 on WindowsXP (one-click-installer).
ActiveRecord 1.14.1
Apollo 0.841a_vcl60

Regards

Sven

--
Posted via http://www.ruby-forum.com/\.

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

- Albert Einstein

4

Hi,

···

In message "Re: Array#size empties the Array??" on Thu, 3 Aug 2006 18:04:41 +0900, Sven Suska <software617rf@suska.org> writes:

Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce the
problem? I am sure your expectation of "exactly same" fails
somewhere.

              matz.

5

So vt is an ActiveRecord object, and vt.rks refers to an ActiveRecord association?

If this is the case, rks is not an Array as you claim; it's an ActiveRecord association object which is a proxy to the data.

Pete Yandell

···

On 03/08/2006, at 7:04 PM, Sven Suska wrote:

Sorry, I had forgotten to give enough information:

1. rks is a local variable!

The statement sequence is actually:

    rks = vt.rks
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    ...

And this is inside a Test::Unit class, where I have not defined a
rks= writer method.

6

Robert Dober wrote:

try not to replace the code, which is equivalent, and run your test
twice
with the same code.
Cheers
Robert

Dear Robert,

thank you for your quick reply.

Although...,
being so quick may mean:
perhaps too little time to get the point
I was trying to make? :wink:

You say the code _is_ equivalent.
And I say the code _should_be_ equivalent.
I definitively found out, that it does not behave "equivalently".

You are suggesting to run my test twice.
What do you mean by "twice"?
Of course, I had already run it plenty of times ...,
But anyway, now I have done the following:

I copied-and-pasted the whole test-method
that contained the mentioned code lines
inside my TestCase class, like this:

class WierdSizeTest < Test::Unit::TestCase
  def test1
    ...
    rks = vt.rks
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    p [:rks_class, rks.class]
    p [:rks_item, rks[0]]
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    ...
  end

  def test2
    ...
    rks = vt.rks
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    p [:rks_class, rks.class]
    p [:rks_item, rks[0]]
    p [:rks_length, rks.length]
    p [:rks_size, rks.size]
    ...
  end

And I also repeated the test with the "equivalent" code exchanged,
ie swapped method calls of #length and #size.

And what I found was this:

  The only factor that influenced the output was
  _the_order_of_the_method_calls_.

  Or, to put it directly:
  Whenever rks.size was the _first_ method call on "rks",
  the output was:
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]
   And the output was normal otherwise. (as posted in the beginning)

   Regardless of how many times a test method was repeated.
   Regardless of the order of test methods. (When one method
   contained the "corrupted" variant and the other did not.)

This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

Any more suggestions?

Cheers

Sven

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

Well, it seems so ... - but how does this affect this particular
case?...
:slight_smile:

···

--
Posted via http://www.ruby-forum.com/\.

7

Yukihiro Matsumoto wrote:

Sven Suska writes:

>Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce the
problem?

Yes I know it would be useful, but it's hard to find and extract the
relevant
parts.

I am sure your expectation of "exactly same" fails
somewhere.

Well, sorry to say that, but I think your assumption is not correct.

Let's see if my new findings give anybody a clue.

At first I tried a different ActiveRecord version - no difference.

Then I put some debug printouts into AR and I found clearly different
behaviour.
I'll just give the raw output,
even if I don't tell you what the printout statements were,
you can see the difference clearly:

1. #length first:

···

================

** RUBY_VERSION = "1.8.4"
** "ar-svs: args=[:all]"
** [:SVSAR_find_sql, "SELECT * FROM rkScGla WHERE (rkScGla.vsnr =
'GK_715') AND (rkScGla.komp = 'R1') AND (rkScGla.vtnr = 1) "]
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]

2. #size first:

** RUBY_VERSION = "1.8.4"
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** [:SVSAR_sel_1_sql, "SELECT count(*) AS count_all FROM rkScGla WHERE
(rkScGla.vsnr = 'GK_715' AND rkScGla.komp = 'R1' AND rkScGla.vtnr = 1) "
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]

---> This looks like a very exciting ActiveRecord hack!!!

It looks like rks is something special in the beginning,
that will be replaced by the real array as soon as methods are
being sent to it.

Well, whatever the explanation may be, it has become clear now,
that this behaviour is caused by ActiveRecord, and so I should
repost the issue in the rails forum.

And now I understand why this issue has not bothered anybody before,
it has only come up because my hand-written database-adapter is not
yet able to execute the "SELECT count(*) ..." statement correctly.

OK... Exciting day, today...

Thank you all for your support

Regards

Sven

--
Posted via http://www.ruby-forum.com/\.

8

Hello to all who have replied to my question,

Thank you a lot for your help!

The issue is now sufficiently cleared up for me.
Althogh I'd like to do some more research into the topic,
I've got other things to get finished first.

Robert Dober wrote:

Hmm I think it is better to give hints instead of saying where the
problem lies.
Well I do not know where the problem lies, but what I meant was the
following [...]

Well yes, in the beginning most people searched the problem
somewhere in the outside environment (even Matz did!).
But then it became evident that the problem was really inside
the method calls.

I think the most misleading information was this debug statement:
    p [:rks_class, rks.class]
producing this output:
[:rks_class, Array]

And still, rks was not an Array!

As Pete later explained:

Pete Yandell wrote:

So vt is an ActiveRecord object, and vt.rks refers to an ActiveRecord
association?

If this is the case, rks is not an Array as you claim; it's an
ActiveRecord association object which is a proxy to the data.

And Daniel had summed up the whole thing very clearly:

Daniel DeLorme wrote:

Sven Suska wrote:

[...] what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
   (class << rks;self;end).ancestors
and you'll probably find this:
   [ActiveRecord::Associations::HasManyAssociation,
ActiveRecord::Associations::AssociationCollection,
ActiveRecord::Associations::AssociationProxy, Object,
Base64::Deprecated,
Base64, Kernel]

Thank you for this idea! I was wondering if there was any means
to get behind this masquerade of the ActiveRecord proxy objects.

ActiveRecord associations forward most messages (including 'class') to
the
underlying array structure, but some messages (like 'size') are part of
the
association class itself.

I think what's happening is that by calling 'size' first, the objects
have not
yet been fetched from the database so instead it uses a COUNT(*) query
to find
the size. When calling 'length' (or any other method) first, the objects
are
fetched from the database and placed in an array and the 'length'
message is
forwarded to that array. Then, when 'size' is invoked again, it notices
that the
array is already present and uses its size instead of wasting time doing
another
DB query. Normally the results should be the same but in your case
something
weird is happening. I don't know what exactly but this should give you
some
avenues to investigate.

Yes, the main thing is very clear now, thank you.
(As I said, my db-interface is unable to do a proper "count".)

Having gone through this "ordeal", I'm arriving at this question:
Is it a good idea that the ActiveRecord proxy objects return Array
as the class of the proxy?
Or, even more: is this somehow "forced" by other design considerations?

For example, if there were "protocols" in ruby, people were less
often tempted to write
  obj.is_a?(Array)
And this is probably the reason, why th AR designers have made
the proxy objects say their class is Array. (And they indeed behave
like Arrays -- provided the db-interface passes all the unit tests.)

Anyway, I don't have the time now to go into a new thread.

I wish you all the best

Sven

···

--
Posted via http://www.ruby-forum.com/\.

9

Sven Suska wrote:
<snip>

  Or, to put it directly:
  Whenever rks.size was the _first_ method call on "rks",
  the output was:
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]
   And the output was normal otherwise. (as posted in the beginning)

   Regardless of how many times a test method was repeated.
   Regardless of the order of test methods. (When one method
   contained the "corrupted" variant and the other did not.)

This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

This may be irrelevant, but what's rks.class.ancestors?

···

--
Alex

10

Sven Suska wrote:

when called as the first method after the object obtained.

Sorry, I ment:

when called as the first method after the object is obtained.

I didn't say "created" because it is created somewhere inside
ActiveRecord,
so I just wanted to say "obtained from ActiveRecord".

Sven

···

--
Posted via http://www.ruby-forum.com/\.

11

Robert Dober wrote:
> try not to replace the code, which is equivalent, and run your test

[snip]

Well, it seems so ... - but how does this affect this particular
case?...
:slight_smile:

I think you should post (or send a link to) the full test case in
quesiton, including the source you have for whatever model makes up
'rks'.

- Rob

···

On 8/3/06, Sven Suska <software617rf@suska.org> wrote:
--

12

Yukihiro Matsumoto wrote:

Sven Suska writes:

>Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce the
problem?

Yes I know it would be useful, but it's hard to find and extract the
relevant
parts.

I am sure your expectation of "exactly same" fails
somewhere.

Well, sorry to say that, but I think your assumption is not correct.

Let's see if my new findings give anybody a clue.

At first I tried a different ActiveRecord version - no difference.

Then I put some debug printouts into AR and I found clearly different
behaviour.
I'll just give the raw output,
even if I don't tell you what the printout statements were,
you can see the difference clearly:

1. #length first:

** RUBY_VERSION = "1.8.4"
** "ar-svs: args=[:all]"
** [:SVSAR_find_sql, "SELECT * FROM rkScGla WHERE (rkScGla.vsnr =
'GK_715') AND (rkScGla.komp = 'R1') AND (rkScGla.vtnr = 1) "]
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]

2. #size first:

** RUBY_VERSION = "1.8.4"
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** [:SVSAR_sel_1_sql, "SELECT count(*) AS count_all FROM rkScGla WHERE
(rkScGla.vsnr = 'GK_715' AND rkScGla.komp = 'R1' AND rkScGla.vtnr = 1) "
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]

---> This looks like a very exciting ActiveRecord hack!!!

It looks like rks is something special in the beginning,
that will be replaced by the real array as soon as methods are
being sent to it.

Well, whatever the explanation may be, it has become clear now,
that this behaviour is caused by ActiveRecord, and so I should
repost the issue in the rails forum.

And now I understand why this issue has not bothered anybody before,
it has only come up because my hand-written database-adapter is not
yet able to execute the "SELECT count(*) ..." statement correctly.

OK... Exciting day, today...

Thank you all for your support

Regards

Sven

AR does all sorts of magic. It has been my experience that whenever something doesn't work as it should or as I expect and I am making use of AR, AR is probably the culprit.

···

On Aug 3, 2006, at 12:00 PM, Sven Suska wrote:

--
Posted via http://www.ruby-forum.com/\.

13

Sven Suska schrieb:

This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

Any more suggestions?

Sven, you could try to find the implementation of the size method:

   p rks.method(:size)

From the output you should be able to find the class/module which implements the size method.

Regards,
Pit

14

Robert Dober wrote:
> try not to replace the code, which is equivalent, and run your test
> twice
> with the same code.
> Cheers
> Robert

Dear Robert,

thank you for your quick reply.

Although...,
being so quick may mean:
perhaps too little time to get the point
I was trying to make? :wink:

I got 15 minutes (and used 2 to be honest).

You say the code _is_ equivalent.

And I say the code _should_be_ equivalent.
I definitively found out, that it does not behave "equivalently".

You are suggesting to run my test twice.
What do you mean by "twice"?

Hmm I think it is better to give hints instead of saying where the problem
lies.
Well I do not know where the problem lies, but what I meant was the
following:

code A
code B

equivilant, we think
so we rite

def test1
   code A
    code A
end

def test2
   code A
  code B
end
...
  code B
code B
end

I thaught it was a possibility to learn more about setup and teardown in
your test classes.
I very strongly believe that the problem is there but I wanted you to find
out yourself.
You take the credit, nobody takes the blame, see my strategy :wink:

Well now I have said it and I will take the blame if wrong.
Believe me I have considered your post seriously and I could have said the
same thing
as all others ***more information*** and they are right to do so, but that's
not my way....

Cheers
Robert

···

On 8/3/06, Sven Suska <software617rf@suska.org> wrote:

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

- Albert Einstein

15

Alex Young wrote:

Sven Suska wrote:

This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

Sven

···

--
Posted via http://www.ruby-forum.com/\.

16

Rob Sanheim wrote:

I think you should post (or send a link to) the full test case in
quesiton, including the source you have for whatever model makes up
'rks'.

Well, it's so big...

I'll try something else first.

Sven

···

--
Posted via http://www.ruby-forum.com/\.

17

A thought I just had:

The strange behaviour of my rks object reminds me of quantum mechanics:
rks is in a "quantum state" - As soon as you try to observe it,
you force it out of it's quantum state into some observable state.
And that depends on the way you have observed it.

Logan Capaldo wrote:

AR does all sorts of magic. [...]

Cheers

Sven

···

--
Posted via http://www.ruby-forum.com/\.

18

Sven Suska wrote:

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
   (class << rks;self;end).ancestors
and you'll probably find this:
   [ActiveRecord::Associations::HasManyAssociation, ActiveRecord::Associations::AssociationCollection, ActiveRecord::Associations::AssociationProxy, Object, Base64::Deprecated, Base64, Kernel]

ActiveRecord associations forward most messages (including 'class') to the underlying array structure, but some messages (like 'size') are part of the association class itself.

I think what's happening is that by calling 'size' first, the objects have not yet been fetched from the database so instead it uses a COUNT(*) query to find the size. When calling 'length' (or any other method) first, the objects are fetched from the database and placed in an array and the 'length' message is forwarded to that array. Then, when 'size' is invoked again, it notices that the array is already present and uses its size instead of wasting time doing another DB query. Normally the results should be the same but in your case something weird is happening. I don't know what exactly but this should give you some avenues to investigate.

Daniel

19

Daniel DeLorme wrote:

Sven Suska wrote:

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
  (class << rks;self;end).ancestors
and you'll probably find this:
  [ActiveRecord::Associations::HasManyAssociation, ActiveRecord::Associations::AssociationCollection, ActiveRecord::Associations::AssociationProxy, Object, Base64::Deprecated, Base64, Kernel]

Ooh, good catch.

···

--
Alex