Pop/push, shift/unshift

ruby-talk
41

Bob, I can only guess that the grief you are talking about has
something to do with memory management, but I don't see it.

Can someone explain -- with the code above -- why tmp does in fact
_not_ take on NilClass? I think it would be good info for newbies.

Todd

···

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 24-Oct-07, at 9:03 AM, Todd Benson wrote:

> def dequeue
> @list.shift
> end

Save yourself some grief, do it this way:

def dequeue
   tmp = @list[0]
   @list[0] = nil
   @list.shift
   tmp
end

Cheers,
Bob

42

Okay, I guess I can answer my own question. Binding is resolved when
an operator like '=' is interpreted, especially with immutable objects
(like Fixnum).

a = 1,2,3
puts a[0].__id__
#output => 3
b = a
puts b.__id__
#output => 3
a[0] = 7
puts a[0].__id__
#output => 15
puts b.__id__
#output => 3

For the newbies out there, keep this in mind. The name b put on
a[0]'s shoes and kept them on. Somebody correct me if I'm wrong.

Todd

···

On 10/24/07, Todd Benson <caduceass@gmail.com> wrote:

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:
>
> On 24-Oct-07, at 9:03 AM, Todd Benson wrote:
>
> > def dequeue
> > @list.shift
> > end
>
> Save yourself some grief, do it this way:
>
> def dequeue
> tmp = @list[0]
> @list[0] = nil
> @list.shift
> tmp
> end
>
> Cheers,
> Bob

Bob, I can only guess that the grief you are talking about has
something to do with memory management, but I don't see it.

Can someone explain -- with the code above -- why tmp does in fact
_not_ take on NilClass? I think it would be good info for newbies.

Todd

43

  def dequeue
    @list.shift
  end

Save yourself some grief, do it this way:

def dequeue
   tmp = @list[0]
   @list[0] = nil
   @list.shift
   tmp
end

Cheers,
Bob

Bob, I can only guess that the grief you are talking about has
something to do with memory management, but I don't see it.

It has to do with an optimisation in the implementation of shift (there's a series of posts regarding this earlier in this thread). In brief, shift optimises itself by 'shifting' the start of the array rather than shifting the contents of the array. The effect is that using shift leaves an old part of the array in front of the start of the array (but you cannot access it). The problem is that the shift implementation doesn't 'clear' the original 0th element, so it is still in memory, is not accessible, but the garbage collector can see it. In effect, the object referenced isn't collected because there is a reference. This is a memory leak. Sooner or later it'll get fixed up, but sometimes it really is later.

Can someone explain -- with the code above -- why tmp does in fact
_not_ take on NilClass? I think it would be good info for newbies.

In the above code, the shift method, through an involved but hidden technique implemented by Ruby, arranges for the 0th element of the list to 'disappear', for the original @list[1] to become @list[0] and so on, for the array size to be reduced by one, and returns the original @list[0].

So, tmp 'remembers' the original @list[0]. The '@list[0] = nil' removes the reference to what is now in tmp, and @list.shift makes the original @list[0] 'disappear', and the last reference to tmp returns it as the result of the method.

Unfortunately it isn't as simple as everyone would like it to be. But, as I've mentioned elsewhere, this will all go away in some future version of Ruby. Already Ruby 1.8.5 is much better than 1.8.4

Cheers,
Bob

···

On 24-Oct-07, at 9:54 AM, Todd Benson wrote:

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 24-Oct-07, at 9:03 AM, Todd Benson wrote:

Todd

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

44

<snip

>
> Can someone explain -- with the code above -- why tmp does in fact
> _not_ take on NilClass? I think it would be good info for newbies.
>

In the above code, the shift method, through an involved but hidden
technique implemented by Ruby, arranges for the 0th element of the
list to 'disappear', for the original @list[1] to become @list[0] and
so on, for the array size to be reduced by one, and returns the
original @list[0].

So, tmp 'remembers' the original @list[0]. The '@list[0] = nil'
removes the reference to what is now in tmp,

I believe that this is not true, imagine the GC freeing all objects
that are nil!
Seems very strange to me.

R.

···

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

45

Hi --

  def dequeue
    @list.shift
  end

Save yourself some grief, do it this way:

def dequeue
   tmp = @list[0]
   @list[0] = nil
   @list.shift
   tmp
end

Cheers,
Bob

Bob, I can only guess that the grief you are talking about has
something to do with memory management, but I don't see it.

Can someone explain -- with the code above -- why tmp does in fact
_not_ take on NilClass? I think it would be good info for newbies.

Todd

Okay, I guess I can answer my own question. Binding is resolved when
an operator like '=' is interpreted, especially with immutable objects
(like Fixnum).

a = 1,2,3
puts a[0].__id__
#output => 3
b = a
puts b.__id__
#output => 3
a[0] = 7
puts a[0].__id__
#output => 15
puts b.__id__
#output => 3

For the newbies out there, keep this in mind. The name b put on
a[0]'s shoes and kept them on. Somebody correct me if I'm wrong.

I'm not seeing what you're seeing:

irb(main):014:0> a = 1,2,3
=> [1, 2, 3]
irb(main):015:0> a[0].__id__
=> 3
irb(main):016:0> b = a
=> [1, 2, 3]
irb(main):017:0> b.__id__
=> 1669980 # Are you really seeing 3 here?
irb(main):018:0> a[0] = 7
=> 7
irb(main):019:0> a[0].__id__
=> 15
irb(main):020:0> b.__id__
=> 1669980 # And here?

When you do b = a, you're binding b to the same object as a. I don't
know why b would think you had bound it to a[0].

David

···

On Thu, 25 Oct 2007, Todd Benson wrote:

On 10/24/07, Todd Benson <caduceass@gmail.com> wrote:

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 24-Oct-07, at 9:03 AM, Todd Benson wrote:

--
Upcoming training by David A. Black/Ruby Power and Light, LLC:
   * Advancing With Rails, Edison, NJ, November 6-9
   * Advancing With Rails, Berlin, Germany, November 19-22
   * Intro to Rails, London, UK, December 3-6 (by Skills Matter)
See http://www.rubypal.com for details!

46

Sorry for replying to my own post, can somebody confirm please that
the following code proves that Array#shift does not leak?
Thx in advance
def count
  i = 0
  ObjectSpace.each_object{ i += 1 }
  i
end

a = []
10.times do
  10_000.times do
    a << "a"
    a.shift
  end
  ObjectSpace.garbage_collect
  p count
end
381
382
382
382
382
382
382
382
382
382

Robert

47

You're right David. That was supposed to be b = a[0] and _not_ b = a.

Todd

···

On 10/24/07, David A. Black <dblack@rubypal.com> wrote:

Hi --

On Thu, 25 Oct 2007, Todd Benson wrote:

> On 10/24/07, Todd Benson <caduceass@gmail.com> wrote:
> Okay, I guess I can answer my own question. Binding is resolved when
> an operator like '=' is interpreted, especially with immutable objects
> (like Fixnum).
>
> a = 1,2,3
> puts a[0].__id__
> #output => 3
> b = a
> puts b.__id__
> #output => 3
> a[0] = 7
> puts a[0].__id__
> #output => 15
> puts b.__id__
> #output => 3
>
> For the newbies out there, keep this in mind. The name b put on
> a[0]'s shoes and kept them on. Somebody correct me if I'm wrong.

I'm not seeing what you're seeing:

irb(main):014:0> a = 1,2,3
=> [1, 2, 3]
irb(main):015:0> a[0].__id__
=> 3
irb(main):016:0> b = a
=> [1, 2, 3]
irb(main):017:0> b.__id__
=> 1669980 # Are you really seeing 3 here?
irb(main):018:0> a[0] = 7
=> 7
irb(main):019:0> a[0].__id__
=> 15
irb(main):020:0> b.__id__
=> 1669980 # And here?

When you do b = a, you're binding b to the same object as a. I don't
know why b would think you had bound it to a[0].

David

48

Hi,

Sorry for replying to my own post, can somebody confirm please that
the following code proves that Array#shift does not leak?
Thx in advance
def count
  i = 0
  ObjectSpace.each_object{ i += 1 }
  i
end

a =
10.times do
  10_000.times do
    a << "a"
    a.shift
  end
  ObjectSpace.garbage_collect
  p count
end
381
382

Robert

Try it this way:

class Thing
end

def count
   i = 0
   ObjectSpace.each_object(Thing){ i += 1 }
   i
end

a =
10.times do
   10_000.times do
     a << Thing.new
   end
   ObjectSpace.garbage_collect
   10_000.times do
     a.shift
   end
   ObjectSpace.garbage_collect
   puts "count(1): #{count} size: #{a.size}"
   a.push(nil).pop
   ObjectSpace.garbage_collect
   puts "count(2): #{count} size: #{a.size}"

   10_000.times do
     a << Thing.new
   end
   ObjectSpace.garbage_collect
   10_000.times do
     a[0] = nil # <<<<<< !!!!
     a.shift
   end
   ObjectSpace.garbage_collect
   puts "count(3): #{count} size: #{a.size}"
end

count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0

···

On 24-Oct-07, at 3:09 PM, Robert Dober wrote:

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

49

Amazing Bob,
it has nothing to do with nil though, but with the assignment to a[0]
change a[0] = nil to a[0]=42, you get the same results.
You can even do the following

def count
  i = 0
  ObjectSpace.each_object{ i += 1 }
  i
end

a =
  10_000.times do
    a << Object.new
  end
  ObjectSpace.garbage_collect
  10_000.times do
    a.shift
  end
  ObjectSpace.garbage_collect
  puts "count(1): #{count} size: #{a.size}"
  a.push(nil).pop
  ObjectSpace.garbage_collect
  puts "count(2): #{count} size: #{a.size}"

  10_000.times do
    a << Object.new
  end
  ObjectSpace.garbage_collect
  10_000.times do
  a[0] = Object.new
    a.shift
  end
  ObjectSpace.garbage_collect
  puts "count(3): #{count} size: #{a.size}"

count(1): 10386 size: 0
count(2): 389 size: 0
count(3): 392 size: 0

There remains one thing to be clarified, is this really a bug?
I do not think so, I just ran this until I got 200 points on my terminal :-0

a =
loop do
a << Array.new( 1_000_000){ Object.new }
a.shift
$stdout.print "."
$stdout.flush
end

on my Zenwalk box memory is allocated and released regualrely, I doubt
if ObjectSpace is the right tool to use.

Anyway forgive me to be blunt but everybody should just use Array#shift.

Cheers
Robert

···

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

Hi,

On 24-Oct-07, at 3:09 PM, Robert Dober wrote:

> Sorry for replying to my own post, can somebody confirm please that
> the following code proves that Array#shift does not leak?
> Thx in advance
> def count
> i = 0
> ObjectSpace.each_object{ i += 1 }
> i
> end
>
> a =
> 10.times do
> 10_000.times do
> a << "a"
> a.shift
> end
> ObjectSpace.garbage_collect
> p count
> end
> 381
> 382
> 382
> 382
> 382
> 382
> 382
> 382
> 382
> 382
>
> Robert
>

Try it this way:

class Thing
end

def count
   i = 0
   ObjectSpace.each_object(Thing){ i += 1 }
   i
end

a =
10.times do
   10_000.times do
     a << Thing.new
   end
   ObjectSpace.garbage_collect
   10_000.times do
     a.shift
   end
   ObjectSpace.garbage_collect
   puts "count(1): #{count} size: #{a.size}"
   a.push(nil).pop
   ObjectSpace.garbage_collect
   puts "count(2): #{count} size: #{a.size}"

   10_000.times do
     a << Thing.new
   end
   ObjectSpace.garbage_collect
   10_000.times do
     a[0] = nil # <<<<<< !!!!
     a.shift
   end
   ObjectSpace.garbage_collect
   puts "count(3): #{count} size: #{a.size}"
end

count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0
count(1): 10000 size: 0
count(2): 0 size: 0
count(3): 0 size: 0

----
Bob Hutchison -- tumblelog at http://
www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/
hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/
cms-for-static-content/home/

50

The code in Ruby that deals with the stuff before the start of the array is non-trivial. The push(nil).pop thing I did, or the use of << you did triggers a cleanup. If you do your latest example without interleaving the << and shift you'll have a problem (change the implementation of the Thing class in my example to allocate the million element array and see what happens -- well DON'T actually, you won't like what it does to your machine).

The trouble here is that the normal use of queues involves filling them up then emptying them. If you use an Array and Array.shift to empty the queue there's going to be at least one element 'stuck' in front of the array, likely more than 1. This might not matter but it might. It mattered to my cache implementation, and it mattered to Mongrel's task management.

Cheers,
Bob

···

On 24-Oct-07, at 5:03 PM, Robert Dober wrote:

There remains one thing to be clarified, is this really a bug?
I do not think so, I just ran this until I got 200 points on my terminal :-0

a =
loop do
a << Array.new( 1_000_000){ Object.new }
a.shift
$stdout.print "."
$stdout.flush
end

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

51

> There remains one thing to be clarified, is this really a bug?
> I do not think so, I just ran this until I got 200 points on my
> terminal :-0
>
> a =
> loop do
> a << Array.new( 1_000_000){ Object.new }
> a.shift
> $stdout.print "."
> $stdout.flush
> end

The code in Ruby that deals with the stuff before the start of the
array is non-trivial. The push(nil).pop thing I did, or the use of <<
you did triggers a cleanup. If you do your latest example without
interleaving the << and shift you'll have a problem (change the
implementation of the Thing class in my example to allocate the
million element array and see what happens -- well DON'T actually,
you won't like what it does to your machine).

The trouble here is that the normal use of queues involves filling
them up then emptying them. If you use an Array and Array.shift to
empty the queue there's going to be at least one element 'stuck' in
front of the array, likely more than 1. This might not matter but it
might. It mattered to my cache implementation, and it mattered to
Mongrel's task management.

Well I guess I have to believe you, but I would *love* to reproduce it
if I do
times do
   <<
end
times do
  shift
end
it will leak?

R.

···

On 10/24/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 24-Oct-07, at 5:03 PM, Robert Dober wrote:

Cheers,
Bob

----
Bob Hutchison -- tumblelog at http://
www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/
hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/
cms-for-static-content/home/

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

52

Hi,

Well I guess I have to believe you, but I would *love* to reproduce it
if I do
times do
   <<
end
times do
  shift
end
it will leak?

How else would you describe the 10k Things left in memory after the shifts in the modified version of your test case?

Cheers,
Bob

···

On 25-Oct-07, at 1:12 AM, Robert Dober wrote:

R.

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

53

But that is assuming that the ObjectSpace tools tell the truth, it
does however not look like that on my box.
I have therefore no reason to believe that the above structure of the
code will leak, and have asked you if it will.
I will kill my box tonight :wink: (or not).
R.

···

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

Hi,

On 25-Oct-07, at 1:12 AM, Robert Dober wrote:

> Well I guess I have to believe you, but I would *love* to reproduce it
> if I do
> times do
> <<
> end
> times do
> shift
> end
> it will leak?

How else would you describe the 10k Things left in memory after the
shifts in the modified version of your test case?

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

54

The problem is that Ruby's COW (copy-on-write) scheme is messed up for
arrays. I've found much worse cases performance and memory-wise. Try
doing a small #slice on a big array a bunch and modify the small
slices. I made a 1.9 patch a couple years ago that fixed the problem
and made ALL operations near the front just as fast as the operations
on the back. Unfortunately, it never made it in. The way I did it
might be unique, not sure. I thought C++ deque does something
similiar, but it doesn't. When I find time, I'll try to do the fix
again.

···

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

Hi,

On 25-Oct-07, at 1:12 AM, Robert Dober wrote:

> Well I guess I have to believe you, but I would *love* to reproduce it
> if I do
> times do
> <<
> end
> times do
> shift
> end
> it will leak?

How else would you describe the 10k Things left in memory after the
shifts in the modified version of your test case?

55

Hi,

Well I guess I have to believe you, but I would *love* to reproduce it
if I do
times do
   <<
end
times do
  shift
end
it will leak?

How else would you describe the 10k Things left in memory after the
shifts in the modified version of your test case?

But that is assuming that the ObjectSpace tools tell the truth, it
does however not look like that on my box.
I have therefore no reason to believe that the above structure of the
code will leak, and have asked you if it will.
I will kill my box tonight :wink: (or not).

I guess you'll have to convince your self :slight_smile:

Cheers,
Bob

···

On 25-Oct-07, at 9:55 AM, Robert Dober wrote:

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 25-Oct-07, at 1:12 AM, Robert Dober wrote:

R.

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

56

Eric, thanks for the information.

Cheers,
Bob

···

On 25-Oct-07, at 12:40 PM, Eric Mahurin wrote:

The problem is that Ruby's COW (copy-on-write) scheme is messed up for
arrays. I've found much worse cases performance and memory-wise. Try
doing a small #slice on a big array a bunch and modify the small
slices. I made a 1.9 patch a couple years ago that fixed the problem
and made ALL operations near the front just as fast as the operations
on the back. Unfortunately, it never made it in. The way I did it
might be unique, not sure. I thought C++ deque does something
similiar, but it doesn't. When I find time, I'll try to do the fix
again.

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

57

>> Hi,
>>
>>
>>> Well I guess I have to believe you, but I would *love* to
>>> reproduce it
>>> if I do
>>> times do
>>> <<
>>> end
>>> times do
>>> shift
>>> end
>>> it will leak?
>>
>> How else would you describe the 10k Things left in memory after the
>> shifts in the modified version of your test case?
> But that is assuming that the ObjectSpace tools tell the truth, it
> does however not look like that on my box.
> I have therefore no reason to believe that the above structure of the
> code will leak, and have asked you if it will.
> I will kill my box tonight :wink: (or not).

I guess you'll have to convince your self :slight_smile:

yes I have because it just did not happen,

526/28 > uname -a && ruby -v
Linux zenwalk 2.6.18.1 #1 SMP PREEMPT Mon Oct 30 15:32:27 CET 2006
i686 athlon-4 i386 GNU/Linux
ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-linux]

the program was
N = 1000
M = 1000
a =
loop do
  N.times do
    a << Array.new( M ){ Object.new }
  end
  N.times do
    a.shift
  end
end

Ruby allocated up to 43M and then decided to release 3M regularly,
seems a very reasonable strategy to me.
I feel that it would be nice to show the exact leaking code so that I
can run it.

Cheers
Robert

···

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 25-Oct-07, at 9:55 AM, Robert Dober wrote:
> On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:
>> On 25-Oct-07, at 1:12 AM, Robert Dober wrote:

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

58

Now this is all very great but could you share with us what is going
on, right now I just see people complaining that it is broken, no
reference to a bug report, no test code, I feel that this is highly
unfair to Ruby.

Whatever I am not going to try to reproduce an error that might not
even be there.
I have not found any reference to this on Ruby core, could you please
give a pointer to your patch Eric?

Robert

···

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

On 25-Oct-07, at 12:40 PM, Eric Mahurin wrote:
> The problem is that Ruby's COW (copy-on-write) scheme is messed up for
> arrays. I've found much worse cases performance and memory-wise. Try
> doing a small #slice on a big array a bunch and modify the small
> slices. I made a 1.9 patch a couple years ago that fixed the problem
> and made ALL operations near the front just as fast as the operations
> on the back. Unfortunately, it never made it in. The way I did it
> might be unique, not sure. I thought C++ deque does something
> similiar, but it doesn't. When I find time, I'll try to do the fix
> again.

Eric, thanks for the information.

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

59

I guess you'll have to convince your self :slight_smile:

yes I have because it just did not happen,

526/28 > uname -a && ruby -v
Linux zenwalk 2.6.18.1 #1 SMP PREEMPT Mon Oct 30 15:32:27 CET 2006
i686 athlon-4 i386 GNU/Linux
ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-linux]

the program was
N = 1000
M = 1000
a =
loop do
  N.times do
    a << Array.new( M ){ Object.new }
  end
  N.times do
    a.shift
  end
end

Ruby allocated up to 43M and then decided to release 3M regularly,
seems a very reasonable strategy to me.
I feel that it would be nice to show the exact leaking code so that I
can run it.

You need to be careful what it is you are testing.

Try the tiny variation I've supplied below as it is, it'll execute relatively benignly. Then comment out the crucial line indicated by the comment. If you dare. BTW, if you run this make sure you keep on top of things and kill it before it starts screwing up your machine. Make sure you don't have anything important unsaved.

Why don't we keep all the 'a' arrays we generate? Shouldn't cause a problem right? If fact if you don't comment out the crucial line memory use will grow relatively slowly. If you do comment it out, then, well, it *is* your machine.

BTW, the *only* thing special about nil is that there is only one of them. You could use a symbol, true/false, a number -- just don't create a new object.

What happens if you comment out the whole second N.times loop? I get strangely similar behaviour to just commenting out the crucial line. Why's that? Because, in neither case, none of the arrays of Object is being collected.

If you get the behaviour I'm getting, how do you explain it other than as a leak?

Innocent bystanders: if you comment out that crucial line before running the program PAY ATTENTION AND BE READY TO KILL THE PROCESS BEFORE IT LOCKS UP YOUR MACHINE (very bad on OS X, it is an abrupt lockup of the system after about 20s on my machine, Ctrl-C won't work, get either your activity monitor up and visible or the force quit dialog).

N = 1000
M = 1000

as =
loop do
   a =
   N.times do
     a << Array.new( M ){ Object.new }
   end
   N.times do
     a[0] = nil # <<< the crucial line
     a.shift
   end

   as << a

   GC.start
   STDERR << '.'
end

Cheers,
Bob

···

On 25-Oct-07, at 1:57 PM, Robert Dober wrote:

On 10/25/07, Bob Hutchison <hutch@recursive.ca> wrote:

Cheers
Robert

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/

60

Now this is all very great but could you share with us what is going
on, right now I just see people complaining that it is broken, no
reference to a bug report, no test code, I feel that this is highly
unfair to Ruby.

Whatever I am not going to try to reproduce an error that might not
even be there.
I have not found any reference to this on Ruby core, could you please
give a pointer to your patch Eric?

Robert, I've provided several pieces of code to illustrate this problem.

I'm afraid this thread has the potential to degenerate very very quickly. I'm sorry I'm involved, and I'd very much like to be uninvolved as of now.

Cheers,
Bob

···

On 25-Oct-07, at 4:41 PM, Robert Dober wrote:

Robert

--
what do I think about Ruby?
http://ruby-smalltalk.blogspot.com/

----
Bob Hutchison -- tumblelog at http://www.recursive.ca/so/
Recursive Design Inc. -- weblog at http://www.recursive.ca/hutch
http://www.recursive.ca/ -- works on http://www.raconteur.info/cms-for-static-content/home/