Weird GC behaviour or memory leak with NArray::to_na?

ruby-talk
1

I have what appears to be a memory leak in my code, and I’m able to
reproduce it using the following test code:

require ‘narray’

a=Array.new(100,2)
200000.times{|x|
#c=a.collect{|b| b+1} #This is OK.
#a.collect!{|b| b+1} #This is OK.
#a=a.collect{|b| b+1} #This is OK.
#c=Array(a) #This is OK.
d=NArray.to_na(a) #This results in huge memory bloat
}

Running this code shows my memory footprint increase monotonically
from ~2M to >20M… I killed the process at this point. Any of the
other commented lines would run with a stable memory footprint.

However, if I add ObjectSpace::garbage_collect into the loop,
everything is fine.

Now to complicate matters further:

require 'narray’
a=Array.new(100,2)
200000.times{|x|
d=a.collect{|b| NArray.to_na([b,b])} #memory bloat despite explicit
GC!
ObjectSpace::garbage_collect
}

This exhibits the growing memory footprint problem… Help? Note:
I realize this example may be a little contrived, but my code does
this plus some more, requiring (IMHO) this approach.

Can someone explain this behaviour? I use the .to_na constructor a
lot in my code, and really need to sort this out, otherwise I cannot
run my code for longer than brief periods before my system comes
crashing on down.

Thanks, Jeff

2

I have what appears to be a memory leak in my code, and I’m able to
reproduce it using the following test code:

require ‘narray’

a=Array.new(100,2)
200000.times{|x|
#c=a.collect{|b| b+1} #This is OK.
#a.collect!{|b| b+1} #This is OK.
#a=a.collect{|b| b+1} #This is OK.
#c=Array(a) #This is OK.
d=NArray.to_na(a) #This results in huge memory bloat
}

Running this code shows my memory footprint increase monotonically
from ~2M to >20M… I killed the process at this point. Any of the
other commented lines would run with a stable memory footprint.

Hmm…

% ruby -v
ruby 1.8.1 (2003-12-25) [i686-linux]

% cat testna.rb
require ‘narray’

a=Array.new(100,2)
system “ps v”
5.times do
200000.times do |x|
d=NArray.to_na(a)
end
system “ps v | grep ruby | grep -v grep”
end

% ruby testna.rb
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
5400 pts/3 S 0:00 253 570 2445 1600 0.2 ruby testna.rb
5400 pts/3 S 0:08 257 570 17565 16736 2.1 ruby testna.rb
5400 pts/3 S 0:16 257 570 25265 24436 3.1 ruby testna.rb
5400 pts/3 S 0:25 257 570 33605 32776 4.2 ruby testna.rb
5400 pts/3 S 0:33 257 570 41301 40472 5.2 ruby testna.rb
5400 pts/3 S 0:41 257 570 49481 48484 6.3 ruby testna.rb

Thanks, it is a bug.

— na_array.c 2003/03/03 11:15:28 1.5
+++ na_array.c 2004/02/19 04:14:29
@@ -107,6 +107,7 @@
for (i=0; r–>0; i++) {
shape[i] = mdai->item[r].shape;
}

  • xfree(mdai->type);
    xfree(mdai->item);
    xfree(mdai);
    return shape;

Then,

% ruby testna.rb
5964 pts/3 R 0:00 253 570 2445 1604 0.2 ruby testna.rb
5964 pts/3 S 0:02 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:03 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:05 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 R 0:07 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:09 260 570 7733 6920 0.9 ruby testna.rb

Masahiro Tanaka

3

This problem was due to a memory leak in NArray 0.5.7. Updating to
0.5.7p2 fixed the problem.

Jeff