So you want to make Ruby leak memory? Here's a simple way

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
   attrs = Hash['this','that','the other','enough','stuff','for six']
   Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1, 1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to run GC after each loop iteration. The behavior is consistent. If you eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

Interesting. Does this patch fixes it?

Index: hash.c

···

===================================================================
RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
        hash = hash_alloc(klass);

        RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
        RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

        return hash;

On 8/28/06, khaines@enigo.com <khaines@enigo.com> wrote:

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object
space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
   attrs = Hash['this','that','the other','enough','stuff','for six']
   Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1,
1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on
WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to
run GC after each loop iteration. The behavior is consistent. If you
eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

--
Kent
---
http://www.datanoise.com

Wow...that seems to be a pretty major leak to have made it this long
w/o being noticed. Maybe that type of behavior isn't repeated often
enough in most programs to expose the leak?

- rob

···

On 8/28/06, khaines@enigo.com <khaines@enigo.com> wrote:

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object
space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
   attrs = Hash['this','that','the other','enough','stuff','for six']
   Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1,
1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on
WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to
run GC after each loop iteration. The behavior is consistent. If you
eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

--
http://www.robsanheim.com


http://www.ajaxian.com

It does indeed. Yay.

Kirk Haines

···

On Tue, 29 Aug 2006, Kent Sibilev wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
      hash = hash_alloc(klass);

      RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
      RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

      return hash;

Bravo, Kent. Props.

···

On 8/28/06, Kent Sibilev <ksruby@gmail.com> wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
        hash = hash_alloc(klass);

        RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
        RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

        return hash;

It's a small enough leak that it takes a lot of iterations to start to add up. And while the threshold of pain involved in chasing that thing down wasn't horrible, it's high enough that I can understand people doing what I have done for a long time:

Oh, that process is getting a bit big. *kill* *restart*

Or they assume that it's a bug in their code somewhere and do the same thing.

Look at the discussion about Mutex leaking last week.

BTW, my test code makes use of a number of mutexes and heavy use of threading, and after running 120k hits through it, the RAM usage is rock solid at 13.8Mb. Normally after that many hits it would have grown to around 45mb. So, this is very pleasing to me.

Kirk Haines

···

On Tue, 29 Aug 2006, Rob Sanheim wrote:

Wow...that seems to be a pretty major leak to have made it this long
w/o being noticed. Maybe that type of behavior isn't repeated often
enough in most programs to expose the leak?

Hi,

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:

Interesting. Does this patch fixes it?

Yes, and modified it slightly.

Index: hash.c

···

===================================================================
RCS file: /cvs/ruby/src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -p -U 2 -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 03:43:58 -0000
@@ -224,7 +224,8 @@ rb_hash_foreach(hash, func, farg)
}

+static VALUE hash_alloc0 _((VALUE));
static VALUE hash_alloc _((VALUE));
static VALUE
-hash_alloc(klass)
+hash_alloc0(klass)
     VALUE klass;
{
@@ -233,9 +234,19 @@ hash_alloc(klass)

     hash->ifnone = Qnil;
- hash->tbl = st_init_table(&objhash);

     return (VALUE)hash;
}

+static VALUE
+hash_alloc(klass)
+ VALUE klass;
+{
+ VALUE hash = hash_alloc0(klass);
+
+ RHASH(hash)->tbl = st_init_table(&objhash);
+
+ return hash;
+}
+
VALUE
rb_hash_new()
@@ -326,7 +337,5 @@ rb_hash_s_create(argc, argv, klass)

     if (argc == 1 && TYPE(argv[0]) == T_HASH) {
- hash = hash_alloc(klass);
-
- RHASH(hash)->ifnone = Qnil;
+ hash = hash_alloc0(klass);
   RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

--
Nobu Nakada

<khaines@enigo.com> wrote in message
news:Pine.LNX.4.61.0608281908340.21678@server1.enigo.com...

···

On Tue, 29 Aug 2006, Kent Sibilev wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
      hash = hash_alloc(klass);

      RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
      RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

      return hash;

It does indeed. Yay.

    Nice!

Sure thing. The table initialization in this case is redundant.

···

On 8/28/06, nobu@ruby-lang.org <nobu@ruby-lang.org> wrote:

Hi,

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:
> Interesting. Does this patch fixes it?

Yes, and modified it slightly.

Index: hash.c

RCS file: /cvs/ruby/src/ruby/hash.c,v

retrieving revision 1.128.2.16
diff -p -U 2 -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 03:43:58 -0000
@@ -224,7 +224,8 @@ rb_hash_foreach(hash, func, farg)
}

+static VALUE hash_alloc0 _((VALUE));
static VALUE hash_alloc _((VALUE));
static VALUE
-hash_alloc(klass)
+hash_alloc0(klass)
     VALUE klass;
{
@@ -233,9 +234,19 @@ hash_alloc(klass)

     hash->ifnone = Qnil;
- hash->tbl = st_init_table(&objhash);

     return (VALUE)hash;
}

+static VALUE
+hash_alloc(klass)
+ VALUE klass;
+{
+ VALUE hash = hash_alloc0(klass);
+
+ RHASH(hash)->tbl = st_init_table(&objhash);
+
+ return hash;
+}
+
VALUE
rb_hash_new()
@@ -326,7 +337,5 @@ rb_hash_s_create(argc, argv, klass)

     if (argc == 1 && TYPE(argv[0]) == T_HASH) {
- hash = hash_alloc(klass);
-
- RHASH(hash)->ifnone = Qnil;
+ hash = hash_alloc0(klass);
        RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

--
Nobu Nakada

--
Kent
---
http://www.datanoise.com

... but no memory leak is small enough to evade you forever, Kirk :slight_smile:

Grrrrreat work!
s.

PS. Did you get my mail last week?

···

On Tue, 29 Aug 2006 10:26:17 +0900, khaines wrote:

It's a small enough leak that it takes a lot of iterations to start to add
up.

Hi,

···

In message "Re: So you want to make Ruby leak memory? Here's a simple way...." on Tue, 29 Aug 2006 12:46:51 +0900, nobu@ruby-lang.org writes:

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:

Interesting. Does this patch fixes it?

Yes, and modified it slightly.

OK, can you commit?

              matz.

> It's a small enough leak that it takes a lot of iterations to start to
add
> up.

... but no memory leak is small enough to evade you forever, Kirk :slight_smile:

it will also appear only shortly after a new release though

Grrrrreat work!

s.

Indeed

PS. Did you get my mail last week?

···

On 8/29/06, Stefan Schmiedl <s@xss.de> wrote:

On Tue, 29 Aug 2006 10:26:17 +0900, khaines wrote:

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

- Albert Einstein