So you want to make Ruby leak memory? Here's a simple way

Kirk_Haines · 28 August 2006 23:54

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
attrs = Hash['this','that','the other','enough','stuff','for six']
Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1, 1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to run GC after each loop iteration. The behavior is consistent. If you eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

Kent_Sibilev1 · 29 August 2006 00:15

Interesting. Does this patch fixes it?

Index: hash.c

···

===================================================================
RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
hash = hash_alloc(klass);

RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

return hash;

On 8/28/06, khaines@enigo.com <khaines@enigo.com> wrote:

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object
space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
attrs = Hash['this','that','the other','enough','stuff','for six']
Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1,
1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on
WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to
run GC after each loop iteration. The behavior is consistent. If you
eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

--
Kent
---

Rob_Sanheim1 · 29 August 2006 01:04

Wow...that seems to be a pretty major leak to have made it this long
w/o being noticed. Maybe that type of behavior isn't repeated often
enough in most programs to expose the leak?

- rob

···

On 8/28/06, khaines@enigo.com <khaines@enigo.com> wrote:

Make a hash:

h = {'a' => 1, 'b' => 2, 'c' => 3}

And make another out of it:

Hash[h]

Do that a lot. Watch RAM vanish. GC cleans up the objects out of object
space just fine, but RAM trickles away nonetheless.

Here is a test program:

loop do
attrs = Hash['this','that','the other','enough','stuff','for six']
Hash[attrs]
end

I have tested it on an installation of RedHat Enterprise Linux with 1.8.1,
1.8.2, 1.8.4, and 1.8.5 and also under 1.8.4 (one click installer) on
WinXP Home.

I've ran it under valgrind and just straight, and have inserted code to
run GC after each loop iteration. The behavior is consistent. If you
eliminate the Hash[attrs] line, there is no leak.

Kirk Haines

--

Kirk_Haines · 29 August 2006 00:22

It does indeed. Yay.

Kirk Haines

···

On Tue, 29 Aug 2006, Kent Sibilev wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
      hash = hash_alloc(klass);

      RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
      RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

      return hash;

Francis_Cianfrocca · 29 August 2006 00:29

Bravo, Kent. Props.

···

On 8/28/06, Kent Sibilev <ksruby@gmail.com> wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
        hash = hash_alloc(klass);

        RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
        RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

        return hash;

Kirk_Haines · 29 August 2006 01:26

It's a small enough leak that it takes a lot of iterations to start to add up. And while the threshold of pain involved in chasing that thing down wasn't horrible, it's high enough that I can understand people doing what I have done for a long time:

Oh, that process is getting a bit big. *kill* *restart*

Or they assume that it's a bug in their code somewhere and do the same thing.

Look at the discussion about Mutex leaking last week.

BTW, my test code makes use of a number of mutexes and heavy use of threading, and after running 120k hits through it, the RAM usage is rock solid at 13.8Mb. Normally after that many hits it would have grown to around 45mb. So, this is very pleasing to me.

Kirk Haines

···

On Tue, 29 Aug 2006, Rob Sanheim wrote:

Wow...that seems to be a pretty major leak to have made it this long
w/o being noticed. Maybe that type of behavior isn't repeated often
enough in most programs to expose the leak?

Nobuyoshi_Nakada1 · 29 August 2006 03:46

Hi,

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:

Interesting. Does this patch fixes it?

Yes, and modified it slightly.

Index: hash.c

···

===================================================================
RCS file: /cvs/ruby/src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -p -U 2 -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 03:43:58 -0000
@@ -224,7 +224,8 @@ rb_hash_foreach(hash, func, farg)
}

+static VALUE hash_alloc0 _((VALUE));
static VALUE hash_alloc _((VALUE));
static VALUE
-hash_alloc(klass)
+hash_alloc0(klass)
VALUE klass;
{
@@ -233,9 +234,19 @@ hash_alloc(klass)

hash->ifnone = Qnil;
- hash->tbl = st_init_table(&objhash);

return (VALUE)hash;
}

+static VALUE
+hash_alloc(klass)
+ VALUE klass;
+{
+ VALUE hash = hash_alloc0(klass);
+
+ RHASH(hash)->tbl = st_init_table(&objhash);
+
+ return hash;
+}
+
VALUE
rb_hash_new()
@@ -326,7 +337,5 @@ rb_hash_s_create(argc, argv, klass)

if (argc == 1 && TYPE(argv[0]) == T_HASH) {
- hash = hash_alloc(klass);
-
- RHASH(hash)->ifnone = Qnil;
+ hash = hash_alloc0(klass);
RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

--
Nobu Nakada

Just_Another_Victim2 · 29 August 2006 02:25

<khaines@enigo.com> wrote in message
news:Pine.LNX.4.61.0608281908340.21678@server1.enigo.com...

···

On Tue, 29 Aug 2006, Kent Sibilev wrote:

Interesting. Does this patch fixes it?

Index: hash.c

RCS file: /src/ruby/hash.c,v
retrieving revision 1.128.2.16
diff -u -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 00:14:17 -0000
@@ -328,6 +328,7 @@
      hash = hash_alloc(klass);

      RHASH(hash)->ifnone = Qnil;
+ st_free_table(RHASH(hash)->tbl);
      RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

      return hash;

It does indeed. Yay.

Nice!

Kent_Sibilev1 · 29 August 2006 03:58

Sure thing. The table initialization in this case is redundant.

···

On 8/28/06, nobu@ruby-lang.org <nobu@ruby-lang.org> wrote:

Hi,

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:
> Interesting. Does this patch fixes it?

Yes, and modified it slightly.

Index: hash.c

RCS file: /cvs/ruby/src/ruby/hash.c,v

retrieving revision 1.128.2.16
diff -p -U 2 -r1.128.2.16 hash.c
--- hash.c 6 Jul 2006 15:44:26 -0000 1.128.2.16
+++ hash.c 29 Aug 2006 03:43:58 -0000
@@ -224,7 +224,8 @@ rb_hash_foreach(hash, func, farg)
}

+static VALUE hash_alloc0 _((VALUE));
static VALUE hash_alloc _((VALUE));
static VALUE
-hash_alloc(klass)
+hash_alloc0(klass)
     VALUE klass;
{
@@ -233,9 +234,19 @@ hash_alloc(klass)

     hash->ifnone = Qnil;
- hash->tbl = st_init_table(&objhash);

     return (VALUE)hash;
}

+static VALUE
+hash_alloc(klass)
+ VALUE klass;
+{
+ VALUE hash = hash_alloc0(klass);
+
+ RHASH(hash)->tbl = st_init_table(&objhash);
+
+ return hash;
+}
+
VALUE
rb_hash_new()
@@ -326,7 +337,5 @@ rb_hash_s_create(argc, argv, klass)

     if (argc == 1 && TYPE(argv[0]) == T_HASH) {
- hash = hash_alloc(klass);
-
- RHASH(hash)->ifnone = Qnil;
+ hash = hash_alloc0(klass);
        RHASH(hash)->tbl = st_copy(RHASH(argv[0])->tbl);

--
Nobu Nakada

--
Kent
---

Stefan_Schmiedl · 29 August 2006 06:20

... but no memory leak is small enough to evade you forever, Kirk

Grrrrreat work!
s.

PS. Did you get my mail last week?

···

On Tue, 29 Aug 2006 10:26:17 +0900, khaines wrote:

It's a small enough leak that it takes a lot of iterations to start to add
up.

Yukihiro_Matsumoto2 · 29 August 2006 07:10

Hi,

···

In message "Re: So you want to make Ruby leak memory? Here's a simple way...." on Tue, 29 Aug 2006 12:46:51 +0900, nobu@ruby-lang.org writes:

At Tue, 29 Aug 2006 09:15:27 +0900,
Kent Sibilev wrote in [ruby-talk:211236]:

Interesting. Does this patch fixes it?

Yes, and modified it slightly.

OK, can you commit?

matz.

Forum · 29 August 2006 08:23

> It's a small enough leak that it takes a lot of iterations to start to
add
> up.

... but no memory leak is small enough to evade you forever, Kirk

it will also appear only shortly after a new release though

Grrrrreat work!

s.

Indeed

PS. Did you get my mail last week?

···

On 8/29/06, Stefan Schmiedl <s@xss.de> wrote:

On Tue, 29 Aug 2006 10:26:17 +0900, khaines wrote:

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

- Albert Einstein

Topic		Replies	Views
Mem leak without add_heap()? ruby-talk	1	136	28 August 2006
Newbie Mem Leak Issue ruby-talk	13	56	22 February 2008
Memory leak in ruby or dumb mistake? ruby-talk	7	102	17 March 2004
Memory leak in 1.8.6-p287 ruby-talk	1	104	25 August 2008
Memory Leak Madness ruby-talk	18	58	13 January 2007

So you want to make Ruby leak memory? Here's a simple way

Index: hash.c

Index: hash.c

Index: hash.c

Index: hash.c

Related Topics