Is this still an issue with the "official" release?
- Wilkes
Is this still an issue with the "official" release?
Nobody has given, yet, the proof that it exist a memory leak with
continuations.
Guy Decoux
This was posted recently:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213
This was posted recently:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213
This is the proof that the GC is conservative, this does not mean that it
exist a memory leak.
Guy Decoux
Sure, but the results might be the same 
Regards,
Michael
···
Am Mittwoch 19 Januar 2005 16:53 schrieb ts:
> This was posted recently:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213This is the proof that the GC is conservative, this does not mean that it
exist a memory leak.
I don't see a callcc in there anywhere, so callback_stream.with_callbacks_for could be doing other naughty things. From personal experience with callcc, I would bet on something referencing live objects over any memory leaks.
PGP.sig (186 Bytes)
···
On 19 Jan 2005, at 08:15, Michael Neumann wrote:
Am Mittwoch 19 Januar 2005 16:53 schrieb ts:
> This was posted recently:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213This is the proof that the GC is conservative, this does not mean that it
exist a memory leak.Sure, but the results might be the same
--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04
>>
>> > This was posted recently:
>> > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213
>>
>> This is the proof that the GC is conservative, this does not mean
>> that it
>> exist a memory leak.
>
> Sure, but the results might be the sameI don't see a callcc in there anywhere, so
callback_stream.with_callbacks_for could be doing other naughty things.
Right. But it's the place where the callbacks are invoked. And the callback
that gets invoked actually uses callcc. It was very strange that, when I
change that line in some manner (introduce an assignment), the memory
consumption changes drastically.
From personal experience with callcc, I would bet on something
referencing live objects over any memory leaks.
I think the problem is due to the conservative GC. But of course, there might
also be some bugs in it (but then, memory consumption should be unbounded as
well, if I remove continuations, which I've done).
Regards,
Michael
···
Am Mittwoch 19 Januar 2005 19:35 schrieb Eric Hodel:
On 19 Jan 2005, at 08:15, Michael Neumann wrote:
> Am Mittwoch 19 Januar 2005 16:53 schrieb ts:
If it is something other than a memory leak, and if it is locatable 
would continuations in Wee become a 'recommended usage'?
···
"Eric Hodel" <drbrain@segment7.net> wrote
On 19 Jan 2005, at 08:15, Michael Neumann wrote:
Am Mittwoch 19 Januar 2005 16:53 schrieb ts:
> This was posted recently:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213This is the proof that the GC is conservative, this does not mean
that it
exist a memory leak.Sure, but the results might be the same
I don't see a callcc in there anywhere, so
callback_stream.with_callbacks_for could be doing other naughty things.
From personal experience with callcc, I would bet on something
referencing live objects over any memory leaks.
Hm, there's still another issue. You can't marshal continuations in Ruby.
Don't know whether they'd become recommended
But it's so easy to leave them out or integrate them back into Wee, so I don't
think about that issue yet.
Regards,
Michael
···
Am Mittwoch 19 Januar 2005 20:41 schrieb itsme213:
If it is something other than a memory leak, and if it is locatable
would continuations in Wee become a 'recommended usage'?
The last Ruby Weekly News has something that might be helpful here:
http://rubygarden.org/ruby?RubyNews/2005-01-03
Tanaka Akira posted a patch to record GC-related information:
* the total number of GC invocation
* the number of GC invocation, when an object is collected
* the location where the last GC is yielded
This patch might help you if you are interested in GC internals.
Due to the way Ruby saves the environment, the problem could be on the line you show, or it could be where callcc is invoked. Without a simple testcase, its difficult to track this problem down to a memory leak, or a result of the conservative GC/environment feature:
def x
w = :stuff
proc do # w is free in this proc, so we don't need to save any reference to it
# in the proc
puts "w: #{eval "w"}" # grab w out of the environment
end
end
x.call # prints w: stuff
# Because ruby keeps the entire local variable table around, we can get to w
PGP.sig (186 Bytes)
···
On 19 Jan 2005, at 11:36, Michael Neumann wrote:
Am Mittwoch 19 Januar 2005 19:35 schrieb Eric Hodel:
On 19 Jan 2005, at 08:15, Michael Neumann wrote:
Am Mittwoch 19 Januar 2005 16:53 schrieb ts:
> This was posted recently:
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/124213This is the proof that the GC is conservative, this does not mean
that it
exist a memory leak.Sure, but the results might be the same
I don't see a callcc in there anywhere, so
callback_stream.with_callbacks_for could be doing other naughty things.Right. But it's the place where the callbacks are invoked. And the callback
that gets invoked actually uses callcc. It was very strange that, when I
change that line in some manner (introduce an assignment), the memory
consumption changes drastically.
--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04
In article <1106173537.085688.40700@z14g2000cwz.googlegroups.com>,
"jc" <james.cromwell@gmail.com> writes:
Tanaka Akira posted a patch to record GC-related information:
* the total number of GC invocation
* the number of GC invocation, when an object is collected
* the location where the last GC is yielded
This patch might help you if you are interested in GC internals.
It was not for memory leak. It was intended to investigate a wrongly
collected object. The memory leak is opposite situation: some objects
are not collected.
So, I updated the patch to investigate a live object.
The attached patch implements GC.trace_object which makes Ruby to
report why a registered object is not collected.
% ./ruby -ve '
k = nil
5.times {
callcc {|k| }
p k
GC.trace_object(k)
GC.start
}
'|&uniq
#<Continuation:0x401c75e8>
gc_mark(1): Scope (0x401d8d34) -> Data (0x401c75e8)
gc_mark(1): root (0x08078711,0x08078f4e) -> Data (0x401c75e8)
#<Continuation:0x401c78a4>
gc_mark(2): Scope (0x401d8d34) -> Data (0x401c78a4)
gc_mark(2): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(2): root (0x08078711,0x08078f4e) -> Data (0x401c78a4)
#<Continuation:0x401c7804>
gc_mark(3): Scope (0x401d8d34) -> Data (0x401c7804)
gc_mark(3): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(3): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(3): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(3): root (0x08078711,0x08078f4e) -> Data (0x401c7804)
#<Continuation:0x401c7728>
gc_mark(4): Scope (0x401d8d34) -> Data (0x401c7728)
gc_mark(4): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(4): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(4): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(4): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(4): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(4): root (0x08078711,0x08078f4e) -> Data (0x401c7728)
#<Continuation:0x401c75ac>
gc_mark(5): Scope (0x401d8d34) -> Data (0x401c75ac)
gc_mark(5): Data (0x401c75ac) -> Data (0x401c7728)
gc_mark(5): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(5): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(5): Data (0x401c78a4) -> Data (0x401c75e8)
gc_mark(5): Data (0x401c7804) -> Data (0x401c78a4)
gc_mark(5): Data (0x401c7728) -> Data (0x401c7804)
gc_mark(5): Data (0x401c75ac) -> Data (0x401c7728)
gc_mark(5): root (0x08078711,0x08078f4e) -> Data (0x401c75ac)
ruby 1.9.0 (2005-01-21) [i686-linux]
This mean that the Nth continuation refers the N-1th continuation.
So the continuations are never collected.
I'm not sure that it should be called "memory leak" or not. But it is
unpleasant behavior anyway. I have no idea to fix it portably,
though.
Index: error.c
···
===================================================================
RCS file: /src/ruby/error.c,v
retrieving revision 1.103
diff -u -p -r1.103 error.c
--- error.c 17 Nov 2004 02:27:37 -0000 1.103
+++ error.c 21 Jan 2005 11:29:24 -0000
@@ -244,6 +244,19 @@ static struct types {
{-1, 0}
};
+char *rb_object_structure_type(VALUE obj)
+{
+ struct types *type = builtin_types;
+ int t = TYPE(obj);
+
+ while (type->type >= 0) {
+ if (type->type == t) {
+ return type->name;
+ }
+ type++;
+ }
+}
+
void
rb_check_type(x, t)
VALUE x;
Index: eval.c
RCS file: /src/ruby/eval.c,v
retrieving revision 1.748
diff -u -p -r1.748 eval.c
--- eval.c 5 Jan 2005 03:49:50 -0000 1.748
+++ eval.c 21 Jan 2005 11:29:24 -0000
@@ -3865,7 +3865,7 @@ rb_eval(self, n)
break;
default:
- rb_bug("unknown node type %d", nd_type(node));
+ rb_bug("unknown node type %d (0x%lx)", nd_type(node), (long)node);
}
finish:
CHECK_INTS;
@@ -5741,8 +5741,8 @@ rb_call(klass, recv, mid, argc, argv, sc
struct cache_entry *ent;
if (!klass) {
- rb_raise(rb_eNotImpError, "method `%s' called on terminated object (0x%lx)",
- rb_id2name(mid), recv);
+ rb_bug("method `%s' called on terminated object (0x%lx)",
+ rb_id2name(mid), recv);
}
/* is it in the method cache? */
ent = cache + EXPR1(klass, mid);
Index: gc.c
RCS file: /src/ruby/gc.c,v
retrieving revision 1.195
diff -u -p -r1.195 gc.c
--- gc.c 7 Jan 2005 09:05:52 -0000 1.195
+++ gc.c 21 Jan 2005 11:29:24 -0000
@@ -284,6 +284,7 @@ typedef struct RVALUE {
struct {
unsigned long flags; /* always 0 for freed obj */
struct RVALUE *next;
+ long gc_count;
} free;
struct RBasic basic;
struct RObject object;
@@ -308,6 +309,7 @@ typedef struct RVALUE {
#endif
} RVALUE;
+static long gc_count = 0;
static RVALUE *freelist = 0;
static RVALUE *deferred_final_list = 0;
@@ -701,6 +703,28 @@ rb_gc_mark_maybe(obj)
#define GC_LEVEL_MAX 250
+static VALUE gc_mark_base = Qundef;
+static st_table *gc_mark_interest = NULL;
+
+static void gc_collected(RVALUE *p, long gc_count)
+{
+ VALUE v = (VALUE)p;
+ p->as.free.gc_count = gc_count;
+ if (gc_mark_interest && st_lookup(gc_mark_interest, v, NULL)) {
+ fprintf(stderr, "gc_collected(%ld): 0x%08lx\n", gc_count, v);
+ st_delete(gc_mark_interest, &v, NULL);
+ }
+}
+
+static VALUE
+gc_trace_object(VALUE self, VALUE obj)
+{
+ if (!gc_mark_interest)
+ gc_mark_interest = st_init_numtable();
+ st_insert(gc_mark_interest, obj, 1);
+ return obj;
+}
+
void
gc_mark(ptr, lev)
VALUE ptr;
@@ -708,6 +732,23 @@ gc_mark(ptr, lev)
{
register RVALUE *obj;
+ extern char *rb_object_structure_type(VALUE obj);
+ if (gc_mark_interest && st_lookup(gc_mark_interest, ptr, NULL) && gc_mark_base != ptr) {
+ if (gc_mark_base == Qundef) {
+ fprintf(stderr, "gc_mark(%ld): root (0x%08lx,0x%08lx) -> %s (0x%08lx)\n",
+ gc_count,
+ (unsigned long)__builtin_return_address(0),
+ (unsigned long)__builtin_return_address(1),
+ rb_object_structure_type(ptr), ptr);
+ }
+ else {
+ fprintf(stderr, "gc_mark(%ld): %s (0x%08lx) -> %s (0x%08lx)\n",
+ gc_count,
+ rb_object_structure_type(gc_mark_base), gc_mark_base,
+ rb_object_structure_type(ptr), ptr);
+ }
+ }
+
obj = RANY(ptr);
if (rb_special_const_p(ptr)) return; /* special const not marked */
if (obj->as.basic.flags == 0) return; /* free cell */
@@ -737,7 +778,7 @@ rb_gc_mark(ptr)
}
static void
-gc_mark_children(ptr, lev)
+gc_mark_children1(ptr, lev)
VALUE ptr;
int lev;
{
@@ -994,6 +1035,17 @@ gc_mark_children(ptr, lev)
}
}
+static void
+gc_mark_children(ptr, lev)
+ VALUE ptr;
+ int lev;
+{
+ VALUE saved_base = gc_mark_base;
+ gc_mark_base = ptr;
+ gc_mark_children1(ptr, lev);
+ gc_mark_base = saved_base;
+}
+
static void obj_free _((VALUE));
static void
@@ -1006,6 +1058,7 @@ finalize_list(p)
if (!FL_TEST(p, FL_SINGLETON)) { /* not freeing page */
p->as.free.flags = 0;
p->as.free.next = freelist;
+ gc_collected(p, gc_count);
freelist = p;
}
p = tmp;
@@ -1040,6 +1093,7 @@ gc_sweep()
unsigned long live = 0;
mark_source_filename(ruby_sourcefile);
+ if (source_filenames)
st_foreach(source_filenames, sweep_source_filename, 0);
freelist = 0;
@@ -1059,11 +1113,13 @@ gc_sweep()
if (need_call_final && FL_TEST(p, FL_FINALIZE)) {
p->as.free.flags = FL_MARK; /* remain marked */
p->as.free.next = final_list;
+ gc_collected(p, gc_count);
final_list = p;
}
else {
p->as.free.flags = 0;
p->as.free.next = freelist;
+ gc_collected(p, gc_count);
freelist = p;
}
n++;
@@ -1115,6 +1171,7 @@ rb_gc_force_recycle(p)
{
RANY(p)->as.free.flags = 0;
RANY(p)->as.free.next = freelist;
+ gc_collected(RANY(p), -1);
freelist = RANY(p);
}
@@ -1290,6 +1347,11 @@ int rb_setjmp (rb_jmp_buf);
#endif /* __human68k__ or DJGPP */
#endif /* __GNUC__ */
+#ifdef __GNUC__
+void *main_return_address;
+static void *last_gc_stacktrace[10];
+#endif
+
static void
garbage_collect()
{
@@ -1312,6 +1374,10 @@ garbage_collect()
if (during_gc) return;
during_gc++;
+ gc_count++;
+ if (gc_count < 0)
+ gc_count = 0;
+
init_mark_stack();
/* mark frame stack */
@@ -1401,6 +1467,21 @@ garbage_collect()
}
}
gc_sweep();
+
+#ifdef __GNUC__
+ memset(last_gc_stacktrace, 0, sizeof(last_gc_stacktrace));
+ if ((last_gc_stacktrace[0] = __builtin_return_address(0)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[2] = __builtin_return_address(2)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[3] = __builtin_return_address(3)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[4] = __builtin_return_address(4)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[5] = __builtin_return_address(5)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[6] = __builtin_return_address(6)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[7] = __builtin_return_address(7)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[8] = __builtin_return_address(8)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[9] = __builtin_return_address(9)) == main_return_address) goto gc_stacktrace_done;
+ gc_stacktrace_done: ;
+#endif
}
void
@@ -1427,6 +1508,20 @@ rb_gc_start()
return Qnil;
}
+VALUE
+numfree()
+{
+ RVALUE *f = freelist;
+ long num = 0;
+
+ while (f != 0) {
+ f = f->as.free.next;
+ num += 1;
+ }
+ printf("numfree:%ld\tmalloc_increase:%lu\tmalloc_limit:%lu\n", num, malloc_increase, malloc_limit);
+ return Qnil;
+}
+
void
ruby_set_stack_size(size)
size_t size;
@@ -1929,6 +2024,9 @@ Init_GC()
rb_define_module_function(rb_mObSpace, "undefine_finalizer", undefine_final, 1);
rb_define_module_function(rb_mObSpace, "_id2ref", id2ref, 1);
+
+ rb_define_module_function(rb_mObSpace, "numfree", numfree, 0);
+ rb_define_module_function(rb_mGC, "trace_object", gc_trace_object, 1);
rb_gc_register_address(&rb_mObSpace);
rb_global_variable(&finalizers);
Index: main.c
RCS file: /src/ruby/main.c,v
retrieving revision 1.13
diff -u -p -r1.13 main.c
--- main.c 23 Jun 2004 12:59:01 -0000 1.13
+++ main.c 21 Jan 2005 11:29:24 -0000
@@ -21,6 +21,10 @@
static void objcdummyfunction( void ) { objc_msgSend(); }
#endif
+#ifdef __GNUC__
+extern void *main_return_address;
+#endif
+
int
main(argc, argv, envp)
int argc;
@@ -31,6 +35,10 @@ main(argc, argv, envp)
#endif
#if defined(__MACOS__) && defined(__MWERKS__)
argc = ccommand(&argv);
+#endif
+
+#ifdef __GNUC__
+ main_return_address = __builtin_return_address(0);
#endif
ruby_init();
Index: variable.c
RCS file: /src/ruby/variable.c,v
retrieving revision 1.119
diff -u -p -r1.119 variable.c
--- variable.c 10 Jan 2005 14:07:53 -0000 1.119
+++ variable.c 21 Jan 2005 11:29:25 -0000
@@ -467,6 +467,7 @@ mark_global_entry(key, entry)
void
rb_gc_mark_global_tbl()
{
+ if (rb_global_tbl)
st_foreach_safe(rb_global_tbl, mark_global_entry, 0);
}
--
Tanaka Akira
+#ifdef __GNUC__
perhaps not a good idea
+ memset(last_gc_stacktrace, 0, sizeof(last_gc_stacktrace));
+ if ((last_gc_stacktrace[0] = __builtin_return_address(0)) == main_return_address) goto gc_stacktrace_done;
+ if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
uln% gdb --quiet ./miniruby
Using host libthread_db library "/lib64/libthread_db.so.1".
(gdb) r lib/fileutils.rb
Starting program: /opt/ts/ruby/r190/tmp/ruby/miniruby lib/fileutils.rb
Program received signal SIGSEGV, Segmentation fault.
0x000000000042dab6 in garbage_collect () at gc.c:1474
1474 if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
(gdb) c
Continuing.
lib/fileutils.rb:1132: [BUG] Segmentation fault
ruby 1.9.0 (2005-01-21) [x86_64-linux]
Program received signal SIGABRT, Aborted.
0x0000002a95a35af9 in kill () from /lib64/libc.so.6
(gdb)
Guy Decoux
In article <200501211209.j0LC9jx10813@moulon.inra.fr>,
ts <decoux@moulon.inra.fr> writes:
> +#ifdef __GNUC__
perhaps not a good idea
Yes. Not for everyone.
0x000000000042dab6 in garbage_collect () at gc.c:1474
1474 if ((last_gc_stacktrace[1] = __builtin_return_address(1)) == main_return_address) goto gc_stacktrace_done;
Maybe gcc doesn't support __builtin_return_address(1) on IA64.
···
--
Tanaka Akira
In article <87vf9q45eg.fsf@serein.a02.aist.go.jp>,
Tanaka Akira <akr@m17n.org> writes:
Maybe gcc doesn't support __builtin_return_address(1) on IA64.
Oops. x86_64, not IA64.
···
--
Tanaka Akira