I just discovered that I have a GC related bug, or that is to say it doesn’t
seem to occur when I disable the GC with GC.disable.
Are there any special GC-related bug finding techniques that are worth
mentioning?
Cheers,
Thomas
Are there any special GC-related bug finding techniques that are worth
mentioning?
For starters, is there a way to make the GC print some information about the
object it reaps?
Cheers,
Thomas
Thomas Sondergaard wrote:
Are there any special GC-related bug finding techniques that are worth
mentioning?For starters, is there a way to make the GC print some information about the
object it reaps?
I wrote a GC debugging patch for the opposite problem (trying to find
why GC wasn’t freeing an object that I though it should). Also, it’s for
1.6.7 and 1.7.2. But it may be some help.
http://redshift.sourceforge.net/debugging-GC/
See also
which would be a good place to make a note of any solution you find.
Q1) I am wondering about the rb_gc_register_address/rb_gc_unregister_address
function pairs.
The both take a VALUE pointer rather than a VALUE. Does that mean that to
unregister I have to pass it the exact same VALUE pointer, or will another
VALUE pointer pointing to the same object be equivalent. I’m asking because
I might rb_gc_register_address and rb_gc_unregister_address multiple times
for the same VALUE, but only once with each VALUE pointer.
Here’s a scenario:
VALUE array = rb_ary_new(); // At some point this array is created
somewhere in ruby
… a little later …
// I need to pass a reference of the object out of ruby’s reach
// so I ‘lock’ the object with rb_gc_register_address and keep the arrayRef
// pointer with me somewhere safe
VALUE *arrayRef = ALLOC(VALUE);
*arrayRef = array;
rb_gc_register_address(arrayRef)
… a little later …
// I need to pass another reference out and do the exact same thing again
VALUE *arrayRef2 = ALLOC(VALUE);
*arrayRef2 = array;
rb_gc_register_address(arrayRef2)
… a little later again …
// The second non-ruby object referencing the ruby object is dead and gone
and I
// unregister using the same pointer as I registered with
rb_gc_unregister_address(arrayRef2);
// I then free the pointer that I allocated, assuming that is my
responsibility
free(arrayRef2);
… a little later again …
// I do it again because the first non-ruby object referencing the ruby
object is gone
rb_gc_unregister_address(arrayRef);
free(arrayRef);
// At this point the ruby object should be ready to be garbage collected
unless there
// are still references to it in ruby.
Is this the way to do it?
Q2) Another thing that I am wondering about. Can the GC be invoked in the
middle of my C code or does that only happen when control is in the “guts of
ruby”. What prevents the VALUEs I have on the stack from being GC’ed in that
case?
/Thomas
Here's a scenario:
The best is to simplify your code to a few lines, and *post* it
completely.
Q2) Another thing that I am wondering about. Can the GC be invoked in the
middle of my C code or does that only happen when control is in the "guts of
ruby". What prevents the VALUEs I have on the stack from being GC'ed in that
case?
The GC can't be invoked in the middle of your C code, if you don't call
ruby functions.
Guy Decoux
Ruby’s GC will not recycle objects that are referenced from the stack.
T
···
On Thu, 11 Sep 2003, Thomas Sondergaard wrote:
What prevents the VALUEs I have on the stack from being GC’ed
Thomas Sondergaard wrote:
Q1) I am wondering about the rb_gc_register_address/rb_gc_unregister_address
function pairs.The both take a VALUE pointer rather than a VALUE. Does that mean that to
unregister I have to pass it the exact same VALUE pointer, or will another
VALUE pointer pointing to the same object be equivalent. I’m asking because
I might rb_gc_register_address and rb_gc_unregister_address multiple times
for the same VALUE, but only once with each VALUE pointer.
It uses the address, not the value.
Here’s a scenario:
VALUE array = rb_ary_new(); // At some point this array is created
somewhere in ruby… a little later …
// I need to pass a reference of the object out of ruby’s reach
// so I ‘lock’ the object with rb_gc_register_address and keep the arrayRef
// pointer with me somewhere safe
VALUE *arrayRef = ALLOC(VALUE);
*arrayRef = array;
rb_gc_register_address(arrayRef)
[…snip…]
Is this the way to do it?
That way would work fine, I think, but it seems a little convoluted.
If your C/C++ object has a VALUE member, you need only register the
address of that just once, and unregister it in the destructor. As long
as your C/C++ object is around, the VALUE is registered and the object
it is tied to will not go away.
Also, if the VALUE you need is just temporary for while you’re in a C
call, just use a regular VALUE stack variable; they’re protected (see
below) while in-use and cleaned up automatically when you’re done with them.
Inversely, if you’re trying to tie an C/C++ object/data to a Ruby
object, just use malloc to allocated it and associate it with a Ruby
object with Data_Wrap_Struct and provide a free function (use the normal
free/malloc routines).
Q2) Another thing that I am wondering about. Can the GC be invoked in the
middle of my C code or does that only happen when control is in the “guts of
ruby”. What prevents the VALUEs I have on the stack from being GC’ed in that
case?
The GC can be invoked whenever you call any Ruby function. The Ruby GC
scans the stack for VALUE variables and “marks” any it finds which point
to valid objects. So long as the current stack pointer is
deeper/further into the stack than your VALUE variable, it is protected.
When your C/C++ function returns, it is eligible for collection and
will be left unmarked during the next GC sweep.
One gotcha I learned about this is, the object must really be on the
stack. Compilers these days all optimize, so to be sure your variable
actually lives on the stack and not in a register somewhere, all VALUEs
that you want marked as in-use, you must declare it as volatile, i.e.:
volatile VALUE tempArray;
Sean O'Dell
It would probably make sense to
#define SVALUE volatile VALUE
···
On Fri, Sep 12, 2003 at 02:53:23AM +0900, Sean O’Dell wrote:
One gotcha I learned about this is, the object must really be on the
stack. Compilers these days all optimize, so to be sure your variable
actually lives on the stack and not in a register somewhere, all VALUEs
that you want marked as in-use, you must declare it as volatile, i.e.:volatile VALUE tempArray;
–
_ _
__ __ | | ___ _ __ ___ __ _ _ __
'_ \ /| __/ __| '__ \ / ` | ’ \
) | (| | |__ \ | | | | | (| | | | |
.__/ _,|_|/| || ||_,|| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com
‘Ooohh… “FreeBSD is faster over loopback, when compared to Linux
over the wire”. Film at 11.’
– Linus Torvalds
If your C/C++ object has a VALUE member, you need only register the
address of that just once, and unregister it in the destructor. As long
as your C/C++ object is around, the VALUE is registered and the object
it is tied to will not go away.
That is exactly what I am doing I just wanted to spell out the scenario as
it may happen when there are more than one C++ wrapped reference to a ruby
object.
I’ll try that volatile thing and see if it makes a difference.
Cheers,
Thomas
Thomas Sondergaard wrote:
If your C/C++ object has a VALUE member, you need only register the
address of that just once, and unregister it in the destructor. As long
as your C/C++ object is around, the VALUE is registered and the object
it is tied to will not go away.That is exactly what I am doing I just wanted to spell out the scenario as
it may happen when there are more than one C++ wrapped reference to a ruby
object.I’ll try that volatile thing and see if it makes a difference.
Hmm … I’m confused about that statement.
If the VALUE is a member of a class, then it’s probably NOT on the
stack, so volatile won’t help you there. In this case, just use
rb_gc_register_address/rb_gc_unregister_address in your
constructor/destructor methods for your C++ class.
Sean O'Dell
Hmm … I’m confused about that statement.
Well that wont do…
If the VALUE is a member of a class, then it’s probably NOT on the
stack, so volatile won’t help you there. In this case, just use
rb_gc_register_address/rb_gc_unregister_address in your
constructor/destructor methods for your C++ class.
-
I am using rb_gc_register/unregister in my C++ wrapper.
-
I have now added the volatile keyword in all the functions where I use an
automatic variable.
Here’s the relevant bit of the C++ wrapper:
public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
_handle = ALLOC(VALUE);
*_handle = handle;
rb_gc_register_address(_handle);
}
~Object() {
rb_gc_unregister_address(_handle);
free(_handle);
}
// stuff for invoking ruby methods from .net removed
private:
VALUE __nogc *_handle;
};
Does it look reasonable?
Cheers,
Thomas
Thomas Sondergaard wrote:
Hmm … I’m confused about that statement.
Well that wont do…
If the VALUE is a member of a class, then it’s probably NOT on the
stack, so volatile won’t help you there. In this case, just use
rb_gc_register_address/rb_gc_unregister_address in your
constructor/destructor methods for your C++ class.
I am using rb_gc_register/unregister in my C++ wrapper.
I have now added the volatile keyword in all the functions where I use an
automatic variable.Here’s the relevant bit of the C++ wrapper:
public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
_handle = ALLOC(VALUE);
*_handle = handle;
rb_gc_register_address(_handle);
}~Object() { rb_gc_unregister_address(_handle); free(_handle); } // stuff for invoking ruby methods from .net removed private: VALUE __nogc *_handle;};
Does it look reasonable?
You don’t need to ALLOC the VALUE though. You can just make it a
regular member of your class and register the &address of it. When you
get the &address of a member variable, it’s true location in memory is
given, and as far as I know that won’t ever change during the lifetime
of your object, so you can really just do it like this:
public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
rb_gc_register_address(&_handle);
}
~Object() {
rb_gc_unregister_address(&_handle);
}
// stuff for invoking ruby methods from .net removed
private:
VALUE __nogc _handle;
};
Aside from that, it sounds like you have got the knack of it!
Sean O'Dell
You don’t need to ALLOC the VALUE though. You can just make it a
regular member of your class and register the &address of it. When you
get the &address of a member variable, it’s true location in memory is
given, and as far as I know that won’t ever change during the lifetime
of your object, so you can really just do it like this:
That need not be true for .net objects. If you hold references to .net
objects in native code you need to tell the CLR to pin the object.
Thomas
~Object() { rb_gc_unregister_address(_handle); free(_handle); }
I just thought of something. This .net finalizer is called asynchronously
and since ruby is not thread safe that might cause problem if ruby is
traversing the linked list that the finalizer above is removing an element
from. Tomorrow I will change the finalizer so it insteads adds the _handle
pointer to an ArrayList that I can read from the “ruby thread” when I feel
like it. A very good time to do this would be on the GC thread just before
the actual GC’ing starts.
Is there a gc_start_hook that I can call? I imagine something like this:
GC.gc_start_hook {
// calls rb_gc_unregister() for all the elements in the array list I
mentioned above
RubyDotNet::GcUtil.unregister_all()
}
Cheers,
Thomas
Thomas Sondergaard wrote:
You don’t need to ALLOC the VALUE though. You can just make it a
regular member of your class and register the &address of it. When you
get the &address of a member variable, it’s true location in memory is
given, and as far as I know that won’t ever change during the lifetime
of your object, so you can really just do it like this:That need not be true for .net objects. If you hold references to .net
objects in native code you need to tell the CLR to pin the object.
Heh … my last MS compiler is from MSVC 4.0, circa 1995. That hasn’t
been installed in probably 2 years or better, either.
Is there a Wiki I can go edit somewhere to include all these little
details regarding extensions, etc.? This stuff has really been tough to
get a handle on because all the formal documentation I’ve read in books
and online is all very simplified, and this stuff bites people in the
behind, but I’ve got a good handle on things and I’d really like to find
a place where I can go insert all this triviata.
Anyone have a URL to a somewhat official Wiki on this topic?
Sean O'Dell
Thomas Sondergaard wrote:
~Object() {
rb_gc_unregister_address(_handle);
free(_handle);
}I just thought of something. This .net finalizer is called asynchronously
and since ruby is not thread safe that might cause problem if ruby is
traversing the linked list that the finalizer above is removing an element
from. Tomorrow I will change the finalizer so it insteads adds the _handle
pointer to an ArrayList that I can read from the “ruby thread” when I feel
like it. A very good time to do this would be on the GC thread just before
the actual GC’ing starts.Is there a gc_start_hook that I can call? I imagine something like this:
GC.gc_start_hook {
// calls rb_gc_unregister() for all the elements in the array list I
mentioned above
RubyDotNet::GcUtil.unregister_all()
}
What you could do is create sort of a “global collector” object that is
a C++ object which collects VALUEs and “marks” them when the GC is invoked.
Create this object, and then wrap it with Data_Wrap_Struct. Register
the VALUE returned as a global, so the collection never goes away so
long as the Ruby environment is loaded.
In your constructors, add their VALUE members to the collector. In your
destructors, remove them. Block on a mutex for both actions.
When your collector object’s free is called, go ahead and free the
collection object; Ruby is going away anyway at that point, so there’s
little you can do about possible dangling VALUEs in living C++ objects.
When the mark function is called, block on the mutex and “mark” all the
VALUEs it is holding.
That will keep the VALUEs you are maintaining as members of your C++
objects around so long as the objects themselves are around. When their
destructor (aka finalizer) is called, they will tell the collector
object to stop marking them, and the GC will collect them eventually.
All access to the list of VALUEs that the collector manipulates blocks
on a single mutex, so it doesn’t matter what thread your objects “die” from.
Sean O'Dell
Is there a Wiki I can go edit somewhere to include all these little
details regarding extensions, etc.? This stuff has really been tough to
get a handle on because all the formal documentation I’ve read in books
and online is all very simplified, and this stuff bites people in the
behind, but I’ve got a good handle on things and I’d really like to find
a place where I can go insert all this triviata.
Excellent idea. I for one had many questions after having read the chapter
on extensions in the pick axe and perused the README.EXT from the
distribution.