During load testing of our Ruby program, we have been getting
segmentation faults fairly consistently (but at unpredictable times)
after the program has been running for awhile under heavy load.
From the stack trace, it appears that Ruby is in garbage collection
at the time, specifically in rb_gc_mark_frame(frame) which calls
mark_locations_array(x, n) with the frame’s argv and argc.
The value of argc in these calls is typically a large negative number,
suggesting that the frame’s argv and argc are uninitialized.
The top of the stack trace is appended below.
We’re running Ruby 1.6.7 with WEBrick (using 20 threads) on RedHat Linux
7.2.
(We started with Ruby 1.7.2, but when we hit this problem,
we tried downgrading to 1.6.7.) Any help would be much appreciated!
Thanks,
Jim
#0 0x400a9a01 in __kill () from /lib/i686/libc.so.6 #1 0x400a97da in raise (sig=6) at …/sysdeps/posix/raise.c:27 #2 0x400aaf82 in abort () at …/sysdeps/generic/abort.c:88 #3 0x080b1eca in rb_bug () at error.c:178 #4 #5 mark_locations_array (x=0x40263ef8, n=-1074526067) at gc.c:354 #6 0x08066ef9 in rb_gc_mark_frame (frame=0x9b9814c) at gc.c:912 #7 0x0805f534 in thread_mark (th=0x870a1d8) at eval.c:7213 #8 0x0806697f in rb_gc_mark (ptr=0x40478838) at gc.c:634 #9 0x0806653c in mark_locations_array (x=0x8edf3c8, n=4250) at gc.c:378 #10 0x0805f431 in thread_mark (th=0x89a93c8) at eval.c:7188 #11 0x0806697f in rb_gc_mark (ptr=0x40478644) at gc.c:634 #12 0x0806653c in mark_locations_array (x=0x9162560, n=17074) at gc.c:378 #13 0x0805f431 in thread_mark (th=0x89b3ae8) at eval.c:7188 #14 0x0806697f in rb_gc_mark (ptr=0x40478464) at gc.c:634 #15 0x080669fb in rb_gc_mark (ptr=0x404c1998) at gc.c:667 #16 0x0806653c in mark_locations_array (x=0x9b81390, n=15691) at gc.c:378 #17 0x0805f431 in thread_mark (th=0x98ba280) at eval.c:7188 #18 0x0806697f in rb_gc_mark (ptr=0x40485d94) at gc.c:634 #19 0x0806653c in mark_locations_array (x=0x8b65988, n=18580) at gc.c:378 #20 0x0805f431 in thread_mark (th=0x8a0e138) at eval.c:7188 #21 0x0806697f in rb_gc_mark (ptr=0x40478fb8) at gc.c:634 #22 0x08066937 in rb_gc_mark (ptr=0x404794f4) at gc.c:617 #23 0x080669bb in rb_gc_mark (ptr=0x4048f04c) at gc.c:656 #24 0x0806653c in mark_locations_array (x=0x8b18580, n=7904) at gc.c:378 #25 0x0805f431 in thread_mark (th=0x897fa60) at eval.c:7188 #26 0x0806697f in rb_gc_mark (ptr=0x40476f24) at gc.c:634 #27 0x0806653c in mark_locations_array (x=0x91995f0, n=21263) at gc.c:378 #28 0x0805f431 in thread_mark (th=0x847b680) at eval.c:7188 #29 0x0806697f in rb_gc_mark (ptr=0x4047915c) at gc.c:634 #30 0x0806653c in mark_locations_array (x=0x91c4ea0, n=18859) at gc.c:378 #31 0x0805f431 in thread_mark (th=0x8975340) at eval.c:7188 #32 0x0806697f in rb_gc_mark (ptr=0x40477140) at gc.c:634 #33 0x08066937 in rb_gc_mark (ptr=0x40263ed0) at gc.c:617 #34 0x0806658e in mark_entry (key=10514, value=1076248272) at gc.c:405 #35 0x0809ccf9 in st_foreach (table=0x820c390, func=0x8066580
<mark_entry>, arg=0x0) at st.c:488 #36 0x080665af in rb_mark_tbl (tbl=0x820c390) at gc.c:414 #37 0x08066993 in rb_gc_mark (ptr=0x40263ef8) at gc.c:638 #38 0x0806653c in mark_locations_array (x=0x8cc2768, n=20859) at gc.c:378 #39 0x0805f431 in thread_mark (th=0x89be208) at eval.c:7188
In message “Segmentation Fault - Ruby Garbage Collection” on 02/07/03, James Eric Birmingham avenger@wpi.edu writes:
We’re running Ruby 1.6.7 with WEBrick (using 20 threads) on RedHat Linux
7.2.
(We started with Ruby 1.7.2, but when we hit this problem,
we tried downgrading to 1.6.7.) Any help would be much appreciated!
Do you mean you don’t get the problem with 1.7.2?
Could you provide the script you used if it’s possible?
The problem did originally occur with 1.7.2. Then we tried downgrading to
1.6.7 but the problem still occurred. Unfortunately we can’t include the
code since it is company sensitive and there is a lot of it (3000+
lines). We will try to duplicate the problem with a smaller script but
this is going to take sometime.
-Jim
···
On Wed, 3 Jul 2002, Yukihiro Matsumoto wrote:
Hi,
In message “Segmentation Fault - Ruby Garbage Collection” > on 02/07/03, James Eric Birmingham avenger@wpi.edu writes:
We’re running Ruby 1.6.7 with WEBrick (using 20 threads) on RedHat Linux
7.2.
(We started with Ruby 1.7.2, but when we hit this problem,
we tried downgrading to 1.6.7.) Any help would be much appreciated!
Do you mean you don’t get the problem with 1.7.2?
Could you provide the script you used if it’s possible?
Has there been any conclusion to this problem yet? I am experiencing
exactly the same problem, i.e., before the garbage collector comes
into action, my code is fine, but after the garbage collector works (I
do a printf when the mark function is called), then I get a
segmentation fault. I am using Ruby 1.6.7 on Mandrake Linux with
kernel 2.4.8-26mdk.
The strange thing is that the segmentation fault still occurs even
after I modify the mark function to do nothing (immediately returns
void). Just for testing, is there a way to disable the garbage
collector? Thanks.
Has there been any conclusion to this problem yet? I am experiencing
Yes,
exactly the same problem, i.e., before the garbage collector comes
into action, my code is fine, but after the garbage collector works (I
do a printf when the mark function is called), then I get a
segmentation fault.
This, generally, means that you have a bug in your extension.
Difficult to say more without the source of your extension.
The strange thing is that the segmentation fault still occurs even
after I modify the mark function to do nothing (immediately returns
void). Just for testing, is there a way to disable the garbage
collector? Thanks.
After I searched the comp.lang.ruby discussions further, I also found
out that I got the concept backwards; I thought that the object that
is marked is marked for deletion, as discussed in a thread with title
“Help: Ruby<->C++ callbacks” in about last April.
Now, clearly the symptom is like this. If I disable garbage collecter
by invoking “GC.disable”, my program runs without any problem.
Otherwise, I get segmentation fault. Does this give a clue that the
problem is very likely to be in my mark functions? Are there any more
rules on how to invoke rb_gc_mark in the mark functions beside just
calling rb_gc_mark for every Ruby objects inside a structure? The
function prototype in Ruby 1.6.7 is rb_gc_mark (void) (what is the
correct prototype? And should I recompile my Ruby installation?); can
this be a source of problem?
The strange thing is that the segmentation fault still occurs even
after I modify the mark function to do nothing (immediately returns
void). Just for testing, is there a way to disable the garbage
collector? Thanks.
In message “Re: Segmentation Fault - Ruby Garbage Collection” on 02/07/31, Bill Tj billtj@glue.umd.edu writes:
Now, clearly the symptom is like this. If I disable garbage collecter
by invoking “GC.disable”, my program runs without any problem.
Otherwise, I get segmentation fault. Does this give a clue that the
problem is very likely to be in my mark functions? Are there any more
rules on how to invoke rb_gc_mark in the mark functions beside just
calling rb_gc_mark for every Ruby objects inside a structure? The
function prototype in Ruby 1.6.7 is rb_gc_mark (void) (what is the
correct prototype? And should I recompile my Ruby installation?); can
this be a source of problem?
Thanks for your response. Actually, after tracing and printing all
the variable addresses, I concluded that my mark functions are fine.
Finally I found out that the segmentation fault happenned because I
have some C global variables in my extension that are of type VALUE
and free floating. Apparently they got swept by the garbage
collector. After I put them under “rb_define_readonly_variable”, my
program works without any problem as expected.
However, using “rb_define_readonly_variable” still exposes my C global
variables to the user, although they are read only. Is there any way
to hide them completely from the user? I.e., is there any other easy
way to mark the global variables?
In message “Re: Segmentation Fault - Ruby Garbage Collection” > on 02/07/31, Bill Tj billtj@glue.umd.edu writes:
Now, clearly the symptom is like this. If I disable garbage collecter
by invoking “GC.disable”, my program runs without any problem.
Otherwise, I get segmentation fault. Does this give a clue that the
problem is very likely to be in my mark functions? Are there any more
rules on how to invoke rb_gc_mark in the mark functions beside just
calling rb_gc_mark for every Ruby objects inside a structure? The
function prototype in Ruby 1.6.7 is rb_gc_mark (void) (what is the
correct prototype? And should I recompile my Ruby installation?); can
this be a source of problem?
However, using "rb_define_readonly_variable" still exposes my C global
variables to the user, although they are read only. Is there any way
to hide them completely from the user? I.e., is there any other easy
way to mark the global variables?
Use rb_global_variable() see README.EXT
** Notice
GC should know about global variables which refers Ruby's objects, but
not exported to the Ruby world. You need to protect them by
In message “Re: Segmentation Fault - Ruby Garbage Collection” on 02/07/31, Bill Tj billtj@glue.umd.edu writes:
However, using “rb_define_readonly_variable” still exposes my C global
variables to the user, although they are read only. Is there any way
to hide them completely from the user? I.e., is there any other easy
way to mark the global variables?
Thanks for the info (and also thanks to Guy who replied earlier).
I am just curious. For the attribute, we have only a single function,
“rb_define_attr (variable, name, read, write)”, and we can control
accessability using the read and write arguments. Why for global
variables we have three functions with unsimilar names?
rb_define_variable - read and write
rb_define_readonly_variable - read only
rb_global_variable - no read and no write
Isn’t it cleaner if we also have only a single function such as
“rb_define_variable (name, value, read, write)”?
In message “Re: Segmentation Fault - Ruby Garbage Collection” on 02/08/01, William Djaja Tjokroaminata billtj@z.glue.umd.edu writes:
I am just curious. For the attribute, we have only a single function,
“rb_define_attr (variable, name, read, write)”, and we can control
accessability using the read and write arguments. Why for global
variables we have three functions with unsimilar names?
rb_define_variable - read and write
rb_define_readonly_variable - read only
rb_global_variable - no read and no write
Isn’t it cleaner if we also have only a single function such as
“rb_define_variable (name, value, read, write)”?
I could read your mind. So I prepared you
rb_define_hooked_variable(name, var, read, write)
But still, rb_global_variable is a different beast. Unlike others,
its “variable” means “C variables”, which is not seen from Ruby.
While we are on the subject, is it a good idea to dig deeper into the Ruby
C implementation from the interface point of view?
What I mean is, for the Ruby language itself, I think you will make effort
so that the Ruby API is backward compatible in the future releases. What
about the RUBY C API? Are you going to make effort so that they are
backward compatible too?
Also, is there any defined set of “official RUBY C API”? Is it only all
the functions/macros/structs listed in ruby.h and intern.h?
The reason is, for efficiency purposes, sometimes I may want to get to the
underlying C function directly, instead of calling “rb_funcall”. Probably
to be efficient, I have to define my own structures and
functions; however, often this is difficult as Ruby requires all
user-accessible objects to be of type VALUE.
What I mean is, for the Ruby language itself, I think you will make effort
so that the Ruby API is backward compatible in the future releases. What
about the RUBY C API? Are you going to make effort so that they are
backward compatible too?
The future Ruby (Rite) will not have compatible C API, but I plan to
provide the compatibility layer if possible.
Also, is there any defined set of “official RUBY C API”? Is it only all
the functions/macros/structs listed in ruby.h and intern.h?
No and yes.
The reason is, for efficiency purposes, sometimes I may want to get to the
underlying C function directly, instead of calling “rb_funcall”. Probably
to be efficient, I have to define my own structures and
functions; however, often this is difficult as Ruby requires all
user-accessible objects to be of type VALUE.
I’m not sure why it is difficult. Could you tell me how difficult?
matz.
···
In message “Re: Segmentation Fault - Ruby Garbage Collection” on 02/08/01, William Djaja Tjokroaminata billtj@y.glue.umd.edu writes:
Probably the most direct problem that I am currently facing is, I will
have many entities that will be created and destroyed so often (millions
of times) such as packets and events. If the user is not going to have
access to these entities, I can create efficient structures for them (even
probably manage the memory usage.) However, when a user is allowed to
interact with them, I am forced to make the packets and events as Ruby
objects (to be honest, I don’t know how much overhead the Ruby objects
have as compared to straight C structs; I have not made comparison yet.)
I am thinking whether it is possible to design some “lazy evaluation” type
of object. In other words, when the user does not interact with them,
they are straight C structs so that the code is highly efficient. Once a
user interact with an entity, however, the struct creates a Ruby object
for the user to interface.
Another (simpler) example is when I want to reuse Ruby array in C, it
forces all the entities to be Ruby objects, because a Ruby array, I think,
cannot hold objects other than Ruby objects. So if I just want to have a
(dynamic) array of double’s, I am forced to create objects of type
rb_cFloat instead of straight C double’s. (Of course, in this case the
solution is to use other C libraries.)
The reason is, for efficiency purposes, sometimes I may want to get to the
underlying C function directly, instead of calling “rb_funcall”. Probably
to be efficient, I have to define my own structures and
functions; however, often this is difficult as Ruby requires all
user-accessible objects to be of type VALUE.
I’m not sure why it is difficult. Could you tell me how difficult?
Probably the most direct problem that I am currently facing is, I will
have many entities that will be created and destroyed so often (millions
of times) such as packets and events. If the user is not going to have
access to these entities, I can create efficient structures for them (even
probably manage the memory usage.) However, when a user is allowed to
interact with them, I am forced to make the packets and events as Ruby
objects (to be honest, I don’t know how much overhead the Ruby objects
have as compared to straight C structs; I have not made comparison yet.)
5 words (20 bytes on int32 platforms) for each objects.
I am thinking whether it is possible to design some “lazy evaluation” type
of object. In other words, when the user does not interact with them,
they are straight C structs so that the code is highly efficient. Once a
user interact with an entity, however, the struct creates a Ruby object
for the user to interface.
I now understand what you want. But I’m not sure if it’s possible.
Probably the solution is wrapping your set of C data as a “lazy
evaluating” Ruby object which wraps your flyweight C data only when
the user starts interact with them.
Another (simpler) example is when I want to reuse Ruby array in C, it
forces all the entities to be Ruby objects, because a Ruby array, I think,
cannot hold objects other than Ruby objects. So if I just want to have a
(dynamic) array of double’s, I am forced to create objects of type
rb_cFloat instead of straight C double’s. (Of course, in this case the
solution is to use other C libraries.)
NumArray? See RAA.
matz.
···
In message “Re: Segmentation Fault - Ruby Garbage Collection” on 02/08/02, William Djaja Tjokroaminata billtj@y.glue.umd.edu writes:
Thanks for the information. Actually I am more concerned with the
execution overhead instead of memory overhead (I think memory is
relatively cheap these days); besides, as the entities are created and
destroyed so often, there will not be too many of them at any instant of
time.
In straight Ruby, one of my code has efficiency of about 30,000
events/second. When I transformed the core into C (so the input is still
in Ruby), the efficiency increases to about 130,000 events/second. I can
only think of either Ruby is indeed relatively efficient (compared to C),
or, because the core still has to deal with Ruby objects, the Ruby objects
are the bottleneck, which may include:
1) Object creation, initialization, and checking from time to time
(I have changed it from Data_Get_Struct to direct DATA_PTR)
2) Garbage collection, which runs occasionally but has to trace all
the Ruby objects
3) Ruby function call overhead, which includes “rb_funcall” and hash
access (which is not as fast as struct member access)
4) Parsing overhead (however, although my code is one big loop, I
guess this parsing overhead is one time only at the beginning)
This is why I am thinking on the lazy evaluation model.
Probably the most direct problem that I am currently facing is, I will
have many entities that will be created and destroyed so often (millions
of times) such as packets and events. If the user is not going to have
access to these entities, I can create efficient structures for them (even
probably manage the memory usage.) However, when a user is allowed to
interact with them, I am forced to make the packets and events as Ruby
objects (to be honest, I don’t know how much overhead the Ruby objects
have as compared to straight C structs; I have not made comparison yet.)
5 words (20 bytes on int32 platforms) for each objects.
Because I am dealing with entities which are created and destroyed so
often (such as 224,000 events/second), and I think the bottleneck is the
Ruby object creation, checking, and garbage collection, I am thinking of
the following Ruby-C interface model.
All my events will be straight C struct. I will create only one (or a
few) Ruby event object, which contains function pointers. Everytime I am
returning event VALUE to the user, I just return this “actually
blank” Ruby object. Only when user starts to interact with this Ruby
event object, some C functions are called that actually return the real
data.
My question is, is this a reasonable model from Ruby philosophy point of
view? Is there any danger in doing it this way? Basically it is a Ruby
object with varying C data (consisting of C function pointers and void
pointers).
I am thinking whether it is possible to design some “lazy evaluation” type
of object. In other words, when the user does not interact with them,
they are straight C structs so that the code is highly efficient. Once a
user interact with an entity, however, the struct creates a Ruby object
for the user to interface.
I now understand what you want. But I’m not sure if it’s possible.
Probably the solution is wrapping your set of C data as a “lazy
evaluating” Ruby object which wraps your flyweight C data only when
the user starts interact with them.
matz.
You might consider reusing objects, rather than creating and destroying
them every time. There are also design patterns, such as Flyweight, that
you might be able to apply to help with this.
cjs
···
On Fri, 2 Aug 2002, William Djaja Tjokroaminata wrote:
Probably the most direct problem that I am currently facing is, I will
have many entities that will be created and destroyed so often (millions
of times) such as packets and events.