Rb_funcall() Ruby code callback invoked from within a native thread?

ruby-talk
1

Dear All,

I have created a Ruby extension that runs under both Windows (Ruby
1.8.4) and Linux (Ruby 1.8.5), using SWIG. The extension is written in
C and is multi-threaded, thanks to PThreads. Several C functions are
exported to Ruby to perform various operations.

One of the C functions is a callback notifier whose usage is as
follows:

1) The main Ruby program must first invoke a C native function:
Libext.ext_notifier_add(dhtid, proc, "method@class", 0) where the
"method@class" is a dummy used normally for the Java version of the
extension, and the main parameter is the proc address corresponding to
a Ruby block like: proc = Proc.new { | value | Libext.ext_put(id, key,
value, index) } . This registers a Ruby callback with the C extension.

2) The main Ruby program will also invoke C native functions that will
result in PThreads being created to receive and process UDP messages.
On average, some 50 PThreads are active at any time inside the C
extension. There are many separate instances of the Ruby program
(processes) running simultaneously on the same or different (networked)
computers.

3) When a Ruby program calls Libext.ext_put(id, key, value, index) a
message will be sent to another node. On the destination node upon
receiving the message some parameters are evaluated and if applicable
the registered Ruby callback is invoked with:

rb_thread_critical = Qtrue;
res = rb_funcall(proc, rb_intern("call"), 4, rb_str_new2(triggerstr),
rb_str_new2(evalstr), INT2NUM(index), LONG2NUM(state->id));
rb_thread_critical = 0;

4) The callback will normally contain only fairly simple statements but
may as well need to invoke C native functions (as in the example
above).

So far I have seen the following problems with this solution:

a) cross-thread violation on rb_thread_schedule() : this happens on the
Windows version of the C extension when NO native function are invoked
from the callback and rb_thread_critical = Qtrue; is not used.

Under heavy load of incoming UDP messages from other nodes, the
callback execution will produce that error. It seems that the Ruby
scheduler is preempting the execution of the Ruby callback and
determines that a native thread has invoked it (a bad thing it seems).

Adding rb_thread_critical = Qtrue; makes it impossible for the
scheduler to change the thread of execution and eliminates the error
except for once every several hundred callbacks under sustained load.

b) SystemStackError: stack level too deep : this happens exclusively on
Linux (Debian) after only a few callbacks when NO native function are
invoked and rb_thread_critical = Qtrue; is used. The C extension uses a
fairly deeply nested function calls (with several layers of callbacks).
I assume that Ruby somehow (how?) detects that the stack has grown past
permitted levels (which are?) and throw that error message.

Now coming to the main issue... I would like to get rid of problem b)
and one possible solution that occurred to me is to create a native
PThread that will perform the Ruby callback and exit when all Ruby and
native method/function calls have completed. This seems possible under
Java JNI through the use of AttachCurrentThread() that converts
temporarily a native thread into a Java thread.

However, I did not discover any similar function in Ruby. Besides, I
read plenty of comments that mixing native threads with the Ruby
interpreter is a bad idea. As a sideline, all the testing has been done
from the command line using ruby.exe, although under Windows I will
soon have to migrate to Active Ruby.

Being new to all this Ruby internal coding I would like to sollicit
your feeback and advice on how I could tackle this problem. I hope I
have overlooked something that will make my life easier :slight_smile: Overall,
Ruby has proven a fantastic language to develop on but I fear that
compared to JNI the support for native threads is severely lacking :frowning:

Cheers,

Serge Kruppa
CTO - Simitel

2

You're making some assumptions here which I think are challenging. Whenever
your Ruby code calls native code, Ruby will block until the native call
returns. If the native call makes calls back into Ruby, you're generally
safe. But if you spin native threads that call Ruby code (as you suggest
here) then you will have no way to completely synchronize the Ruby code that
is now running on multiple native threads.

There have been several discussions on this list about how to send events
back and forth between Ruby and native code running on native threads. The
ideas which seem to work generally involve sending messages on socketpairs
or pipes. A look through the archives may help you.

If you have the freedom to re-architect your application, look at the
EventMachine library. It was designed to enable applications like yours to
be implemented without threads.

···

On 10/4/06, Serge Kruppa <serge.kruppa@simitel.com> wrote:

Dear All,

I have created a Ruby extension that runs under both Windows (Ruby
1.8.4) and Linux (Ruby 1.8.5), using SWIG. The extension is written in
C and is multi-threaded, thanks to PThreads. Several C functions are
exported to Ruby to perform various operations.

One of the C functions is a callback notifier whose usage is as
follows:

3

If you have the freedom to re-architect your application, look at the
EventMachine library. It was designed to enable applications like yours to
be implemented without threads.

Actually, the EventMachine library looks like a fantastic solution!

Do you think it would be wise to implement a Ruby to C RPC mechanism on top
of it? Thus instead of using native calls I would invoke an RPC stub from
Ruby that will deliver a UDP packet with the native function name and
parameters to a PThread that will in turn call the C function. Any result
from C would be passed back to Ruby using the same channel, allowing the
Ruby code to synchronize using traditional methods and avoid an utter
threading mess.

4

Without knowing enough about your application, my reaction to your question
is this: it seems like you have plenty of good code already written in C for
handling what I assume are requests-for-service that come to you in UDP
packets. Why not wrap your C code in a Ruby extension (generally an easy
thing to do) and then have an EventMachine protocol handle simply call your
C code with the data from the sockets? That way your C code doesn't need to
know or care that network communications are even involved, and you can Ruby
to do all the format-massaging, which Ruby is really good at.

What's the execution profile of your C-code? Does it block locally on things
like database calls? If so, then you should use EM's patterns for deferred
execution. If not, then it's real easy and straighforward.

···

On 10/4/06, Serge Kruppa <serge.kruppa@simitel.com> wrote:

Do you think it would be wise to implement a Ruby to C RPC mechanism on
top
of it? Thus instead of using native calls I would invoke an RPC stub from
Ruby that will deliver a UDP packet with the native function name and
parameters to a PThread that will in turn call the C function. Any result
from C would be passed back to Ruby using the same channel, allowing the
Ruby code to synchronize using traditional methods and avoid an utter
threading mess.

5

Without knowing enough about your application, my reaction to your question
is this: it seems like you have plenty of good code already written in C
for handling what I assume are requests-for-service that come to you in UDP
packets. Why not wrap your C code in a Ruby extension (generally an easy
thing to do) and then have an EventMachine protocol handle simply call your
C code with the data from the sockets? That way your C code doesn't need to
know or care that network communications are even involved, and you can
Ruby to do all the format-massaging, which Ruby is really good at.

What's the execution profile of your C-code? Does it block locally on
things like database calls? If so, then you should use EM's patterns for
deferred execution. If not, then it's real easy and straighforward.

An issue we have is that all our code must run in an embedded Ruby
environment (Active Ruby), packaged as a traditional Windows application
(MFC). Ruby is the hidden jewel inside a fairly standard Windows program. If
I understood your suggestion correctly, the idea would be to layer an
EventMachine on top of our C code already packaged as a Ruby extension. The
EM would act as a UDP event-driven middleware, passing function requests and
responses back and forth between our main Ruby program and the C code? How
do you suggest we interface the EM with the main Ruby program?

Our Ruby program structure is not event driven unfortunately. I did convert
during the night the C native function that invokes the Ruby callbacks to a
simple UDP message send with a specially formatted text string containing
the parameters. The EM receives these UDP callback notifications very
effectively and passes them on to actual Ruby Proc objects for further
processing.

However, taking the example of my unit test suite, I'm using a simple fall
through logic of test cases executed in logical sequence. I found it
impossible to naively insert the EM into my test suite (I'm ashamed to admit
that I tried to run the EM within a Thread, which resulted in the EM
catching only the first UDP callback and missing all the subsequent ones).

Regarding your question about the execution profile of our C code, it is
mostly RPC style: one computer sends a request to another and blocks on a
PThread condition variable until a response is received from the other end
or a timeout occurs. We do have database operations as well, protected by a
series of PThread mutexes.

I have been looking for more information about the EM deferred execution
pattern but did not find any straightforward description. Any pointers to
suitable Web pages would be most appreciated :slight_smile:

6

An issue we have is that all our code must run in an embedded Ruby
environment (Active Ruby), packaged as a traditional Windows application
(MFC). Ruby is the hidden jewel inside a fairly standard Windows program.
If
I understood your suggestion correctly, the idea would be to layer an
EventMachine on top of our C code already packaged as a Ruby extension.
The
EM would act as a UDP event-driven middleware, passing function requests
and
responses back and forth between our main Ruby program and the C code? How
do you suggest we interface the EM with the main Ruby program?

Our Ruby program structure is not event driven unfortunately. I did
convert
during the night the C native function that invokes the Ruby callbacks to
a
simple UDP message send with a specially formatted text string containing
the parameters. The EM receives these UDP callback notifications very
effectively and passes them on to actual Ruby Proc objects for further
processing.

The best way to structure an event-driven program (or a traditional program
that responds to events) depends on that factors that influence the
program's control flow. It's always easiest if you can let EM run your
program, meaning that all of your program's work is done by responding to
timers, signals, or exogenous events. However, your program may have a GUI
or very-long-running batch jobs that don't like to be interrupted. In these
cases, you can run EM on its own Ruby thread. Bear in mind that all of the
code you write that responds to external events controlled by EM will then
be executed on the separate EM thread, and you'll need to synchronize this
code (using Ruby sync objects) with your main line. But EM works seamlessly
with Ruby threads, so none of this is too difficult. Another alternative is
to separate your program into multiple processes, with the traditional logic
in one piece and the event-driven stuff in a different one.

However, taking the example of my unit test suite, I'm using a simple fall

through logic of test cases executed in logical sequence. I found it
impossible to naively insert the EM into my test suite (I'm ashamed to
admit
that I tried to run the EM within a Thread, which resulted in the EM
catching only the first UDP callback and missing all the subsequent ones).

It's definitely messy to unit-test event-driven code because test suites by
nature want to manage their own sequential logic. What I generally do is to
call EventMachine#run inside each test function in a test suite, and have
one of my event handlers call EventMachine#stop as soon as the interesting
code has executed. This works quite well. If you're afraid that a bug in
your code on any given test run will prevent #stop from being called (and
hang your test run), then just add an EM timer that will call stop after one
second and then throw an exception that your test suite will respond to.

Regarding your question about the execution profile of our C code, it is

mostly RPC style: one computer sends a request to another and blocks on a
PThread condition variable until a response is received from the other end
or a timeout occurs. We do have database operations as well, protected by
a
series of PThread mutexes.

I have been looking for more information about the EM deferred execution
pattern but did not find any straightforward description. Any pointers to
suitable Web pages would be most appreciated :slight_smile:

Now I see why you typically have 50 threads, it's because that's how many
pending requests you have in flight at any given time. Doing this with UDP
can be an interesting challenge for a couple of different reasons, but I've
found that there generally is a simple and graceful way to build these. This
application pattern is fundamentally event-driven so I think you'll find
that a tool like EM will work with you, whereas a threaded approach will
generally work against you. If you want design hints, feel free to write me
privately and give me some more detail on how these "RPCs" work.

About deferred operations: there are two kinds of these, EventMachine#defer
and the Deferrable module. The first is straightforward and was designed to
deal with cases that block your program locally, the classic example being a
database call using a DB library that is not event-ready. EventMachine#defer
manages an internal pool of Ruby threads. Read the rdoc for #defer, there is
complete information there and sample code.

I've found that the Deferrable pattern is rather hard for people to
understand (and it's not well documented yet). It's very much like the
Deferred objects in Python's Twisted. It basically allows you to kick off an
event-driven operation that may (or may not) take some time to execute, but
bundle in the processing that will be executed on success or failure. If can
give ridiculously high performance for things like accessing Web Services or
other HTTP servers if you use it right. Examples: I've used it to open 1000
simultaneous connections to servers in a LAN in about two-tenths of a
second; 500 completed HTTP GETs to Internet hosts in less than a second, and
so on.

···

On 10/5/06, Serge Kruppa <serge.kruppa@simitel.com> wrote:

7

Hello Francis,

Please allow me to sidestep for a moment on our current thread of discussion
to report three issues we have detected with EventMachine on Windows:

1) The latest Windows binary gem posted on RubyForge cannot be successfully
installed and comes back with the following error message:

ERROR: Error installing gem eventmachine-0.5.3-mswin32[.gem]: buffer error

2) I have posted on the RubyForge EventMachine bug tracker a possible bug in
building EventMachine from the sources using Visual Studio Express 2005.

3) Another person in the team reported the same problem under Windows
(EventMachine 0.5.2) that I saw under Linux when running EM inside a thread:

There is one serious issue left. The Event Machine udp server that is

used to replace the callbacks from the C native code to the ruby code dies
after processing just one message. Apparently this has to do with the
necessity of it running in the main thread. The main thread in the windows
program is however used by the windows gui event loop. Probably the best
approach is to use a simple udp server, just like in the java version.

I'm sure I should have posted this in the EventMachine-Talk list but I just
can't get access to the Web page that is supposed to validate me :frowning:

Best Regards,

Serge

8

We solved the binary-gem issues with Windows quite a while ago, but have yet
to release new binary gems. I'll send you one by private email. Some new
features went into EM this week so we really need to get a new release up
there. (Unfortunately rubyforge is down at the moment.)

Once Rubyforge is back, you can sync to the current EM source code and try
your Windows compilation. I know we've tested successfully on VC7, VC2003
and 2005 Express. However (and I'm not the biggest expert on this subject-
Austin, are you reading this?) I think the consensus view is that Ruby
extensions should only be compiled on VC6 in order to stay compatible with
Ruby itself. What that means for EM on Windows is that published binary gems
(which are always compiled with VC6) will be the way to go. Especially since
production Windows servers generally don't have compilers installed.

Another thing we've done recently is to add a pure-Ruby implementation of
the EM functionality. EM code will degrade transparently to pure-Ruby
whenever you run it in an environment that doesn't have the compiled
extension for whatever reason. Performance is roughly half as good as
compiled with really large loads. On small loads you might not notice a
difference.

3) Another person in the team reported the same problem under Windows

(EventMachine 0.5.2) that I saw under Linux when running EM inside a
thread:

> There is one serious issue left. The Event Machine udp server that is
used to replace the callbacks from the C native code to the ruby code dies
after processing just one message. Apparently this has to do with the
necessity of it running in the main thread. The main thread in the windows
program is however used by the windows gui event loop. Probably the best
approach is to use a simple udp server, just like in the java version.

I want to know more about this UDP-server consuming only one message. It's
probably a simple coding problem. Can you send me a test case by private
email that exhibits the problem?

···

On 10/6/06, Serge Kruppa <serge.kruppa@simitel.com> wrote:

9

Serge, thanks for sending me your code snippets. In regard to the problem
you're having with EM only processing one UDP message on Windows if you run
it in a thread: this is caused by the $stdin.getc call you were using to
keep your main loop alive, and has nothing to do with EM. This problem has
come up on this list before but I'm not aware that anyone has resolved it.
Does anyone on the list know?

This code does what you expect on Unix but not on Windows (tested with 1.8.4,
Christmas 2005 build from the 1-click installer):

···

On 10/6/06, Serge Kruppa <serge.kruppa@simitel.com> wrote:

> There is one serious issue left. The Event Machine udp server that is
used to replace the callbacks from the C native code to the ruby code dies
after processing just one message. Apparently this has to do with the
necessity of it running in the main thread. The main thread in the windows
program is however used by the windows gui event loop. Probably the best
approach is to use a simple udp server, just like in the java version.

#---------------------------------------------------------
Thread.new {
  loop {
    sleep 1
    p Time.now
  }
}

$stdin.getc
#--------------------------------------------------------