Ruby 1.9.2-p0: segmenfault in multithreading when using a custom C extension

Hi, I've coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)... until now when I've tryed to add multithread logic to
handle the parser SIP message. It seems complex but I've been able to
create a very simple test script that generates the segmentfault.

To summarize, the following code DOES NOT crash:

···

-----------------------------------------------------------------------------
data = "INVITE sip:alice@example.org SIP/2.0\r\n\r\n"
parser = SIP::MessageParser.new

100000.times do
  msg = SIP::Message.new
  puts "msg.object_id = #{msg.object_id}"
  parser.execute(msg, data.to_str, 0)
  parser.reset

  t = Thread.new { }
end
-----------------------------------------------------------------------------

but the following code DOES crash:

-----------------------------------------------------------------------------
threads = [ ]
data = "INVITE sip:alice@example.org SIP/2.0\r\n\r\n"
parser = SIP::MessageParser.new

100000.times do
  msg = SIP::Message.new
  puts "msg.object_id = #{msg.object_id}"
  parser.execute(msg, data.to_str, 0)
  parser.reset

  threads << Thread.new { }
end
-----------------------------------------------------------------------------

The segmentfault occurs randomly in any iteration of the loop (usually
after 4900 iterations).
As you can see I do nothing within the thread. The only difference
between both cases is the fact that in the second case (the crashing
one) the thread is stored in an array.

And worse: I get segmentfault for MANY reasons (randomly too):

a) In line: threads << Thread.new { }
-- control frame ----------
c:0008 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :initialize
c:0007 p:---- s:0030 b:0030 l:000029 d:000029 CFUNC :new

b) In line: parser.execute(msg, data.to_str, 0)

c) When Message#initialize() executes "@timestamp = Time.now":
-- control frame ----------
c:0011 p:---- s:0039 b:0039 l:000038 d:000038 CFUNC :(null)
c:0010 p:---- s:0037 b:0037 l:000036 d:000036 CFUNC :now
c:0009 p:0017 s:0034 b:0034 l:000033 d:000033 METHOD /xxxx/message.rb:66

d) In line: puts "msg.object_id = #{msg.object_id}"

In other server with ruby 1.9.1p376 I've got, just once, a different
error (not a segmentfault in fact):

  in `execute': method `hash' called on terminated
  object (0x00000017c48ac8) (NotImplementedError)

Any help please? I cannot imagine the reason of this issue, neither
I'm sure that it's caused by my C extension.

Thanks a lot.

--
Iñaki Baz Castillo
<ibc@aliax.net>

The segment fault also occurs with Ruby 1.9.2-p136.

···

2011/1/11 Iñaki Baz Castillo <ibc@aliax.net>:

Hi, I've coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)... until now when I've tryed to add multithread logic to
handle the parser SIP message. It seems complex but I've been able to
create a very simple test script that generates the segmentfault.

--
Iñaki Baz Castillo
<ibc@aliax.net>

I've repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn't crash, so I expect some bug in my
C code :frowning:

···

2011/1/12 Iñaki Baz Castillo <ibc@aliax.net>:

2011/1/11 Iñaki Baz Castillo <ibc@aliax.net>:

Hi, I've coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)... until now when I've tryed to add multithread logic to
handle the parser SIP message. It seems complex but I've been able to
create a very simple test script that generates the segmentfault.

The segment fault also occurs with Ruby 1.9.2-p136.

--
Iñaki Baz Castillo
<ibc@aliax.net>

I think I've found the problem! and it could be related to a bug in Ruby:

In my SIP parser (Ruby C extension like Ragel HTTP parser) I generate
Ruby string encoded in UTF-8 using:

  rb_enc_str_new(s, len, rb_utf8_encoding());

But if I change it to:

  rb_str_new(s, len)

then the segment fault does NEVER occur!

···

2011/1/12 Iñaki Baz Castillo <ibc@aliax.net>:

I've repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn't crash, so I expect some bug in my
C code :frowning:

--
Iñaki Baz Castillo
<ibc@aliax.net>

In order to demostrate the problem I've reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

···

2011/1/12 Iñaki Baz Castillo <ibc@aliax.net>:

I think I've found the problem! and it could be related to a bug in Ruby:

In my SIP parser (Ruby C extension like Ragel HTTP parser) I generate
Ruby string encoded in UTF-8 using:

rb_enc_str_new(s, len, rb_utf8_encoding());

But if I change it to:

rb_str_new(s, len)

then the segment fault does NEVER occur!

--
Iñaki Baz Castillo
<ibc@aliax.net>

Hello Iñaki,

"Iñaki Baz Castillo" <ibc@aliax.net> wrote in post #974123:

In order to demostrate the problem I've reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

Thanks for taking the time to file that issue, along with instructions
on how to reproduce the problem.

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

Cheers!

···

2011/1/12 Iñaki Baz Castillo <ibc@aliax.net>:

--
Posted via http://www.ruby-forum.com/\.

Hi Suraj, thanks a lot for your response :slight_smile:

BTW I've tryed the test script with ruby-1.9.1-p376 and it DOES NOT crash!

···

2011/1/12 Suraj Kurapati <sunaku@gmail.com>:

"Iñaki Baz Castillo" <ibc@aliax.net> wrote in post #974123:

2011/1/12 Iñaki Baz Castillo <ibc@aliax.net>:
In order to demostrate the problem I've reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

Thanks for taking the time to file that issue, along with instructions
on how to reproduce the problem.

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

--
Iñaki Baz Castillo
<ibc@aliax.net>