Problem wrapping the C++ STL

Hi all,

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

#include "ruby.h"
#include <vector>
using namespace std;

struct cvector{
    vector<VALUE> array(0);
};

void Init_cvector(){
    VALUE cVector = rb_define_class("CVector", rb_cObject);
}

When I try to compile this (using g++ on OS X) I get:

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of enum 'rb_thread_status' without previous declaration
/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid type in declaration before ';' token
cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant
cvector.c: In function 'void Init_cvector()':
cvector.c:10: warning: unused variable 'cVector'
make: *** [cvector.o] Error 1

Thanks,

Dan

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

Yes. I wrapped set for instance

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
enum 'rb_thread_status' without previous declaration

This is not a C++ bug. This is a bug in 1.8.6-p36, for which I know no
workaround. rb_thread_status is defined in node.h but ruby.h inclu

cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant

You're trying to initialize a struct field. You need to do
struct cvector {
  vector<VALUE> array;
};
an resize the array if needed.

Note that you can wrap std::vector directly (no need to go through an
intermediate cvector structure)

Sylvain

Have you tried swig? Gonzalo Garramuño has done some work to make the
ruby<->STL interface better (faster and more STL functionality). I've
also done similar python<->STL modifications for swig. I'd love to
see all of this completed with full STL functionality (containers,
iterators, algorithms) and the big-O performance preserved. C++ STL
is quite rich w/ a lot of thought put into big-O performance (great
when dealing with large datasets).

···

On 6/18/07, Daniel Berger <djberg96@gmail.com> wrote:

Hi all,

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

#include "ruby.h"
#include <vector>
using namespace std;

struct cvector{
    vector<VALUE> array(0);
};

void Init_cvector(){
    VALUE cVector = rb_define_class("CVector", rb_cObject);
}

When I try to compile this (using g++ on OS X) I get:

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
enum 'rb_thread_status' without previous declaration
/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid
type in declaration before ';' token
cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant
cvector.c: In function 'void Init_cvector()':
cvector.c:10: warning: unused variable 'cVector'
make: *** [cvector.o] Error 1

Thanks,

Dan

Hi,

At Mon, 18 Jun 2007 23:15:26 +0900,
Daniel Berger wrote in [ruby-talk:256045]:

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

In general, C++ isn't supported in 1.8.

···

--
Nobu Nakada

Absolutely. But I suggest you download the latest swig from svn, as
it contains a lot of enhancements for this.

This is how you do it in swig.

---- mystl.i ----

%module mystl
%include std_vector.i

// GC_VALUE is a special class that knows how to mark itself.
%template(ValueVector) std::vector< GC_VALUE >;

----- end mystl.i ----

On a console:

$ swig -ruby -c++ mystl.i # creates mystl_wrap.cxx

Compile:

# using extconf.rb (recommended) or...

···

On Jun 18, 11:15 am, Daniel Berger <djber...@gmail.com> wrote:

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

#
# posix
#
$ g++ -fPIC -DPIC -O2 mystl_wrap.cxx -o mystl_wrap.so -lruby -lstdc++

#
# Windows
#
$ vcvars32.bat
$ cl.exe -MD -LD -O2 mystl_wrap.cxx -o mystl_wrap.so /link msvcrt-
ruby1.8.dll

------ usage -----
require 'mystl'
include Mystl

a = ValueVector.new
puts a.methods # you'll get iterators, most array methods, etc.

-------------------

g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1
-I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
-fno-common -c cvector.c
cvector.c: In function 'void Init_cvector()':
cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
VALUE)' to 'VALUE (*)(...)'
cvector.c:10: error: initializing argument 3 of 'void
rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
make: *** [cvector.o] Error 1

C++ is more pissy than C about type convertions. You have to use the
RUBY_METHOD_FUNC macro to convert cvector_init into the right type:

  rb_define_method(cVector, "initialize",
      RUBY_METHOD_FUNC(cvector_init), -1);

> Note that you can wrap std::vector directly (no need to go through an
> intermediate cvector structure)

I guess I could store it as an instance variable within the constructor
and refer back to it that way. I don't think it's faster, though. Or,
did you have something else in mind?

The idea is *not* to use a cvector structure. Since you'll have to provide
an alloc/free method pair anyway, allocate std::vector in them. Check
value_set_alloc/value_set_free in
http://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc

Sylvain

> g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1
> -I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
> -fno-common -c cvector.c
> cvector.c: In function 'void Init_cvector()':
> cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
> VALUE)' to 'VALUE (*)(...)'
> cvector.c:10: error: initializing argument 3 of 'void
> rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
> make: *** [cvector.o] Error 1

C++ is more pissy than C about type convertions. You have to use the
RUBY_METHOD_FUNC macro to convert cvector_init into the right type:

  rb_define_method(cVector, "initialize",
      RUBY_METHOD_FUNC(cvector_init), -1);

Ah, thanks.

> > Note that you can wrap std::vector directly (no need to go through an
> > intermediate cvector structure)

> I guess I could store it as an instance variable within the constructor
> and refer back to it that way. I don't think it's faster, though. Or,
> did you have something else in mind?

The idea is *not* to use a cvector structure. Since you'll have to provide
an alloc/free method pair anyway, allocate std::vector in them. Check
value_set_alloc/value_set_free inhttp://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc

Interesting, thank you. For kicks, I tried to compile your source code
on my Solaris 10 box (after installing boost). It built (with some
warnings), but I can't get it to load.

Here's the extconf.rb file I used:

require 'mkmf'
dir_config('set2')

case RUBY_PLATFORM
   when /sunos|solaris/
      CONFIG['CC'] = 'CC'
   when /mswin/i
      CONFIG["COMPILE_C"].sub!(/-Tc/, '-Tp')
   else
      CONFIG['CC'] = 'g++ -Wall'
end

create_makefile('set2')

Here was the result of the build step:

djberge-/export/home/djberge/workspace/set/ext-635>ruby extconf.rb --
with-set2-include=/opt/csw/include
creating Makefile

djberge-/export/home/djberge/workspace/set/ext-636>make
CC -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -I/opt/csw/include -KPIC -dalign -fns -
xbuiltin=%all -xlibmil -xtarget=ultra2e -xO5 -xipo -c set.c
"set.c", line 349: Warning (Anachronism): Formal argument 3 of type
extern "C" unsigned long(*)(...) in call to rb_iterate(extern "C"
unsigned long(*)(unsigned long), unsigned long, extern "C" unsigned
long(*)(...), unsigned long) is being passed unsigned long(*)(...).
"set.c", line 369: Warning (Anachronism): Formal argument 2 of type
extern "C" unsigned long(*)(unsigned long) in call to
rb_define_alloc_func(unsigned long, extern "C" unsigned long(*)
(unsigned long)) is being passed unsigned long(*)(unsigned long).
2 Warning(s) detected.
ld -G -o set2.so set.o -L'.' -L'/usr/local/lib' -R'/usr/local/lib' -
L. -lrt -lpthread -ldl -lcrypt -lm -lc

Ok, a couple warnings. I proceed to try to "require 'set2'" and I get
this:

djberge-/export/home/djberge/workspace/set/ext-637>ruby test.rb
/export/home/djberge/workspace/set/ext/set2.so: ld.so.1: ruby: fatal:
relocation error: file /export/home/djberge/workspace/set/ext/set2.so:
symbol __1cDstdJbad_allocG__vtbl_: referenced symbol not found - /
export/home/djberge/workspace/set/ext/set2.so (LoadError)
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from test.rb:2

This was with Ruby 1.8.6-p38 (today's svn checkout of the 1.8.6
branch).

Any ideas?

Dan

···

On Jun 19, 1:07 am, Sylvain Joyeux <sylvain.joy...@polytechnique.org> wrote:

Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.

It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.

Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.

boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.

-mental

Maybe we should just rewrite the interpreter in C++ then. Think of all
the STL and Boost stuff we could then integrate "for free". I can only
guess what other advantages this might bring in the long run.

I know Matz has said in the past that the C++ object model would get
in the way, but I'm convinced it could be done (by smarter minds than
I).

Dan

···

On Jun 19, 11:13 am, MenTaLguY <men...@rydia.net> wrote:

Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.

It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.

Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.

boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.

-mental

It should be nearly sufficient to replace the setjmp/longjump bits
with C++ exceptions and build the interpreter with a C++ compiler.

However, that just exchanges one problem for another since the bulk of Ruby
extensions are written in C, and embedding Ruby becomes even more
difficult.

I think we'd all be much better off if someone wrote a Ruby equivalent to
boost.python. There is a partially-completed one out there, but it's
unmaintained I'm not able to find it again at the moment.

-mental

···

On Wed, 20 Jun 2007 02:37:39 +0900, Daniel Berger <djberg96@gmail.com> wrote:

Maybe we should just rewrite the interpreter in C++ then. Think of all
the STL and Boost stuff we could then integrate "for free". I can only
guess what other advantages this might bring in the long run.

I'd make that trade. The potential benefits far exceed the pain of
reworking extensions IMHO, especially when you consider that *most*
people using Ruby don't do embedding or extending.

And this is coming from a guy who has written quite a few extensions.

Regards,

Dan

···

On Jun 19, 11:47 am, MenTaLguY <men...@rydia.net> wrote:

On Wed, 20 Jun 2007 02:37:39 +0900, Daniel Berger <djber...@gmail.com> wrote:
> Maybe we should just rewrite the interpreter in C++ then. Think of all
> the STL and Boost stuff we could then integrate "for free". I can only
> guess what other advantages this might bring in the long run.

It should be nearly sufficient to replace the setjmp/longjump bits
with C++ exceptions and build the interpreter with a C++ compiler.

However, that just exchanges one problem for another since the bulk of Ruby
extensions are written in C, and embedding Ruby becomes even more
difficult.