[ANN] rocaml 0.6.0: fast, easy Ruby extensions in Objective Caml

ruby-talk
1

rocaml allows you to write Ruby extensions in Objective Caml.

http://eigenclass.org/hiki/rocaml

Developing Ruby extensions with rocaml is easier and more convenient than
writing a plain old C extension because rocaml performs Ruby <-> OCaml
conversions for a wide range of types, including abstract types and arrays,
tuples, variants and records of values of any supported type (e.g. arrays of
arrays of variants of tuples of ...).

Moreover, exceptions raised in the OCaml code are captured by the generated
extension and raised inside Ruby.

Making an extension with rocaml involves two steps:
* implementing the desired functionality in Objective Caml, and registering
  the functions to be exported (using Callback.register : string -> 'a -> unit
  or the included camlp4 extension)
* creating the extconf.rb file (just modify the sample extconf.rb distributed
  with rocaml) defining the interface of your Objective Caml code.

** At no point is there any need to write a single line of C code. **

The mandatory trivial example

···

=============================

This example doesn't do justice to the usefulness of rocaml because the
extension is beyond trivial and you could as well have written it in C using
RubyInline. The advantages of rocaml (and of Objective Caml) usually become
visible when the extension takes more than two lines (take a look at the
3-line Marshal replacement that is 3 times faster than Ruby's, though...).
Here follows a minimal example however, merely to show how easily rocaml
extensions can be made.

Here's the OCaml code placed in fib.ml:
let rec fib n = if n < 2 then 1 else fib (n-1) + fib (n-2)
export fib

Here's the interface declaration in your extconf.rb:
Interface.generate("fib") do
   def_module("Fib") do
     fun "fib", INT => INT
   end
end

That's it. The extension can be built like any run-of-the-mill C extension
with
  ruby extconf.rb
  make

The resulting Ruby extension that can be used as usual:
  require 'fib'
  p Fib.fib 10

Download

You can get rocaml at http://eigenclass.org/hiki/rocaml

License

rocaml is distributed under the same terms as Ruby.

rocaml copyright (c) 2007 Mauricio Fernandez <mfp@acm.org>

--
Mauricio Fernandez - http://eigenclass.org

2

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

Thanks,

Dan

···

On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:

rocaml allows you to write Ruby extensions in Objective Caml.

3

I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/\) and
try again. The extconf.rb script succeeds now. However, now the make
fails:

ocamldep.opt rubyOCamlUtil.ml > .depend
sh: ocamldep.opt: not found
*** Error code 1
make: Fatal error: Command failed for target `.depend'

I have ocamldep, but no ocamldep.opt. Where do I go from here?

Thanks,

Dan

···

On Oct 17, 1:10 pm, Daniel Berger <djber...@gmail.com> wrote:

On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:

> rocaml allows you to write Ruby extensions in Objective Caml.

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

4

I see you solved this by installing camlp5, but I'd still like to fix the
camlp* selection code. What does camlp4 -v return for you?

$ camlp4 -v
Camlp4 version 3.10.0

camlp4 in 3.10.0 is incompatible with 3.09.2's (OTOH camlp5 *is* backwards
compatible).

The code that detects camlp5 is

have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?

For some reason it didn't work properly but I can't see why. Any idea?

···

On Thu, Oct 18, 2007 at 04:10:55AM +0900, Daniel Berger wrote:

On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:
> rocaml allows you to write Ruby extensions in Objective Caml.

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

5

> > rocaml allows you to write Ruby extensions in Objective Caml.

> <snip>

> I grabbed the latest and greatest Ocaml tarball and installed it.
> However, when I try to install rocaml I get this:

> sh: camlp5: not found
> *** extconf.rb failed ***

> I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
> comments in rocaml_extconf.rb suggest that either should suffice.

> What should I do?

I see you solved this by installing camlp5, but I'd still like to fix the
camlp* selection code. What does camlp4 -v return for you?

$ camlp4 -v
Camlp4 version 3.10.0

Same for me.

camlp4 in 3.10.0 is incompatible with 3.09.2's (OTOH camlp5 *is* backwards
compatible).

The code that detects camlp5 is

have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?

For some reason it didn't work properly but I can't see why. Any idea?

If camlp5 isn't found, you'll end up with nil. Changing 'empty?' to
'nil?' should do the trick, although people will still see a "sh:
camlp5: not found" echoed to stdout, which may be confusing.

Any ideas on the make failure?

Regards,

Dan

···

On Oct 17, 2:04 pm, Mauricio Fernandez <m...@acm.org> wrote:

On Thu, Oct 18, 2007 at 04:10:55AM +0900, Daniel Berger wrote:
> On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:

6

>
> > rocaml allows you to write Ruby extensions in Objective Caml.

[...]

> I grabbed the latest and greatest Ocaml tarball and installed it.
> However, when I try to install rocaml I get this:
>
> sh: camlp5: not found
> *** extconf.rb failed ***
>
> I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
> comments in rocaml_extconf.rb suggest that either should suffice.

[...]

I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/\) and
try again. The extconf.rb script succeeds now. However, now the make
fails:

ocamldep.opt rubyOCamlUtil.ml > .depend
sh: ocamldep.opt: not found
*** Error code 1
make: Fatal error: Command failed for target `.depend'

I have ocamldep, but no ocamldep.opt. Where do I go from here?

You didn't
  make opt.opt
(or make world.opt) when you built ocaml-3.10.0, did you?
(BTW, on Debian the .opt binaries are in ocaml-native-compilers, in case
somebody is reading this)

For the time being, you can just install the native-code compilers or
hand-edit the generated Makefile; scroll down to

...

···

On Thu, Oct 18, 2007 at 04:48:13AM +0900, Daniel Berger wrote:

On Oct 17, 1:10 pm, Daniel Berger <djber...@gmail.com> wrote:
> On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:

#############################################################################
# #
# Objective Caml #
# #
#############################################################################

OCAMLC = ocamlc.opt
OCAMLOPT = ocamlopt.opt
OCAMLDEP = ocamldep.opt
OFLAGS = -pp 'camlp5o -I . pa_rocaml.cmo'
OCAML_INCLUDES =
OCAML_LIBS = nums.cmxa
...

and remove the .opt extensions. I will add some code to detect whether the
.opt binaries are available.

One last thing; what's your platform? rocaml extensions might or might not
work on AMD64 and OSX because their linkers aren't very smart (they can't
handle non-PIC code, it seems). According to OCaml's 3.10.0 changelog,
  - Intel/AMD 64 bits: generate position-independent code by default.
but I don't know if that's enough or something special must be done at compile
time.

The next Objective Caml release will feature dynamic loading of native code,
so this won't remain a problem for long.
Meanwhile, static extensions can be used.

--
Mauricio Fernandez - http://eigenclass.org

7

[...]

> > I grabbed the latest and greatest Ocaml tarball and installed it.
> > However, when I try to install rocaml I get this:
>
> > sh: camlp5: not found
> > *** extconf.rb failed ***
>
> > I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
> > comments in rocaml_extconf.rb suggest that either should suffice.
>
> > What should I do?
>
> I see you solved this by installing camlp5, but I'd still like to fix the
> camlp* selection code. What does camlp4 -v return for you?

[...]

> The code that detects camlp5 is
>
> have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?
>
> For some reason it didn't work properly but I can't see why. Any idea?

If camlp5 isn't found, you'll end up with nil. Changing 'empty?' to
'nil?' should do the trick, although people will still see a "sh:
camlp5: not found" echoed to stdout, which may be confusing.

That's what 2>&1 is meant to prevent, but you're very right, it should be
nil?. Now I wonder why you didn't get a NoMethodError.

Any ideas on the make failure?

Yes, the native code compilers are missing, see my other message for a couple
solutions. There was another bug in the code that detects whether they are
present; I have pushed the fix to the repository at

  http://eigenclass.org/repos/rocaml/head/

and it will be in the next tarball, 0.6.1, to follow shortly.

···

On Thu, Oct 18, 2007 at 05:17:02AM +0900, Daniel Berger wrote:

On Oct 17, 2:04 pm, Mauricio Fernandez <m...@acm.org> wrote:
> On Thu, Oct 18, 2007 at 04:10:55AM +0900, Daniel Berger wrote:

Wed Oct 17 23:29:10 CEST 2007 Mauricio Fernandez <mfp@acm.org>
  * rocaml_extconf.rb: fixed native compiler detection.
Wed Oct 17 23:21:08 CEST 2007 Mauricio Fernandez <mfp@acm.org>
  * rocaml_extconf.rb: fix camlp[45] detection.
diff -rN -u old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-17 23:29:42.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-17 23:29:42.000000000 +0200
@@ -43,7 +43,7 @@
   exit
end

-maybe_opt = lambda{|x| opt = "#{x}.opt"; system(x) ? opt : x }
+maybe_opt = lambda{|x| opt = "#{x}.opt"; system(opt) ? opt : x }

if OCAML_PACKAGES.empty? then
   OCAMLC = maybe_opt["ocamlc"]
@@ -91,9 +91,9 @@

# determine whether camlp4 (or camlp5) can be used:

-have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?
+have_camlp5 = ! `camlp5 -v 2>&1`["version"].nil?
camlp4version = `camlp4 -v 2>&1`[/version\s+(\d.*)/, 1]
-have_camlp4 = ! camlp4version.empty?
+have_camlp4 = ! camlp4version.nil?

pa_rocaml_revdeps = Dir["*.ml"].map do |f|
   "#{f.sub(/\.ml$/, ".cmx")}: pa_rocaml.cmo"

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

8

> > > rocaml allows you to write Ruby extensions in Objective Caml.
[...]
> > I grabbed the latest and greatest Ocaml tarball and installed it.
> > However, when I try to install rocaml I get this:

> > sh: camlp5: not found
> > *** extconf.rb failed ***

> > I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
> > comments in rocaml_extconf.rb suggest that either should suffice.
[...]
> I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/\) and
> try again. The extconf.rb script succeeds now. However, now the make
> fails:

> ocamldep.opt rubyOCamlUtil.ml > .depend
> sh: ocamldep.opt: not found
> *** Error code 1
> make: Fatal error: Command failed for target `.depend'

> I have ocamldep, but no ocamldep.opt. Where do I go from here?

You didn't
  make opt.opt
(or make world.opt) when you built ocaml-3.10.0, did you?

Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
"transitional". Now it seems I'm missing some header files:

make

ocamldep.opt rubyOCamlUtil.ml > .depend
cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
"foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
mlvalues.h>
"foo_rocaml_wrapper.c", line 15: cannot find include file: <caml/
callback.h>
"foo_rocaml_wrapper.c", line 16: cannot find include file: <caml/
memory.h>
"foo_rocaml_wrapper.c", line 17: cannot find include file: <caml/
alloc.h>
"foo_rocaml_wrapper.c", line 18: cannot find include file: <caml/
fail.h>
...

I don't think OCaml's installation instructions could be any more
confusing. What option did I miss?

Regards,

Dan

···

On Oct 17, 2:21 pm, Mauricio Fernandez <m...@acm.org> wrote:

On Thu, Oct 18, 2007 at 04:48:13AM +0900, Daniel Berger wrote:
> On Oct 17, 1:10 pm, Daniel Berger <djber...@gmail.com> wrote:
> > On Oct 17, 9:31 am, Mauricio Fernandez <m...@acm.org> wrote:

9

> You didn't
> make opt.opt
> (or make world.opt) when you built ocaml-3.10.0, did you?

Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
"transitional". Now it seems I'm missing some header files:

>make
ocamldep.opt rubyOCamlUtil.ml > .depend
cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
"foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
mlvalues.h>

[...]

I don't think OCaml's installation instructions could be any more
confusing. What option did I miss?

I'm sorry for all the issues and appreciate your patience. The basic problem
is that I'm using Debian and everything Just Works(TM) with its OCaml
packages, so it is harder for me to anticipate problems like those you ran
into. I see that you're using Solaris, which adds further uncertainty because
I'm not sure its linker can build shared objects with non-PIC code [1]. Some
magic incantations in the form of $LDFLAGS might be required. You're stepping
on new ground :slight_smile:

The missing headers should be in /usr/local/lib/ocaml/3.10.0/caml (or maybe
/usr/local/lib/ocaml/caml; in my system I have a symlink from
/usr/include/caml to /usr/lib/ocaml/3.10.0/caml, which is why this never
happened to me). The extension should build correctly after symlinking or
applying this one-line patch:

···

On Fri, Oct 19, 2007 at 12:26:04AM +0900, Daniel Berger wrote:
Thu Oct 18 19:14:58 CEST 2007 Mauricio Fernandez <mfp@acm.org>
  * rocaml_extconf.rb: add ocaml_native_lib_path to the INCFLAGS (-Idir).
diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 19:25:49.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 19:25:49.000000000 +0200
@@ -43,6 +43,8 @@
   exit
end

+$INCFLAGS << " -I#{ocaml_native_lib_path}"
+
maybe_opt = lambda{|x| opt = "#{x}.opt"; system(opt) ? opt : x }

if OCAML_PACKAGES.empty? then

Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
  Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
  (you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
  converted between Ruby and OCaml. Abstract types become objects, records
  turn into hashes with symbol keys and variants are mapped to symbols or
  arrays.

Thank you,

[1] According to Re: Half-solved ? Re: Sparc code question, -fpic or -fPIC

SunOS and Solaris will quite happily build .so from non-PIC objects - they
just don't "share" very well as the relocations cause copy-on-write and hence
private pages.

So there's a good chance it will work.

--
Mauricio Fernandez - http://eigenclass.org

10

It would seem so. Your one line patch to rocaml_extconf.rb worked
(thanks!), but now I get this:

irb(main):002:0> require 'foo'
LoadError: ld.so.1: ruby: fatal: relocation error: file /export/home/
djberge/src/ruby/rocaml-0.6.1/foo.so: symbol __muldi3: referenced
symbol not found - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from (irb):2

This is Sun Studio 12 on Sparc/Solaris 10, btw.

I already tried installing a gcc add-on found here: http://cooltools.sunsource.net/gcc/

No joy. Any ideas?

Oh, and these warnings showed up during the build. Dunno if you're
interested:

"foo_rocaml_wrapper.c", line 61: warning: statement not reached
"foo_rocaml_wrapper.c", line 68: warning: statement not reached
"foo_rocaml_wrapper.c", line 338: warning: statement not reached
"foo_rocaml_wrapper.c", line 388: warning: statement not reached
"foo_rocaml_wrapper.c", line 438: warning: statement not reached
"foo_rocaml_wrapper.c", line 485: warning: statement not reached
"foo_rocaml_wrapper.c", line 532: warning: statement not reached
"foo_rocaml_wrapper.c", line 582: warning: statement not reached
"foo_rocaml_wrapper.c", line 632: warning: statement not reached
"foo_rocaml_wrapper.c", line 681: warning: statement not reached
"foo_rocaml_wrapper.c", line 730: warning: statement not reached
"foo_rocaml_wrapper.c", line 779: warning: statement not reached
"foo_rocaml_wrapper.c", line 829: warning: statement not reached

Regards,

Dan

···

On Oct 18, 12:09 pm, Mauricio Fernandez <m...@acm.org> wrote:

On Fri, Oct 19, 2007 at 12:26:04AM +0900, Daniel Berger wrote:
> > You didn't
> > make opt.opt
> > (or make world.opt) when you built ocaml-3.10.0, did you?

> Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
> "transitional". Now it seems I'm missing some header files:

> >make
> ocamldep.opt rubyOCamlUtil.ml > .depend
> cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
> ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
> "foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
> mlvalues.h>
[...]

> I don't think OCaml's installation instructions could be any more
> confusing. What option did I miss?

I'm sorry for all the issues and appreciate your patience. The basic problem
is that I'm using Debian and everything Just Works(TM) with its OCaml
packages, so it is harder for me to anticipate problems like those you ran
into. I see that you're using Solaris, which adds further uncertainty because
I'm not sure its linker can build shared objects with non-PIC code [1]. Some
magic incantations in the form of $LDFLAGS might be required. You're stepping
on new ground :slight_smile:

11

<snip>

Some more information that may or may not be useful:

jberge-/export/home/djberge/src/ruby/rocaml-0.6.1-697>ldd foo.so
        librt.so.1 => /lib/librt.so.1
        libpthread.so.1 => /lib/libpthread.so.1
        libdl.so.1 => /lib/libdl.so.1
        libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
        libm.so.2 => /lib/libm.so.2
        libc.so.1 => /lib/libc.so.1
        libaio.so.1 => /lib/libaio.so.1
        libmd5.so.1 => /lib/libmd5.so.1
        libgen.so.1 => /lib/libgen.so.1
        /platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
        /platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

Regards,

Dan

···

On Oct 18, 12:30 pm, Daniel Berger <djber...@gmail.com> wrote:

12

It seems we're almost there (we went through the critical non-PIC code
relocation phase :slight_smile:

According to google, __muldi3 is an internal GCC library function. Do you have
a native linker in addition to GCC's? GCC's will automatically link against
libgcc, but it seems that's not the case with the native one.

At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
something like this maybe?

diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
@@ -89,7 +89,9 @@

# needed by mkmf's create_makefile
$LOCAL_LIBS = "#{CAML_TARGET} #{ocaml_native_lib_path}/libasmrun.a #{extra_caml_libs.join(" ")}"

···

On Fri, Oct 19, 2007 at 03:30:06AM +0900, Daniel Berger wrote:

On Oct 18, 12:09 pm, Mauricio Fernandez <m...@acm.org> wrote:
> I'm sorry for all the issues and appreciate your patience. The basic problem
> is that I'm using Debian and everything Just Works(TM) with its OCaml
> packages, so it is harder for me to anticipate problems like those you ran
> into. I see that you're using Solaris, which adds further uncertainty because
> I'm not sure its linker can build shared objects with non-PIC code [1]. Some
> magic incantations in the form of $LDFLAGS might be required. You're stepping
> on new ground :slight_smile:

It would seem so. Your one line patch to rocaml_extconf.rb worked
(thanks!), but now I get this:

irb(main):002:0> require 'foo'
LoadError: ld.so.1: ruby: fatal: relocation error: file /export/home/
djberge/src/ruby/rocaml-0.6.1/foo.so: symbol __muldi3: referenced
symbol not found - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from (irb):2

-
+# try to add GCC's libgcc, required on Sparc
+libgcc = Dir["/lib/libgcc*"].first # maybe some other search path(s)
+$LOCAL_LIBS << " " << libgcc if libgcc

# determine whether camlp4 (or camlp5) can be used:

You might have to change the search path. On my platform, libgcc can also be
found in /usr/lib/gcc/i486-linux-gnu/<gcc version>. Linking against either
libgcc.a or libgcc_s.so should do.

I haven't pushed the patch yet; want to know if /lib/libgcc* is OK or if the
path needs to be changed. Also, can I rely on say
  require 'rbconfig'
  libgcc_needed = Config::CONFIG["arch"] =~ "sparc"
?

Oh, and these warnings showed up during the build. Dunno if you're
interested:

"foo_rocaml_wrapper.c", line 61: warning: statement not reached
"foo_rocaml_wrapper.c", line 68: warning: statement not reached
"foo_rocaml_wrapper.c", line 338: warning: statement not reached

These are all things like

  rb_raise(klass, "%s", s);
  return Qnil; /* not reached */

or

  if(exception == Qnil && !status) {
    return ret;
  } else if(status) { /* exception in Ruby -> caml conversions */
    rb_jump_tag(status);
  } else { /* OCaml exception*/
    rb_raise(rb_eStandardError, StringValuePtr(exception));
  }

  return Qnil; /* never reached */

It seems your GCC is smarter than mine and knows that rb_raise, rb_jump_tag,
etc. do not return. Know any macros in ruby.h to handle such things?
They are harmless and can be safely ignored anyway.

--
Mauricio Fernandez - http://eigenclass.org

13

<snip>

It seems we're almost there (we went through the critical non-PIC code
relocation phase :slight_smile:

According to google, __muldi3 is an internal GCC library function. Do you have
a native linker in addition to GCC's? GCC's will automatically link against
libgcc, but it seems that's not the case with the native one.

Did I mention that I built Ruby with Sun Studio 12, but OCaml with gcc
4? Ocaml seemed to demand it. :expressionless:

At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
something like this maybe?

diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
@@ -89,7 +89,9 @@

# needed by mkmf's create_makefile
$LOCAL_LIBS = "#{CAML_TARGET} #{ocaml_native_lib_path}/libasmrun.a #{extra_caml_libs.join(" ")}"
-
+# try to add GCC's libgcc, required on Sparc
+libgcc = Dir["/lib/libgcc*"].first # maybe some other search path(s)
+$LOCAL_LIBS << " " << libgcc if libgcc

# determine whether camlp4 (or camlp5) can be used:

You might have to change the search path. On my platform, libgcc can also be
found in /usr/lib/gcc/i486-linux-gnu/<gcc version>. Linking against either
libgcc.a or libgcc_s.so should do.

In my particular case it's /opt/csw/gcc4/lib/libgcc which I added
directly:

djberge-/opt/csw/gcc4/lib-663>ll libg*
lrwxrwxrwx 1 root root 15 Dec 12 2006 libgcc_s.so -> ./
libgcc_s.so.1
-rwxr-xr-x 1 root bin 63928 Oct 10 2005 libgcc_s.so.1

It builds correctly. But then I get a different linker error:

LoadError: ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such
file or directory - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from (irb):2

LDD led me to the fix:

djberge-/export/home/djberge/src/ruby/rocaml-0.6.1-546>ldd foo.so
        libgcc_s.so.1 => (file not found)
        librt.so.1 => /lib/librt.so.1
        libpthread.so.1 => /lib/libpthread.so.1
        libdl.so.1 => /lib/libdl.so.1
        libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
        libm.so.2 => /lib/libm.so.2
        libc.so.1 => /lib/libc.so.1
        libaio.so.1 => /lib/libaio.so.1
        libmd5.so.1 => /lib/libmd5.so.1
        libgen.so.1 => /lib/libgen.so.1
        /platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
        /platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

So, I slapped $LDFLAGS += " -L/opt/csw/gcc4/lib -R/opt/csw/gcc4/lib"
into the rocaml_extconf.rb file. Looks better after that:

djberge-/export/home/djberge/src/ruby/rocaml-0.6.1-605>ldd foo.so
        libgcc_s.so.1 => /opt/csw/gcc4/lib/libgcc_s.so.1
        librt.so.1 => /lib/librt.so.1
        libpthread.so.1 => /lib/libpthread.so.1
        libdl.so.1 => /lib/libdl.so.1
        libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
        libm.so.2 => /lib/libm.so.2
        libc.so.1 => /lib/libc.so.1
        libaio.so.1 => /lib/libaio.so.1
        libmd5.so.1 => /lib/libmd5.so.1
        libgen.so.1 => /lib/libgen.so.1
        /platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
        /platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

Back to irb:

irb(main):002:0> require 'foo'
NameError: (eval): uninitialized constant Kernel::Some
        from (eval)
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from (irb):2

Hu...what? WTF? Never seen that one before. I dug through /usr/local/
lib/ruby to make sure there wasn't some other "foo" it was picking up
- there isn't.

Any ideas?

Thanks,

Dan

···

On Oct 18, 1:18 pm, Mauricio Fernandez <m...@acm.org> wrote:

14

<snip>

> It seems we're almost there (we went through the critical non-PIC code
> relocation phase :slight_smile:
>
> According to google, __muldi3 is an internal GCC library function. Do you have
> a native linker in addition to GCC's? GCC's will automatically link against
> libgcc, but it seems that's not the case with the native one.

Did I mention that I built Ruby with Sun Studio 12, but OCaml with gcc
4? Ocaml seemed to demand it. :expressionless:

> At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
> something like this maybe?

[...]

It builds correctly. But then I get a different linker error:

LoadError: ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such
file or directory - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
        from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
        from (irb):2

[...]

So, I slapped $LDFLAGS += " -L/opt/csw/gcc4/lib -R/opt/csw/gcc4/lib"
into the rocaml_extconf.rb file. Looks better after that:

[...]

Back to irb:

irb(main):002:0> require 'foo'
NameError: (eval): uninitialized constant Kernel::Some

[...]

Hu...what? WTF? Never seen that one before. I dug through /usr/local/
lib/ruby to make sure there wasn't some other "foo" it was picking up
- there isn't.

Any ideas?

Congrats, it seems you got it to work :slight_smile:

This is what you missed in a previous message:

Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
   Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
   (you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
   converted between Ruby and OCaml. Abstract types become objects, records
   turn into hashes with symbol keys and variants are mapped to symbols or
   arrays.

BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
I'd like to have as much as possible work out of the box (trying to detect
common cases) and offer a convenient way to specify the things that cannot be
inferred. Could you give me the output of
$ ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"
?

Thanks,

···

On Fri, Oct 19, 2007 at 05:16:37AM +0900, Daniel Berger wrote:

On Oct 18, 1:18 pm, Mauricio Fernandez <m...@acm.org> wrote:

--
Mauricio Fernandez - http://eigenclass.org

15

<snip>

Congrats, it seems you got it to work :slight_smile:

This is what you missed in a previous message:

Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
   Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
   (you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
   converted between Ruby and OCaml. Abstract types become objects, records
   turn into hashes with symbol keys and variants are mapped to symbols or
   arrays.

Oh, heh, whoops. :slight_smile:

BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
I'd like to have as much as possible work out of the box (trying to detect
common cases) and offer a convenient way to specify the things that cannot be
inferred.

No, I'm afraid not. The "/opt/csw" prefix is the standard blastwave
package directory (as in, Solaris packages installed from
blastwave.org). I think you'll have to setup a "--with-gcc-dir" option
or something.

Could you give me the output of
$ ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"

Sure, here it is:

ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"

ruby 1.8.6 (2007-09-23 patchlevel 110) [sparc-solaris2.10]
{"sitedir"=>"/usr/local/lib/ruby/site_ruby",
"MAKEFILES"=>"Makefile",
"LIBRUBY"=>"libruby-static.a",
"MAKEDIRS"=>"mkdir -p",
"PACKAGE_VERSION"=>"",
"GREP"=>"/usr/sfw/bin/ggrep",
"prefix"=>"/usr/local",
"COMMON_LIBS"=>"",
"rubylibdir"=>"/usr/local/lib/ruby/1.8",
"target"=>"sparc-sun-solaris2.10",
"DLLWRAP"=>"",
"AR"=>"ar",
"target_alias"=>"",
"MANTYPE"=>"man",
"docdir"=>"/usr/local/share/doc/$(PACKAGE)",
"RDOCTARGET"=>"",
"dvidir"=>"/usr/local/share/doc/$(PACKAGE)",
"AS"=>"as",
"GNU_LD"=>"no",
"MAINLIBS"=>"",
"WINDRES"=>"",
"XCFLAGS"=>" -DRUBY_EXPORT",
"datarootdir"=>"/usr/local/share",
"RUBY_INSTALL_NAME"=>"ruby",
"LN_S"=>"ln -s",
"archdir"=>"/usr/local/lib/ruby/1.8/sparc-solaris2.10",
"LINK_SO"=>"",
"MINIRUBY"=>"./miniruby",
"DLDLIBS"=>" -lc",
"EXEEXT"=>"",
"target_vendor"=>"sun",
"RUNRUBY"=>"./miniruby $(srcdir)/runruby.rb --extout=.ext --",
"DESTDIR"=>"",
"sbindir"=>"/usr/local/sbin",
"LIBPATHENV"=>"LD_LIBRARY_PATH",
"sitearch"=>"sparc-solaris2.10",
"psdir"=>"/usr/local/share/doc/$(PACKAGE)",
"host_cpu"=>"sparc",
"DLEXT2"=>"",
"LIBS"=>"-lrt -lpthread -ldl -lcrypt -lm ",
"localedir"=>"/usr/local/share/locale",
"rubyw_install_name"=>"",
"EXTOUT"=>".ext",
"ECHO_C"=>"",
"OBJDUMP"=>"",
"arch"=>"sparc-solaris2.10",
"MAJOR"=>"1",
"ruby_version"=>"1.8",
"CC"=>"cc",
"EGREP"=>"/usr/sfw/bin/ggrep -E",
"COMMON_HEADERS"=>"",
"COMMON_MACROS"=>"",
"PACKAGE_TARNAME"=>"",
"build_cpu"=>"sparc",
"build_vendor"=>"sun",
"host_alias"=>"",
"RANLIB"=>"ranlib",
"LDSHARED"=>"ld -G",
"LIBRUBYARG_SHARED"=>"-R /usr/local/lib -L/usr/local/lib ",
"YFLAGS"=>"",
"htmldir"=>"/usr/local/share/doc/$(PACKAGE)",
"MINOR"=>"8",
"INSTALL_SCRIPT"=>"/opt/csw/bin/ginstall -c",
"EXPORT_PREFIX"=>"",
"LIBRUBY_ALIASES"=>"libruby.so",
"LDFLAGS"=>"-L. ",
"datadir"=>"/usr/local/share",
"NM"=>"",
"includedir"=>"/usr/local/include",
"infodir"=>"/usr/local/share/info",
"host_os"=>"solaris2.10",
"build"=>"sparc-sun-solaris2.10",
"host"=>"sparc-sun-solaris2.10",
"INSTALL_DATA"=>"/opt/csw/bin/ginstall -c -m 644",
"build_os"=>"solaris2.10",
"DLDFLAGS"=>"",
"ruby_install_name"=>"ruby",
"DLEXT"=>"so",
"LIBRUBY_SO"=>"libruby.so.1.8.6",
"TEENY"=>"6",
"CPP"=>"cc -E",
"ALLOCA"=>"",
"sysconfdir"=>"/usr/local/etc",
"exec_prefix"=>"/usr/local",
"PATH_SEPARATOR"=>":",
"LIBEXT"=>"a",
"mandir"=>"/usr/local/share/man",
"libdir"=>"/usr/local/lib",
"build_alias"=>"",
"target_cpu"=>"sparc",
"ECHO_N"=>"-n",
"sharedstatedir"=>"/usr/local/com",
"YACC"=>"bison -y",
"configure_args"=>" '--enable-pthread' 'CC=cc'",
"SOLIBS"=>"",
"host_vendor"=>"sun",
"TRY_LINK"=>"",
"PACKAGE_STRING"=>"",
"target_os"=>"solaris2.10",
"oldincludedir"=>"/usr/include",
"CP"=>"cp",
"SET_MAKE"=>"",
"LIBRUBYARG_STATIC"=>"-lruby-static",
"LIBRUBYARG"=>"-lruby-static",
"RUBYW_INSTALL_NAME"=>"",
"PACKAGE_NAME"=>"",
"pdfdir"=>"/usr/local/share/doc/$(PACKAGE)",
"EXTSTATIC"=>"",
"ECHO_T"=>"",
"RPATHFLAG"=>" -R'%1$-s'",
"SHELL"=>"/bin/bash",
"STATIC"=>"",
"ASFLAGS"=>"",
"INSTALL"=>"/opt/csw/bin/ginstall -c",
"sitearchdir"=>"/usr/local/lib/ruby/site_ruby/1.8/sparc-solaris2.10",
"STRIP"=>"strip",
"ARCHFILE"=>"",
"LIBRUBY_DLDFLAGS"=>"",
"OBJEXT"=>"o",
"XLDFLAGS"=>"",
"LIBRUBY_LDSHARED"=>"ld -G",
"ENABLE_SHARED"=>"no",
"RM"=>"rm -f",
"CCDLFLAGS"=>" -KPIC",
"setup"=>"Setup",
"CPPOUTFILE"=>"-o conftest.i",
"topdir"=>"/usr/local/lib/ruby/1.8/sparc-solaris2.10",
"RUBY_SO_NAME"=>"ruby",
"CFLAGS"=>"-g",
"localstatedir"=>"/usr/local/var",
"LIBPATHFLAG"=>" -L'%1$-s'",
"bindir"=>"/usr/local/bin",
"sitelibdir"=>"/usr/local/lib/ruby/site_ruby/1.8",
"NROFF"=>"/usr/bin/nroff",
"CPPFLAGS"=>"",
"INSTALL_PROGRAM"=>"/opt/csw/bin/ginstall -c",
"PACKAGE_BUGREPORT"=>"",
"libexecdir"=>"/usr/local/libexec",
"OUTFLAG"=>"-o ",
"LIBRUBY_A"=>"libruby-static.a",
"PREP"=>"miniruby",
"ARCH_FLAG"=>""}

BTW, I guess MS Windows is out of the question then, unless I want to
install Cygwin, correct?

Many thanks,

Dan

···

On Oct 18, 3:36 pm, Mauricio Fernandez <m...@acm.org> wrote:

16

> Congrats, it seems you got it to work :slight_smile:

[...]

> BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
> I'd like to have as much as possible work out of the box (trying to detect
> common cases) and offer a convenient way to specify the things that cannot be
> inferred.

No, I'm afraid not. The "/opt/csw" prefix is the standard blastwave
package directory (as in, Solaris packages installed from
blastwave.org). I think you'll have to setup a "--with-gcc-dir" option
or something.

Yes, that seems the most reasonable way.

[...]

BTW, I guess MS Windows is out of the question then, unless I want to
install Cygwin, correct?

I used to think so, but I've been googling a bit and it turns out that what I
thought would be the major problem (non-PIC code relocation in a shared
object) just can't happen: "On Win32, the standard executable format is
non-PIC"(!!) [1].

So it seems it is indeed possible to get rocaml to work on Win32. Quoting from
OCaml's README.win32:

There are no less than four ports of Objective Caml for MS Windows available:
   - a native Win32 port, built with the Microsoft development tools;
   - a native Win32 port, built with the Cygwin/MinGW development tools;
   - a port consisting of the Unix sources compiled under the Cygwin
     Unix-like environment for Windows;
   - a native Win64 port (64-bit Windows), built with the Microsoft
     development tools.

Building Ruby extensions on Windows has always been a PITA, since you have to
use the same compiler that was used to build Ruby itself. Fortunately, the
binary used by the One Click Installer is compatible with extensions built
with MinGW, so the MinGW ocaml build should work, as should the Cygwin one
along with Cygwin's ruby. And it might even be possible to get it to work with
MS's tools assuming Ruby itself can be compiled cleanly.

Of course, I'll have to add some options to make it possible to specify where
ocaml is installed, since the auto-detection I'm doing will obviously fail:

[rocaml_extconf.rb]
  ...
  ocaml_native_lib_path = %w[
    /usr/lib/ocaml/**/libasmrun.a
    /usr/local/lib/ocaml/**/libasmrun.a
  ].map{|glob| Dir[glob]}.flatten.sort.map{|x| File.dirname(x)}.last
  ...

Some further LDFLAGS and/or CFLAGS magic might be needed, as was the case in
Solaris, and there might be some un*ix-isms in rocaml_extconf.rb here and
there, but it seems it can be made to work. I don't have access to a Windows
box so it all depends on people willing to explore new ways, though :wink:

[1] https://www.gelato.unsw.edu.au/archives/comp-arch/2006-December/005669.html

···

On Fri, Oct 19, 2007 at 06:59:29AM +0900, Daniel Berger wrote:

On Oct 18, 3:36 pm, Mauricio Fernandez <m...@acm.org> wrote:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby