Ruby and Mono

Tim,

Thanks for your reply on -talk. Hmm, guess I'll cc there.

I'm very interested in this issue, but I don't yet know enough
about .NET (or Mono) to fully grasp it.

Here I'm using Mono and .NET interchangeably -- if there's a
good reason not to, let me know.

There's been talk about an IronRuby (though I wouldn't want to
call it that) to match the IronPython that recently came into
being.

However, I'm not sure exactly what this should consist of:

1. A Ruby interpreter ported to Mono?
2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
3. A combo of these? An interpreter ported to Mono that outputs Mono?
4. A library that simply allows calling the CLR and such?
5. Or something else entirely?

4 seems weak.
1 does also.
2 (essentially cross-compiling) seems inconsistent.
3 tentatively seems best to me.
5 -- who knows?

Thanks for any insights...

Hal

Hal Fulton wrote:

Here I'm using Mono and .NET interchangeably -- if there's a
good reason not to, let me know.

I think for most things that's OK, though the full .Net platform has things not in Mono.

There's been talk about an IronRuby (though I wouldn't want to
call it that) to match the IronPython that recently came into
being.

However, I'm not sure exactly what this should consist of:

1. A Ruby interpreter ported to Mono? 2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
3. A combo of these? An interpreter ported to Mono that outputs Mono?
4. A library that simply allows calling the CLR and such?
5. Or something else entirely?

4 seems weak.
1 does also.
2 (essentially cross-compiling) seems inconsistent.
3 tentatively seems best to me.
5 -- who knows?

I think the main goal should be a ruby-to-CLR compiler, such that one could write code in Ruby and run it under mono/.net, and have it call other CLR objects (such as those written in IronPython). Ruby compiled to CLR should also be callable by other CLR languages (e.g., C#).

And yeah, "IronRuby" is a less than ideal name.

Anyone know where that 'Iron' part came from?

James

i've got a small c extension to bind to system.reflection.emit
thusly allowing generation and execution of cil bytecode / dlls.
if anyones interested just mail me. got the basics of a trivial
ruth/ast based interpreter working also, got so far as to make
it successfully execute stuff like "a = 5; a += 8; puts a" :stuck_out_tongue:

but... mono ain't all that interesting to me other than
for interoperability so i've since switched to the parrot
backend route... and then gave up on that as it was so buggy
several months ago when i tested it, and now i've restarted
the effort using a pure native code generator : libjit.

cheers,
Alex

···

On Wed, Aug 11, 2004 at 09:54:53AM +0900, Hal Fulton wrote:

1. A Ruby interpreter ported to Mono?
2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
3. A combo of these? An interpreter ported to Mono that outputs Mono?
4. A library that simply allows calling the CLR and such?
5. Or something else entirely?

Tim,

Thanks for your reply on -talk. Hmm, guess I'll cc there.

I'm very interested in this issue, but I don't yet know enough
about .NET (or Mono) to fully grasp it.

Here I'm using Mono and .NET interchangeably -- if there's a
good reason not to, let me know.

None the "bridge" projects I described currently work in Mono, since they
all use (fairly small) amounts of "Managed C++" (C++ for .NET) and Mono
doesn't have a compiler for this.

However, the amount of Managed C++ used in each is small, and e.g. in my
project (rubydotnetproxy) I have implemented Mono support in about 20 lines
of code. (There are some serious bugs though, not sure if these are my fault
or Monos. I haven't released Mono support yet.)

But in general, if you avoid Managed C++ then Mono and .NET support will be
the same.

There's been talk about an IronRuby (though I wouldn't want to
call it that) to match the IronPython that recently came into
being.

However, I'm not sure exactly what this should consist of:

1. A Ruby interpreter ported to Mono?
2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
3. A combo of these? An interpreter ported to Mono that outputs Mono?
4. A library that simply allows calling the CLR and such?
5. Or something else entirely?

4 seems weak.
1 does also.
2 (essentially cross-compiling) seems inconsistent.
3 tentatively seems best to me.
5 -- who knows?

Thanks for any insights...

One of the important issues is "how do (statically-typed) languages like C#
call Ruby code".

In the worst case, you have to do
  foo.Call("SomeRubyMethod", 123);

This is "bad" because it doesn't look like a normal method call. You really
want to write
  foo.SomeRubyMethod(123);

This requires generating a Ruby interface at compile time. The simplest
interface would look like e.g. (C# code)
  class String extends RbObject {
    public RbObject capitalize(params RbObject args) {
      // ...
    }
    // ...
  }

i.e. you don't need to know the number of arguments (although it can help
performance).

Obviously we can't know all interfaces at compile time, but knowing "most
things" and having to use Call("Foo") for the rest could still be usable.

I've been musing over the idea of using documentation tools (like rdoc) for
generating public interfaces. People already have to deal with issues like
"this method exists but you can't tell at compile time" in documentation.

Microsoft have said they're considering adding an opcode to do dynamic
method invocation. If they implement this, I hope that they change languages
like C# so if a program uses a type (class) that's marked as 'dynamic', and
attempts to call a method that is not declared, then the compiler will
insert a 'dynamic_call' operation instead of failing to compile.

This would be very very good for Ruby and other dynamically typed languages.
It would make them "first class citizens" on the CLR.

As for the choice of bridging the current Ruby interpreter and the CLR or
writing a compiler/interpreter for it:

Bridge advantages

  - Since it's using the normal Ruby interpreter, all your Ruby code will
    continue to work. You can use existing libraries, C extensions etc.
    without any problems. Always up to date with current Ruby.

    If you were compiling to the CLR you would either have to drop support
    for current C extensions or painfully reimplement Ruby's C interface.
    The latter is hard since lots of low-level things are exposed, like
    implementation of Strings, Arrays etc.

    On the other hand, if Ruby 2.0 is going to change the C interface
    anyway...

  - A lot less work to implement. It's easy to get something simple that has
    a limited amount of interfacing, but is still useful.

    In comparison, writing a compiler and new runtime implementation
    requires a lot of work before it can be used.

Bridge disadvantages
   - Have two garbage collectors and threading models. Threading is a bit of
     a pain, since .NET uses system threads (and may in the future also use
     user-level threads) while Ruby has user threads.

     This is the same problem Ruby has interfacing to any library that uses
     sytem threads.
   
Advantages of new compiler/interpreter
  - Perception that Ruby is working "properly" on .NET.

  - Don't have the "Bridge disadvantages" - have one garbage collector and
    threading model.

As far as performance goes, I don't think there is a huge difference between
the two approaches. e.g. IronPython runs a bit faster than CPython in
general, and since it's a new project and lots of work is being done I'd
expect it to improve faster than CPython.

At the moment there is a belief that "bridges are slow", since they all use
..NET's Type#InvokeMember (which is horrendously slow) when Ruby calls .NET
methods. But:

  - Ruby code runs at normal speed.
  - .NET code runs at normal speed.
  - It's only when Ruby calls .NET or vice versa that we get slowness
  (method invoking and marshalling).

There are better ways of doing things than using InvokeMember, e.g.
GetMethod, which searches for a method and then lets you call it multiple
times. There's some future stuff called "Lightweight Code Generation",
which is to let you output small amounts of IL (.NET assembly) and load
them efficiently.

Performance of bridges is what I'm working on at the moment. Things like
caching method searches and reducing the amount of marshalling (e.g. passing
strings/arrays between Ruby and .NET) will improve the performance
dramatically I think.

···

In article <41196E4F.1060404@hypermetrics.com>, Hal Fulton wrote:

--
Tim Sutherland <timsuth@ihug.co.nz>
2004 SDKACM President
Software Developers' Klub - the University of Auckland ACM Student Chapter
http://www.sdkacm.com/

James Britt wrote:

I think the main goal should be a ruby-to-CLR compiler, such that one could write code in Ruby and run it under mono/.net, and have it call other CLR objects (such as those written in IronPython). Ruby compiled to CLR should also be callable by other CLR languages (e.g., C#).

That would be very cool.

And yeah, "IronRuby" is a less than ideal name.

Sticking with metals, there's Rubidium: it's stable, but reacts with its environment releasing large amounts of energy :wink:
Sam

er... what i mean by this is. i have a small ruby/c extension.
code using it looked a bit like this for example:

   $se.OpCodeVoid(OpCodeFactory.NewLdloc_1)
   $se.OpCodeVoid(OpCodeFactory.NewLdc_I4_1())
   $se.OpCodeVoid(OpCodeFactory.NewAdd())

could be neatened up majorly. but for a few days of hacking
it ain't too bad at all. it can be downloaded from here:

   http://www.lypanov.net/websitedata/monoruby-2003-10-17.tar.bz2

the basic idea:
   could write a ruby -> clr compiler in ruby
   which could later on be bootstrapped if the
   system.reflection.emit wrapper itself was
   bound from within.

the method:
   very trivial mono binding based on paolo mauros
   mono-perl. this provides access only to static
   methods. no instances. so rather than hack lots
   in c. i write a c# extension which makes s.r.e
   to static interfaces only. this could of course
   be majorly improved by integrated better with
   mono but i just wanted a proof of concept clr
   byte code generation + execution library.

mvg
Alex

···

On Wed, Aug 11, 2004 at 07:53:55AM +0200, Alexander Kellett wrote:

On Wed, Aug 11, 2004 at 09:54:53AM +0900, Hal Fulton wrote:
> 1. A Ruby interpreter ported to Mono?
> 2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
> 3. A combo of these? An interpreter ported to Mono that outputs Mono?
> 4. A library that simply allows calling the CLR and such?
> 5. Or something else entirely?

i've got a small c extension to bind to system.reflection.emit
thusly allowing generation and execution of cil bytecode / dlls.

--
Ummmmm ... Beeeeeeeeerrr.....

   - Simpson, Homer

Alexander Kellett wrote:

···

On Wed, Aug 11, 2004 at 09:54:53AM +0900, Hal Fulton wrote:

1. A Ruby interpreter ported to Mono?
2. A native (x86/whatever) interpreter that spits out CIL bytecodes?
   
3. A combo of these? An interpreter ported to Mono that outputs Mono?
4. A library that simply allows calling the CLR and such?
5. Or something else entirely?
   
i've got a small c extension to bind to system.reflection.emit thusly allowing generation and execution of cil bytecode / dlls.
if anyones interested just mail me. got the basics of a trivial ruth/ast based interpreter working also, got so far as to make
it successfully execute stuff like "a = 5; a += 8; puts a" :stuck_out_tongue:

but... mono ain't all that interesting to me other than
for interoperability so i've since switched to the parrot
backend route... and then gave up on that as it was so buggy
several months ago when i tested it, and now i've restarted
the effort using a pure native code generator : libjit.

The libjit path is interesting; I think that is a very promising one. Have you managed to wrap libjit as a Ruby extension? I started on that but got stuck on the C++ requirement of libjit which made it painful so I got sidetracked.

Regards,

Robert

Hoary old chestnut, I know, but what's your view on libjit's GPL
licence? Wouldn't that imply that Ruby based on libjit was GPL, which
in turn would imply that software that embedded Ruby would have to be
too? If so, that's not as liberal as current Ruby or, afaik, the P
languages.

[I hope I'm not too deluded on this matter, cos this is what stopped
me using vmgen for ByteCodeRuby :slight_smile: ]

-- George

···

Alexander Kellett <ruby-lists@lypanov.net> wrote:

.. the effort using a pure native code generator : libjit.

ummm c++ requirement? llvm is c++ but libjit uses c++ exceptions internally
and provides a c only interface. (it does provide jitplus also but i see
no reason to use that :)). but yeah i've got a working libjit ruby/c
extension now. code at the moment:

   http://www.lypanov.net/websitedata/blub-0.0.0a.tbz2

cheers
Alex

···

On Thu, Aug 12, 2004 at 02:42:20PM +0900, Robert Feldt wrote:

The libjit path is interesting; I think that is a very promising one.
Have you managed to wrap libjit as a Ruby extension? I started on that
but got stuck on the C++ requirement of libjit which made it painful so
I got sidetracked.

i don't worry myself with licensing issues just
choose the technicallly most sound approach. i'm
sure that my work could be retargetted for another
jitting backend later on and the license changed
so i'd prefer not to get into the licensing game
at all. for the moment i just want to prove that
the basic method is sound.

Alex

···

On Thu, Aug 12, 2004 at 08:56:11PM +0900, George Marrows wrote:

Alexander Kellett <ruby-lists@lypanov.net> wrote:
> .. the effort using a pure native code generator : libjit.

Hoary old chestnut, I know, but what's your view on libjit's GPL
licence? Wouldn't that imply that Ruby based on libjit was GPL, which
in turn would imply that software that embedded Ruby would have to be
too? If so, that's not as liberal as current Ruby or, afaik, the P
languages.

--
Ummmmm ... Beeeeeeeeerrr.....

   - Simpson, Homer

ummm c++ requirement? llvm is c++ .. <snip>

Robert - weren't you or one of your students working on an llvm-based
compiler? How did that go? Anything demonstratable? Any pitfalls or
pleasant surprises to report?

- George

Alexander Kellett wrote:

···

On Thu, Aug 12, 2004 at 02:42:20PM +0900, Robert Feldt wrote:

The libjit path is interesting; I think that is a very promising one. Have you managed to wrap libjit as a Ruby extension? I started on that but got stuck on the C++ requirement of libjit which made it painful so I got sidetracked.
   
ummm c++ requirement? llvm is c++ but libjit uses c++ exceptions internally
and provides a c only interface. (it does provide jitplus also but i see no reason to use that :)). but yeah i've got a working libjit ruby/c extension now. code at the moment:

  http://www.lypanov.net/websitedata/blub-0.0.0a.tbz2

Yeah, I had some problem with the C++ exceptions together with Ruby mkmf/makefiles but maybe Rhys has simplified things since then. Anyway, great to see progress there. Keep us posted on how it goes...

/Robert

George Marrows wrote:

ummm c++ requirement? llvm is c++ .. <snip>
   
Robert - weren't you or one of your students working on an llvm-based
compiler? How did that go? Anything demonstratable? Any pitfalls or
pleasant surprises to report?

Good reminder; I never uploaded that thesis. But now it is available on the page

http://www.pronovomundo.com/htu/theses2004/

here's the short version:

Anders Alexandersson, “RubyComp – a Ruby-to-LLVM Compiler Prototype”

Abstract
Dynamic programming languages are not generally precompiled, but are interpreted at run-time. This approach has some serious drawbacks, e.g. complex deployment, human readable source code not preserving the intellectual properties of the developers and no ability to do optimizations at compile-time or run-time.
In this paper we study the possibility to pre-compile the Ruby language, a dynamic object-oriented language, into Low Level Virtual Machine (LLVM) code for execution by the LLVM run-time, a compiler framework for lifelong optimization
of an application. The result of the project is a Ruby compiler prototype, describing the infrastructure and
overall design principles to map the highly dynamic properties of the Ruby language into low-level static constructs
of the LLVM language.
The LLVM framework supports different hardware platforms, and by using LLVM as the target of compilation the
benefits of that portability are gained.

Conclusions
The core of the prototype design is to handle dynamic behavior by recompilation from one static state to another, by
the use of meta-data. Between recompilations static pointers are used, which enables good performance.
The prototype contains an LLVM Modelling Layer, hiding the LLVM syntax which is used to process the AST.
Some important obstacles have been identified in the LLVM framework, preventing support for full dynamic behavior,
and to solve this, the framework needs modifications.
As a whole and provided that the LLVM framework can be modified to fit the needs of the Ruby dynamic requirements,
compiling Ruby to LLVM is indeed feasible.

My comment: I haven't checked what state the code is in, the paper should contain the majority of the details. For me LLVM is (still) a very interesting project and I think it would be a feasible path for a Ruby compilation back-end (a very good thing is that it is well supported and quickly evolving, ie. it's future seem bright), however there are some hurdles for supporting Ruby's eval etc which seems a bit more complex than I initially thought.

Regards,

Robert