I seem to have been missing in action for the best part of six months so I thought it was about time I put together a progress report and pimped the vapourware in hopes of finding some likeminded miscreants/dilettantes to turn it into something more substantial.
There're probably only a handful of people on here who'll be interested in what follows so I've marked it OT for those of you interested exclusively in Ruby. Feel free to press delete, I won't be offended 
== BACKGROUND ==
For those of you without long memories, a quick recap. At the tail-end of last year Google released their Go language (http://www.golang.org) onto an unsuspecting world. The brainchild of Rob Pike and Ken Thompson, Go's sort of a reworking of Limbo from Plan 9/Inferno with baked-in concurrency and an inside-out approach to object orientation.
As those of you who've had the dubious pleasure of either sharing bar space with me or sitting through one of my conference sessions on Ruby & Unix, for all that Ruby's my language of choice I still spend a fair amount of time grubbing around in the C world. Love them or loathe them, there are times when we all need systems-level languages to efficiently solve certain kinds of problems.
The longterm goal of RubyGoLightly is to make Ruby a first-class citizen in that sense, moving systems coding firmly and permanently into our high-level world. I know it can be done as Ruby's pretty much as flexible as Lisp and Symbolics proved that an OS could be written in Lisp with minimal dependence on assembly language. So the aspiration isn't so lousy even if I'm far from certain that I'm personally capable of achieving it.
There is however one major roadblock to turning Ruby into a systems language: existing runtimes just aren't written that way. The closest we have at present is JRuby, and even there if you want complete control you still have to step outside of the Ruby mindset and hack Java, JVM bytecode or probably even write JNI extensions in C.
Then there's the question of an optimal Ruby runtime. There's a lot of implicit concurrency in the average Ruby program which to the best of my knowledge isn't being addressed by any of the existing implementations. We live in a multicore world where it's only natural that collection enumerators etc. should be exploiting those cores for cheap scalability. Admittedly writing concurrent systems in C can be a right pain in the arse, and even in Java there are subtleties arising from the architecture of the JVM so I'm not levelling criticism at anyone. It's just a fair observation that if I have four processor cores and a list with 10K entries that map/reduce algorithms should use all four cores if at all possible and return their results in approximately 1/3rd the time of a naive sequential implementation.
Given that Intel are playing with 80+ core processors in the lab - and that modern graphics cores have even more available horsepower via OpenCL - there's clearly an opportunity worth investigating.
Go does concurrency pretty well, especially considering that the runtime is still in its infancy and will probably undergo considerable optimisation in the future. It's basically an implementation of Hoare's CSP so all the tricks that can be used in Occam etc. for clean concurrency can be used in Go, and whilst it doesn't yet support OpenCL that's just a matter of time and a sufficiently nasty itch being worth scratching.
Another desirable trait for a Ruby runtime is decent browser integration. I get really annoyed having to write heavyweight client-side web code in JavaScript as to me it's an ugly language. I've probably never recovered from my painful experiences with NewtonScript as I find the entire Self language family unbearable, and the same goes for the slot-based GTK GUI stuff (sorry webGTK). Go has support for Google's Native Client so is ideal for injecting native code into web content, including a Ruby runtime were it to exist. I love that idea. Write your whole web app in Ruby, with identical abstractions.
The same thing could be achieved using a Ruby runtime written in JavaScript (such as HotRuby) but to date that approach has stalled. Basically writing any significant runtime platform is a significant cost and taking anything beyond the complexity of a simple bytecode interpreter means months of full-time hacking and serious cogitation.
The final thing in Go's favour is that it can compile to native code for bare metal - i.e. no OS at all, just a bootloader and an executable. I've yet to play with that side of the language, but having written assembler for embedded systems with no OS I would love to see a Ruby runtime deployed in the same manner. From there it's a short step to an OS written in Ruby, and that means an OS any half-competent programmer could understand in detail.
My colleague Romek and I have been boring people with that idea for about five years now, and mad though it probably seems I still believe it's a goal worth pursuing.
Anyway, to recap: the world might be a better place with a Ruby runtime that embraced pervasive concurrency in the runtime implementation, that was platform agnostic, and that allowed systems-level coding to be performed with the same ease we associate with Rails or even Sinatra. I won't pretend RubyGoLightly will tick all these boxes - right now it's an idea and ideas are by their nature transient. However it is certainly a framework in which to explore whether such a Ruby is possible.
== GoLightly ==
There are two aspects to this project. One is the Ruby runtime, which right now I have only the vaguest of ideas about how to implement. I'm following a hunch that a Forth-like Threaded Interpreter might allow optimisations that we're not going to see with mainstream bytecode interpreters and a lot of current effort is devoted to writing such a Forth implementation. Although I hesitate to call it Forth given the usual expectations of F79, F83, figForth or ANS94 compliance - none of them particular of particular importance.
Threaded interpretation isn't a million miles removed from the naive view of Ruby's object model so I'm hoping my current work will generate insights relevant to that, however its main purpose is to drive design of the GoLightly virtual machine which is an abstraction of the kind of hardware I'm interested in seeing Ruby run on.
There's an early branch of GoLightly on github (http://github.com/feyeleanor/GoLightly) which exposes some basic despatch loops (inlined bytecode and indirect function call) and a preliminary model for SIMD-style opcodes for basic vector processing and a supporting register model.
There are a few examples of hand-coded assembler in the tests to give a feel for the machine language along with some micro-benchmarks for anyone who fancies playing. GoLightly already copes with multiple virtual processors, all working on their discrete problems but there's a lot of work yet to be done in inter-core communication, the memory allocation model, and parallelising SIMD instructions.
One of the few things I dislike about Go as it currently stands - and which alas is considered a strength by the core team so is unlikely to change - is that the language doesn't support a representation-agnostic datatype suitable for holding bit-strings: basically numeric types always convert their contents to suit the type to which they're being coerced, or else a panic occurs. Getting around this involves playing games with pointers that make a fair amount of (void *) usage in C look clear and reasonable. The justification is that this sort of thing shouldn't be encouraged (with which I happen to disagree anyway) and that the extra pain is a bit of nudge to prevent it. I'll keep raising the issue with the core team every few weeks as I personally believe maintainability is much more important than type safety: after all, we can always write tests...
Anyway GoLightly is stalled whilst I evolve a reasonable API for handling bitstring manipulations, which is proving a subtly intractable problem. I'm hoping that hacking together a few languages will help me figure out most of the implementation details which are currently eluding me and then I'll roll a bitstring library back into the Go core library if the team will have it.
The unnamed Forth I'm currently working on is running into the same problems, but I'm still hoping to get a reasonable build ready for release on github over the summer. So far I have a good first cut of a compile/execute/decompile model which supports dynamic coding and I hope to reduce much of the Go code for implementing a core vocabulary to GoLightly-style bytecode. These bytecodes (which are effectively unique to the currently running system, so more a dynamic object code than a formal bytecode) would themselves be a first-class element of the language so that's an interesting model for both a native code interface and for allowing inline use of multiple languages.
== The Roadmap ==
From Forth to Ruby is one hell of a leap, and whilst I don't doubt it can be done in one step there are other problems I want to solve along the way.
I definitely see a simple Lisp being in the mix as I've always felt there is an equivalence between Forth and Lisp which goes unremarked: I guess there aren't that many people who've hacked with both, let alone who've written runtimes for both. Lisp will be the testbed for doing standard function call stack allocations and a few other aspects of implementing a high-level language which Forth skirts. Common Lisp it most definitely won't be though (or Arc for that matter).
From there I want to get a Scheme working, mostly to figure out how to handle tail-call recursion efficiently. It could be that this will be a specific compiler optimisation required separately for each supported language, but I'd really prefer it to work at the object code level so that it's applied across all compiled code automatically. Go uses relatively small stacks to improve concurrent performance compared to pThreads and whilst GoLightly threads are currently specced to use their own user-space threads I'd like to move beyond that in the future so tail call culling will be essential to not swamping the runtime stack.
After Scheme I'm hoping to implement a simple Logo system to get a handle on the interfaces offered by Google's Native Client etc. - it's a bit of a gratuitous diversion as Logo is hardly a mainstream language, but the underlying structure is similar enough to Lisp that I think it will be an easy build and there's been some interesting work in Logo-land to do with multi-threaded graphics which might have interesting tie-ins with OpenCL.
Much of the impetus to start hacking on RubyGoLightly is thanks to TinyRb, a really sweet subset of Ruby running on a Lua-style VM. Once I've solved the main runtime problems addressed by the Lisp arc of languages I plan to write a runtime interpreter for the TinyRb bytecode so its compiled form can run natively in GoLightly. My gut instinct is that this will be a great starting point for runtime optimisations that apply to a richer subset of Ruby.
At LSRC last year I showed off some of the code from a port of SQLite to Ruby that I was working on at the time. That project got put on hold when I started RubyGoLightly but given that SQLite is a really sweet engine and internally uses a register based VM it'd be easy to port to GoLightly and provide something like CoreData out of the box.
There could be a few steps after this: for one thing I've this gut feeling that doing a dirty implementation of Io could be instructive, though I can't really explain way. Something to do with the method passing semantics. A roadmap's no good if it doesn't allow for interesting detours
However the ultimate destination is RubyGoLightly, the Ruby runtime that scratches the itch of using Ruby as a systems language. No promises about where or when, and very few about how... but I promise to check in with another status report in a few months that will hopefully be more insightful.
In the meantime, happy hacking,
Ellie
Eleanor McHugh
Games With Brains
http://feyeleanor.tel
路路路
----
raise ArgumentError unless @reality.responds_to? :reason
