Seven new VMs, all in a row

Hi,

···

In message "Re: Seven new VMs, all in a row" on Fri, 8 Apr 2005 13:57:13 +0900, Bill Guindon <agorilla@gmail.com> writes:

Yes, and my point is that there's a fine line between "I'm not sure if
it's possible" and "I doubt it's possible". One challenges, the other
discourages. Btw, I admire that you're not sure, and if you aren't
sure, how can anyone else be sure until it's tried?

I take your point. I don't want to discourage anyone.
The world is before you, Peter, to be proved to be wrong.

              matz.

Hello Peter,

I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk
VM will be allocating Objects and JIT-ing bytecodes natively. There is
almost no Smalltalk in the Smalltalk VMs, and what semantics there are
are a very good fit for Ruby anyhow. All I'm doing is providing enough
meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby
compiler written in Ruby).

Is there a public available technical specification for the VM ?

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk

VM will be allocating Objects and JIT-ing bytecodes natively. There

is

almost no Smalltalk in the Smalltalk VMs, and what semantics there

are

are a very good fit for Ruby anyhow. All I'm doing is providing

enough

meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby
compiler written in Ruby).

I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
not, how easily can they be emulated?

- blocks (with Ruby semantics in case they're different from
Smalltalk's)
- mixins
- instance variables that don't need to be predeclared with the class
- class variables & constants
- throw/catch
- exceptions
- continuations
- support for OS facilities like select, signals etc

Too great a mismatch between Ruby and Smalltalk semantics will mean
that an emulation layer will be needed, which will presumably reduce
the performance gains. Of course, I hope that's not true! Good luck
with the project.

-- George

Makes sense to me. All in all, this sounds like a really cool project and I'm looking forward to trying it.

My Ruby skills are still a bit weak, but I know a lot about various low-level architectures. Koichi's talk last year was very interesting.

-- Matt
Nothing great was ever accomplished without _passion_

···

On Fri, 8 Apr 2005, Peter Suk wrote:

I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk VM will be allocating Objects and JIT-ing bytecodes natively. There is almost no Smalltalk in the Smalltalk VMs, and what semantics there are are a very good fit for Ruby anyhow. All I'm doing is providing enough meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby compiler written in Ruby).

(But this language defined in itself is a weird concept to most programmers, and I'll probably go blue in the face repeating this.)

Yukihiro Matsumoto wrote:

I take your point. I don't want to discourage anyone.
The world is before you, Peter, to be proved to be wrong.

            matz.

I feel impossibly good right now. Go now, Peter the young-breasted. Defeat is an accomplishment not even the best of us could achieve.

_why

Hello George,

I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
not, how easily can they be emulated?

I would like to add

- singleton methods

to this list as i think this does not exist in SmallTalk, and this is
a feature that could knock out the whole method compilation algorithm.

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
not, how easily can they be emulated?

- blocks (with Ruby semantics in case they're different from
Smalltalk's)
- mixins
- instance variables that don't need to be predeclared with the class
- class variables & constants
- throw/catch
- exceptions
- continuations
- support for OS facilities like select, signals etc

I still have to look into continuations and OS facilities in detail. (Pointers to where these are described in detail for Ruby?) However I believe continuations are supported for Smalltalk exceptions. Mixins will be supported by the Compiler manipulating Single Inheritance when Classes are defined. But as for everything else:

- blocks (with Ruby semantics in case they're different from
Smalltalk's)

Almost the same as Smalltalk's.

- instance variables that don't need to be predeclared with the class

Can be added dynamically in Smalltalk.

- class variables & constants

Check.

- throw/catch

Really the same as below.

- exceptions

Check.

From now on, if you are completely ignorant of Smalltalk, I will cease answering these questions here.

--Peter

···

On Apr 8, 2005, at 3:39 AM, George wrote:

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

George wrote:

I'm curious - how good a fit are Ruby semantics for Smalltalk VMs?

They fit almost exactly, which is why 20-30x is not an outrageous speed
claim, and why comparisons with Mono, JVM, Parrot etc are entirely
missing the point: this is not equivalent to compiling Ruby to Java
classes, say, but to realising that Sun's Hotspot team had been writing
a VM *specifically to run Ruby* all along - which they effectively were
before Sun crippled it, but that's another story.

Do
Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
not, how easily can they be emulated?

- blocks (with Ruby semantics in case they're different from
Smalltalk's)

Yes.

- mixins

Yes. Lothar also asked about singleton methods, to which the answer is
also Yes.

- instance variables that don't need to be predeclared with the class

Not exactly as Ruby does, but you can simply add new instance variables
to the class definition whenever you compile a method that uses a new
one.

- class variables & constants

Yes.

- throw/catch

Yes.

- exceptions

Yes.

- continuations

Yes (depending on the VM; Squeak, VisualWorks, and Dolphin all support
them).

- support for OS facilities like select, signals etc

Not portably; the OS facilities and binary extension mechanisms are
highly specific to VMs.

One thing nobody's asked about is variable and optional arguments for
methods. Some VMs (like Squeak's) can handle this just fine, but some
may require that the number of args pushed onto the stack match the
selector. In any case, to do it optimally it would require generating
bytecode, not Smalltalk code, which makes targetting a whole range of
VMs difficult; to do it portably would mean creating an extra Array on
every message send.

Avi

http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/oe-tour-sept19.pdf

Check.

http://c2.com/cgi/wiki?CallWithCurrentContinuation

···

On Apr 8, 2005, at 3:39 AM, George wrote:

- continuations

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

Nope. (Rest is left as an excercise.)

···

On Apr 8, 2005, at 3:49 AM, Lothar Scholz wrote:

Hello George,

> I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
> Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
> not, how easily can they be emulated?

I would like to add

- singleton methods

to this list as i think this does not exist in SmallTalk, and this is
a feature that could knock out the whole method compilation algorithm.

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

Sorry, my last post was meant to be in reply to Lothar's request for a
VM spec. I still haven't got the hang of this new google groups UI...

Avi

Great, thanks Avi. Can I assume the commercial VMs are similar to this
or do you know of detailed descriptions of them?

BTW, wasn't Peter Deutsch himself onto a similar project for Python:

http://blog.amber.org/2004/08/12/python-on-smalltalks-vm/

Anyone seen something come out from that?

Best,

Robert

···

On Apr 8, 2005 11:54 AM, Avi Bryant <avi.bryant@gmail.com> wrote:

http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/oe-tour-sept19.pdf

Avi Bryant wrote:

- instance variables that don't need to be predeclared with the class

Not exactly as Ruby does, but you can simply add new instance variables
to the class definition whenever you compile a method that uses a new
one.

Isn't that jumping the gun just a bit? An instance variable (in Ruby) should not exist in an object until the line that assigns/creates it is actually executed. It's a subtle point, but it could impact some types of reflective programming. Maybe you can mask its existence somehow?

···

--
Glenn Parker | glenn.parker-AT-comcast.net | <http://www.tetrafoil.com/&gt;

Avi / Peter -- thanks for providing the feedback I was looking for.
Ruby and Smalltalk do indeed sound like a good match. I remember Robert
Feldt saying a few years ago that he'd done some experiments with Ruby

···

on Smallscript's VM that looked promising. Peter said:

From now on, if you are completely ignorant of Smalltalk,
I will cease answering these questions here.

Yikes! I _am_ almost completely ignorant of Smalltalk (I'm know the
basic concepts of the language, but I've never programmed in it), but
I've done some work on VMs and was genuinely interested in the answers.

"Lothar Scholz" <mailinglists@scriptolutions.com> wrote in message

I would like to add

- singleton methods

to this list as i think this does not exist in SmallTalk, and this is
a feature that could knock out the whole method compilation algorithm.

I doubt it will. Even in Ruby, singleton methods are implemented with
classes under the covers.

The fact that Ruby instance variables appear on demand, and so references to
instance-vars cannot be directly mapped to integer offsets from a known
class definition, will likely mean some performance trade-off.

Hello Avi,

Sorry, my last post was meant to be in reply to Lothar's request for a
VM spec. I still haven't got the hang of this new google groups UI...

Yes, nice but this is Squeak. It seems that this is just a bytecode
machine without a JIT. Is there any document available vor VisualWorks ?

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

Robert Feldt wrote:

>

http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/oe-tour-sept19.pdf

>
Great, thanks Avi. Can I assume the commercial VMs are similar to

this

or do you know of detailed descriptions of them?

I don't know of any detailed descriptions. Apart from the obvious fact
that they have JITs and the stock Squeak engine doesn't, one difference
I know is that VisualWorks still uses an object table rather than
direct object references, which makes #become: much faster and probably
helps with garbage collection. However, for the purpose of knowing how
to model Ruby on top of the VM (the object format, classes and method
dictionaries, how context stacks work), they should all be very
similar, at least on the surface - never mind what optimizations are
going on below.

BTW, wasn't Peter Deutsch himself onto a similar project for Python:

http://blog.amber.org/2004/08/12/python-on-smalltalks-vm/

Yes, but that's a much harder problem, since Python has very different
semantics from Smalltalk, whereas Ruby's are for most purposes
identical.

Avi

···

On Apr 8, 2005 11:54 AM, Avi Bryant <avi.bryant@gmail.com> wrote:

That's right George. However, I concluded Smallscript was not open
enough to build on at that time and haven't followed it since then.
Also I just did some ruby2smallscript (source to source) translation
experiments so it was a different approach.

Best,

Robert

···

On Apr 8, 2005 3:19 PM, George <george.marrows@ps.ge.com> wrote:

Avi / Peter -- thanks for providing the feedback I was looking for.
Ruby and Smalltalk do indeed sound like a good match. I remember Robert
Feldt saying a few years ago that he'd done some experiments with Ruby
on Smallscript's VM that looked promising.

Glenn Parker wrote:

Avi Bryant wrote:
>
>>- instance variables that don't need to be predeclared with the

class

>
> Not exactly as Ruby does, but you can simply add new instance

variables

> to the class definition whenever you compile a method that uses a

new

> one.

Isn't that jumping the gun just a bit? An instance variable (in

Ruby)

should not exist in an object until the line that assigns/creates it

is

actually executed. It's a subtle point, but it could impact some

types

of reflective programming. Maybe you can mask its existence somehow?

Yes, pretty easily I'd think. For example: when you create a new
instance in Smalltalk, all the instance variables start out initialized
to Smalltalk's nil value (of class UndefinedObject). As soon as it was
referenced or assigned to, that would get replaced with some Ruby value
(possibly Ruby's nil, of class NilClass). So you could always tell for
a given instance which instance variables already "exist" inside the
Ruby semantics.

You could also just use a hashtable for each instance to hold all of
the variables, like Ruby does, but the fact that Smalltalk can do
direct instance variable access ends up being a nice speed and memory
gain, so I'd rather not give that up.

Avi