Seven new VMs, all in a row

ruby-talk
21

On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:

Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM. But you can never get all the binary extensions to work,
an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,
since this would not have so much todo with the current ruby anymore.

  OTOH, if it is 30x faster and makes an alternate implementation of socket
(which likely would not have any speed deficiency) and a few other extensions
it may become an attractive branch to use for many people.

  Seems like an idea worth pursuing to me...Proofs in the pudding though.
It will be interesting to see what gains materialize...

-Tom

···

--
+ http://www.tc.umn.edu/~enebo +---- mailto:enebo@acm.org ----+

Thomas E Enebo, Protagonist | "A word is worth a thousand |
                             > pictures" -Bruce Tognazzini |

22

Wow, could you possibly be more negative?

This kind of dismissive overlording behavior does not reflect the usual spirit of acceptance and togetherness found on this list, especially since so many projects of the same spirit are in progress at the same time (YARV, Rubidium, JRuby, MetaRuby and a few less-publicized ones).

This is a time for offering constructive criticism and helpful insight into the difficult problems ahead, not a time to say things are impossible!

PGP.sig (194 Bytes)

···

On 07 Apr 2005, at 14:32, Lothar Scholz wrote:

Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM. But you can never get all the binary extensions to work,
an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

23

Hello Peter,

> to the class being displayed. The effect is basically an
> "irb-anywhere" which you can arbitrarily interleave with any number of

So it can crash anywhere :slight_smile: SCNR.

In general this sort of thing doesn't crash under Smalltalk. In fact, the debugger is just a normal application doing a few funky things with Processes. It's just a browser for the reified context stack. In fact, you can write your own feature-poor, but working debugger in Smalltalk and have it running in a matter or minutes.

Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM.

Not what I'm trying to do. The Smalltalk VM has basically *none* of the *language* in it.

But you can never get all the binary extensions to work,

Why not? There are plenty of binary extensions to the various Smalltalks. They are designed to make it easy to build in binary extensions.

an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,
since this would not have so much todo with the current ruby anymore.

You missed something in my previous posts, or do not have some pieces of background concerning how Smalltalk works. There is no emulation going on, except stepping during debugging, and that happens at the level of the bytecodes so all VM/Object semantics are preserved. Ruby will be running on top of a Just-In-Time compiler, translating the bytecodes into native machine instructions. The VisualWorks JIT has been in development since 1989, so is one of the most mature JIT interpreters around. Its speed rivals Hotspot in real-world situations where GC is significant. Object allocation will be native -- Objects will just be Objects to the VM/garbage collector, with the Ruby/Smalltalk implementation language being moot.

But i'm courious whats your result in the next years.
Is this your Ph.D thesis ?

No. I want to make the world safe for Pure-OO. A way I can help Pure-OO development is to vastly increase the power of the Ruby community by giving it access to the great technology developed for Smalltalk.

Other things that putting Ruby on top of Smalltalk VMs could garner for the Ruby community:

The Smalltalk MT environment can produce native Windows DLLs that are indistinguishable from C++ DLLs. We would theoretically be able to do the same for Ruby when we can host Ruby on that VM. There could be a Gemstone Ruby -- with a name like that there *should* be -- giving the community access to a great OODB. There is also a fast Smalltalk VM for .NET which could be used for Ruby. If we can overcome the lack of real Namespaces in Squeak, then we'd have a Ruby VM that runs completely bit-identical on 32 platforms.

Also, I'd like to do automated syntax + type-inference Ruby <--> Smalltalk translations so that Ruby can benefit from the wealth of great stuff done in Smalltalk, and Smalltalkers can be invigorated by contact with a larger, very active community.

--Peter

···

On Apr 7, 2005, at 4:32 PM, Lothar Scholz wrote:

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

24

Squeak alone has been ported to 32 platforms, running in all of them bit-identically, including Acorn Risc, WinCE, Mac OSX, Mac System 9, Windows, Linux, various Unixes. There is a Smalltalk VM for PalmOS. (slow, though) The first target, VisualWorks, runs on 8 platforms, including Various Windows, WinCE, Mac 8, Mac 9, Mac OSX, HPUX, AIX, IRIX, Solaris-SPARC, Linux x86, Linux SPARC, Linux PPC.

--Peter

···

On Apr 7, 2005, at 4:45 PM, Glenn Parker wrote:

So, what are the potential platforms for Ruby on a Smalltalk VM?

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

25

In article <20050407220145.GI23956@garnet.tc.umn.edu>,

···

Thomas E Enebo <enebo@acm.org> wrote:

On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:

Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM. But you can never get all the binary extensions to work,
an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,
since this would not have so much todo with the current ruby anymore.

OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

Phil

26

Hello Eric,

Wow, could you possibly be more negative?

Yes, but i don't think i was very negative. Its just the observation
that some of the most important parts of ruby are not written in ruby
and must be manually convert (Basic library, GUI Bindings, etc.)
The easy creation of binary extensions is a very important point for
me and the reason why i switched from TCL to Ruby.

This kind of dismissive overlording behavior does not reflect the usual
spirit of acceptance and togetherness found on this list, especially
since so many projects of the same spirit are in progress at the same
time (YARV, Rubidium, JRuby, MetaRuby and a few less-publicized ones).

The great number of projects which tries to do this and never come up with
anything useable yet (except JRuby which is an interpreter rewrite
in Java and does not belong to this list) should give anybody
a break to think about the problems a little bit more.

This is a time for offering constructive criticism and helpful insight
into the difficult problems ahead, not a time to say things are
impossible!

I think it's extremely hard to do this task. And that's why i asked if
this is his Ph.D. to see if he has the huge amount of time to get this
up and running. And i did a google search and found only 5 postings from
Peter Suk since 2.Okt 2002 on this mailing list and some other
information - like that he did a port of the Smalltalk refactoring
browser. So it seems to be okay that i'm a little bit suspicious if
this is one of the many ruby newbie ideas that we see here every few
weeks or if we can start serious discussions.

I wish him good luck for this project and hope he is successful.
I'm sure i will try it out and offer constructive criticism in a later
stage of the project.

···

On 07 Apr 2005, at 14:32, Lothar Scholz wrote:

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

27

Hello Peter, hello List,

Slightly OT: Does Python ship with a JIT or JIT option?

Personally I always wondered why interpreted languages dont ship with any kind of JIT.
From a newbies point of view I understood this, before you mentioned the JIT, like the idea to take the ruby language and put it onto a very fast, general purpose VM, instead of using its own interpreter, which might be great but maybe not so fast.
If that's the point, then its a great thing - as long as security and features are kept more speed is always better.

Peter Suk wrote:

···

[...]
Ruby will be running on top of a Just-In-Time compiler, translating the bytecodes into native machine instructions. The VisualWorks JIT has been in development since 1989, so is one of the most mature JIT interpreters around. Its speed rivals Hotspot in real-world situations where GC is significant. Object allocation will be native -- Objects will just be Objects to the VM/garbage collector, with the Ruby/Smalltalk implementation language being moot.

> [...]

No. I want to make the world safe for Pure-OO. A way I can help Pure-OO development is to vastly increase the power of the Ruby community by giving it access to the great technology developed for Smalltalk.

The Smalltalk MT environment can produce native Windows DLLs that are indistinguishable from C++ DLLs. We would theoretically be able to do the same for Ruby when we can host Ruby on that VM.

> [...]

--
ionas
Yes, I am a newbie.

28

I might be skeptical, but if I voice my skepticism loud enough, I may
discourage those who might try (and may succeed).

So yeah... I think that 30x faster is quite possible :wink:

···

On Apr 7, 2005 10:04 PM, Phil Tomson <ptkwt@aracnet.com> wrote:

In article <20050407220145.GI23956@garnet.tc.umn.edu>,
Thomas E Enebo <enebo@acm.org> wrote:
>On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
>>
>> Yes maybe its possible to transfer part of the language (all?) into an
>> smalltalk VM. But you can never get all the binary extensions to work,
>> an emulation layer would kill all your speed benefits if possible
>> at all. So maybe you should simply announce a ruby branch called SmallRuby,
>> since this would not have so much todo with the current ruby anymore.
>
> OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

--
Bill Guindon (aka aGorilla)

29

> OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

Phil

For what I've seen, reusing something like the Mono native backend
(available for major processor architectures) can yield even higher
performance gains.
A custom tailored VM in assembler can provide 100x and possibly more
depending on the architecture.

30x actually looks very conservative to me, but may well be worth the
effort to reuse an already existing codebase.

30

Phil Tomson ha scritto:

OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

well, take a look at the OO Richards Benchmark
http://pws.prserv.net/dlissett/ben/
the VW vms is something like 70x faster than ruby, so if not too much is wasted in the layering, there is still a fast vm.
IIRC there is also someone working on a python port on top of VW, which should be quite fast.

(btw, remember the usual "lies, damn lies & benchmarks"

31

Jonas Hartmann ha scritto:

Hello Peter, hello List,

Slightly OT: Does Python ship with a JIT or JIT option?

no, Python is just a "normal" vm. There is a python JIT for x86, psyco, sich works as a mofule for Python.

Personally I always wondered why interpreted languages dont ship with any kind of JIT.

I guess... because it's hard :slight_smile:

PS
please try to avoid top posting
http://www.caliburn.nl/topposting.html

32

Phil Tomson ÐÉÛÅÔ:

In article <20050407220145.GI23956@garnet.tc.umn.edu>,

On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:

Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM. But you can never get all the binary extensions to work,
an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,
since this would not have so much todo with the current ruby anymore.

OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

Don't be too sceptical:

fib(32): ruby - 7297ms, visualworks 7.2.1 - 79ms

~90x :wink:

···

Thomas E Enebo <enebo@acm.org> wrote:

--
wbr, ps
sphilippov-at-gmail-com

33

In article <67a2229205040719147fec0f8a@mail.gmail.com>,

In article <20050407220145.GI23956@garnet.tc.umn.edu>,
>On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
>>
>> Yes maybe its possible to transfer part of the language (all?) into an
>> smalltalk VM. But you can never get all the binary extensions to work,
>> an emulation layer would kill all your speed benefits if possible
>> at all. So maybe you should simply announce a ruby branch called

SmallRuby,

>> since this would not have so much todo with the current ruby anymore.
>
> OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

I might be skeptical, but if I voice my skepticism loud enough, I may
discourage those who might try (and may succeed).

So yeah... I think that 30x faster is quite possible :wink:

Please, prove my skepticism to be wrong. A Ruby that was 30x faster (on
the same hardware) would be great. As I said in the other post
responding to Avi, if it's possible to speed things up that much with a
SmallTalk VM, maybe we should be studying said VM to figure out how to
make it happen in YARV (since matz has recently said that YARV is the
future VM for Ruby).

Phil

···

Bill Guindon <agorilla@gmail.com> wrote:

On Apr 7, 2005 10:04 PM, Phil Tomson <ptkwt@aracnet.com> wrote:

Thomas E Enebo <enebo@acm.org> wrote:

34

Hello vruz,

> OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

Phil

For what I've seen, reusing something like the Mono native backend
(available for major processor architectures) can yield even higher
performance gains.
A custom tailored VM in assembler can provide 100x and possibly more
depending on the architecture.

No. You simply forget that the Mono/.NET VM's are optimized for static
typed languages and they compile a frontend language into a classic
3 register instruction machine, not a stack based machine like JAVA
giving much more chance for optimizations.
There i don't see a problem for such high performance boosts.

But if we can't come up with a type inference machine, the theoretical
improvements (and thats what is done by YARV or Parot) are far from
that. If you are willing to spend a huge amount of memory you can go
the python way and keep multiple compiled versions (based on
differently use types) of small code fragments translated.

In 1995 i've written an emacs lisp compatible interpreter and it was
easy to get 8x more speed then the current one with program flow
optimizations (a FORTH like direct threaded interpreter) and together with
integration of the Boehm-Weisser GC and more optimizations i think it's easy
to get even a 20x improvement (especially on Ruby where the implementation
is not very sophisticated).

We can use techniques like Eiffel's system wide optimizations to get the
cost of a method call down to a "call 0x05346403" in most cases, so with
elimination of a VMT we can be even faster then C++.

But then we still need cascaded "if (argument = TYPE(Integer)) x += VALUE(argument)".

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

35

vruz ha scritto:

OTOH, if it is 30x faster and makes an alternate implementation of socket

I'm very skeptical about the 30x faster claim.

Phil

For what I've seen, reusing something like the Mono native backend
(available for major processor architectures) can yield even higher
performance gains.

I disagree. The IronPython implementation, on mono/x86 runs just 1.2 times faster than CPython.
On PPC IronPyrthon/mono runs even slower than the real CPython.
Surely there are still work to do, but I guess those "the clr is not great for dynamic languages" claims may be true (I still think you may found value in using it for interoperability)

36

In article <67a2229205040719147fec0f8a@mail.gmail.com>,
>> In article <20050407220145.GI23956@garnet.tc.umn.edu>,
>> >On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
>> >>
>> >> Yes maybe its possible to transfer part of the language (all?) into an
>> >> smalltalk VM. But you can never get all the binary extensions to work,
>> >> an emulation layer would kill all your speed benefits if possible
>> >> at all. So maybe you should simply announce a ruby branch called
>SmallRuby,
>> >> since this would not have so much todo with the current ruby anymore.
>> >
>> > OTOH, if it is 30x faster and makes an alternate implementation of socket
>>
>> I'm very skeptical about the 30x faster claim.
>>
>
>I might be skeptical, but if I voice my skepticism loud enough, I may
>discourage those who might try (and may succeed).
>
>So yeah... I think that 30x faster is quite possible :wink:
>

Please, prove my skepticism to be wrong.

I leave that job to Peter, my job is to encourage him to try.

A Ruby that was 30x faster (on
the same hardware) would be great. As I said in the other post
responding to Avi, if it's possible to speed things up that much with a
SmallTalk VM, maybe we should be studying said VM to figure out how to
make it happen in YARV (since matz has recently said that YARV is the
future VM for Ruby).

Agreed, and I hope Peter considers that, but if he's hell bent on
building one on top of a SmallTalk VM, then I say go right ahead, and
if it works, we can dissect that later.

···

On Apr 8, 2005 12:19 AM, Phil Tomson <ptkwt@aracnet.com> wrote:

Bill Guindon <agorilla@gmail.com> wrote:
>On Apr 7, 2005 10:04 PM, Phil Tomson <ptkwt@aracnet.com> wrote:
>> Thomas E Enebo <enebo@acm.org> wrote:

--
Bill Guindon (aka aGorilla)

37

Hi,

···

In message "Re: Seven new VMs, all in a row" on Fri, 8 Apr 2005 13:19:41 +0900, ptkwt@aracnet.com (Phil Tomson) writes:

Please, prove my skepticism to be wrong. A Ruby that was 30x faster (on
the same hardware) would be great. As I said in the other post
responding to Avi, if it's possible to speed things up that much with a
SmallTalk VM, maybe we should be studying said VM to figure out how to
make it happen in YARV (since matz has recently said that YARV is the
future VM for Ruby).

It's not impossible to make Ruby run 30x faster for some tasks, for
example, YARV runs more than 100x faster in some small benchmarks.
But I'm not sure if it's possible to make it 30x faster in average.

              matz.

38

Yes, and my point is that there's a fine line between "I'm not sure if
it's possible" and "I doubt it's possible". One challenges, the other
discourages. Btw, I admire that you're not sure, and if you aren't
sure, how can anyone else be sure until it's tried?

Not knowing if it's possible, if Peter wants to try, we should
encourage it. If he's right, and we're wrong, and we convince him not
to try, we all lose. If he's wrong, what has he wasted but his own
time? He's not on my payroll, so try he should! If he were on my
payroll, I might think differently :wink:

Here's the way I see it, in Ruby...

class Person
  def initialize
    srand Time.now.to_i
    @wants = (rand(24) + 64).chr
  end

  # what a person wants is (somewhat) read only, and fairly random.
  def wants
    "Wants: " + @wants
  end

  # what a person works on is usually the same as what they want.
  def works_on
    "Works on: " + @wants
  end
end

p = Person.new
puts p.wants
puts p.works_on

p.wants = "Z"

···

On Apr 8, 2005 12:45 AM, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:

Hi,

In message "Re: Seven new VMs, all in a row" > on Fri, 8 Apr 2005 13:19:41 +0900, ptkwt@aracnet.com (Phil Tomson) writes:

>Please, prove my skepticism to be wrong. A Ruby that was 30x faster (on
>the same hardware) would be great. As I said in the other post
>responding to Avi, if it's possible to speed things up that much with a
>SmallTalk VM, maybe we should be studying said VM to figure out how to
>make it happen in YARV (since matz has recently said that YARV is the
>future VM for Ruby).

It's not impossible to make Ruby run 30x faster for some tasks, for
example, YARV runs more than 100x faster in some small benchmarks.
But I'm not sure if it's possible to make it 30x faster in average.

--
Bill Guindon (aka aGorilla)

39

Hello Yukihiro,

It's not impossible to make Ruby run 30x faster for some tasks, for
example, YARV runs more than 100x faster in some small benchmarks.
But I'm not sure if it's possible to make it 30x faster in average.

Since Smalltalk has only very few binary implemented functions i think
we should set the benchmark at something like the REXML parser which
does a lot of pure ruby processing.

And yes i think for tasks like this it is possible to get 30x faster,
but i think it might even be possible for YARV to get 20x faster.

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

40

Bill,

I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk VM will be allocating Objects and JIT-ing bytecodes natively. There is almost no Smalltalk in the Smalltalk VMs, and what semantics there are are a very good fit for Ruby anyhow. All I'm doing is providing enough meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby compiler written in Ruby).

(But this language defined in itself is a weird concept to most programmers, and I'll probably go blue in the face repeating this.)

--Peter

···

On Apr 7, 2005, at 11:28 PM, Bill Guindon wrote:

On Apr 8, 2005 12:19 AM, Phil Tomson <ptkwt@aracnet.com> wrote:

In article <67a2229205040719147fec0f8a@mail.gmail.com>,
Bill Guindon <agorilla@gmail.com> wrote:
(since matz has recently said that YARV is the
future VM for Ruby).

Agreed, and I hope Peter considers that, but if he's hell bent on
building one on top of a SmallTalk VM, then I say go right ahead, and
if it works, we can dissect that later.

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.