Zed and Luis drop the bomb on Ruby's poor performance

ruby-talk
1

The O'Reilly blog has two great interviews posted from the last week or so. The first [1] is with Zed Shaw, the author of Mongrel (and a bunch of other Ruby frameworks and utilities). A follow-up interview with one of the contributors on the Mongrel project, Luis Lavena [2], covered some similar ground.

What I found most striking about both interviews was their mention of YARV and Ruby's poor computational performance. Zed said, and I quote:

I’ll be honest right away though and say that Ruby is slow. The Ruby community has been ignoring the huge “performance” elephant standing in the room and they need to start talking about it so it goes away. Elephants hate being talked about. There are a few efforts to make Ruby faster, but I see a lot less action than is needed to solve the problem. One solution in the works is a real virtual machine called Rite (or YARV depending on who you talk to) which is showing some real promise and seems to be speed competitive with the fastest Java implementations.

I tend to agree with this. Whenever the subject of computational performance comes up, quite a few posts end up in the thread questioning the meaning of the word performance and turn it into an argument over semantics. Is it coding speed, clarity of thought, readability, etc. To me that kind of crazy semantic dodge doesn't serve us well. Those discussions should be remain on computational performance since the language has been (mostly) stable in syntax for years.

So what is being done to get YARV the attention it deserves? Why aren't we talking about it all the time? What can I personally do to help? Are there valid excuses for not focusing on improving Ruby's poor computational performance?

cr

[1] O'Reilly Media - Technology and Business Training
[2] O'Reilly Media - Technology and Business Training

2

One question I have to ask is "Wouldn't it make more sense to piggy back on parrot (the Perl v6) engine?" and get all the benefit of the performance improvements that parrot will pick up and the fact that there are many, many very clever people working on parrot whose work we could make use of.

Is there a reason why we are reinventing the wheel?

3

unknown wrote:

So what is being done to get YARV the attention it deserves?

AFAIK YARV will be the default VM when ruby 2.0 comes out. And I think
2.0 will be the next release of ruby. Becoming the default VM seems like
as much attention as you can get.

Why aren't we talking about it all the time?

It will be in the next major release, what more is there to talk about?
:slight_smile: At the moment it is still not 100% compatible with ruby 1.8 (and
might never be), but it works for a lot of ruby scripts/programs.

What can I personally do to help?

Check out their site:
http://www.atdot.net/yarv/#i-1

Edwin

···

--
Posted via http://www.ruby-forum.com/\.

4

*snip*

So what is being done to get YARV the attention it deserves? Why
aren't we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby's
poor computational performance?

There vertainly are perfectly good "excuses" (I didn't know I needed
one! :slight_smile: ) for not focussing on Ruby's performance - it's performance is
only a concern for one (poor or otherwise) if it's actually causing one
a problem.

In my previous incarnation as a signal processing and volume rendering
coder, I imagine Ruby's performance would have been unnaceptable. In my
current role as a testing framework programmer, it's plenty fast thanks,
and jolly expressive. I've no reason (at the moment) to require more
performance from it. When I switch back to doing graphics again, I'll be
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it'll probably be fast enough for that too.

All that said - I'll crack open a drink to Ruby's performance improving.
I just want to point out why it doesn't (pragmatically) matter to me
personally atm.

Cheers,
  Benjohn

5

The O'Reilly blog has two great interviews posted from the last week
or so. The first [1] is with Zed Shaw, the author of Mongrel (and a
bunch of other Ruby frameworks and utilities). A follow-up interview
with one of the contributors on the Mongrel project, Luis Lavena [2],
covered some similar ground.

First, thanks for reading and pointing out the interviews. I'm glad people
are enjoying them. I'm hoping to do some more in the coming weeks/months.

What I found most striking about both interviews was their mention of
YARV and Ruby's poor computational performance. Zed said, and I quote:

> I'll be honest right away though and say that Ruby is slow. The
> Ruby community has been ignoring the huge "performance" elephant
> standing in the room and they need to start talking about it so it
> goes away. Elephants hate being talked about. There are a few
> efforts to make Ruby faster, but I see a lot less action than is
> needed to solve the problem. One solution in the works is a real
> virtual machine called Rite (or YARV depending on who you talk to)
> which is showing some real promise and seems to be speed
> competitive with the fastest Java implementations.

I think you're quoting selectively here. There are some counter points in
both interviews. Luis said:
  "For us, Ruby on Rails (RoR) lets us build *fast*, clean
  (especially clean) prototypes within hours or days, not
  weeks or months. Then we can move from prototype to
  final product within weeks, even for big projects (like
  migrating a whole application from Zope to RoR)."
(emphasis added)

Zed said:
  "Ruby's advantage though is not in it's blazing
  execution speed but it's blazing coding speed.
  I'll put it to you this way: I wrote mongrel in about
  3 months. That's a full featured stable web server
  that can run four Ruby web application frameworks
  and is already powering many Ruby web sites. This
  wouldn't be possible without Ruby the language."
He also wrote:
  "[A] Rails application many times can outperform
  similar applications in Java or PHP."

This doesn't mean Ruby is a speed demon, or that Zed and Luis
don't want to see general speed increases (we all do, and they
seem to be on the horizon).

For most users (as has been repeated on this thread a few times),
Ruby is fast enough, and easy to optimize when it's not (RubyInline
for pure Ruby, cacheing for Rails).

I tend to agree with this. Whenever the subject of computational
performance comes up, quite a few posts end up in the thread
questioning the meaning of the word performance and turn it into an
argument over semantics. Is it coding speed, clarity of thought,
readability, etc. To me that kind of crazy semantic dodge doesn't
serve us well. Those discussions should be remain on computational
performance since the language has been (mostly) stable in syntax for
years.

I think the main argument you will run into (over and over again) is, have
you profiled your code? Do you know what's running slowly? If not, you
may just be moving through a bottleneck a little faster once YARV gets
here. Really, I'm all for improvements to Ruby in general, but profiling
and optimizing your bottlenecks by writing better code will help you out
of a number of tight spots.

So what is being done to get YARV the attention it deserves? Why
aren't we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby's
poor computational performance?

Are you on the Ruby Core mailing list? Are you posting bugs and
feature requests to the trackers? Have you downloaded and built
YARV so that you can test it on your system? All of these are ways
you can get involved and help out.

···

On 5/22/06, cremes.devlist@mac.com <cremes.devlist@mac.com> wrote:

cr

[1] O'Reilly Media - Technology and Business Training
[2] O'Reilly Media - Technology and Business Training

--
thanks,
-pate
-------------------------

6

I've always had a sense that some of what makes Ruby so beautiful is
_precisely_ what makes it slow. Some of this is a matter of
implementation (if anyone remembers the early implementations of ML,
you know what good tail-recursion can do for you. Plus Ruby seems to
get super-linearly worse as its working set grows), but some of it is
fundamental. Metaprogramming is very hard to do without depending on a
lot of string comparing at runtime. I've got a language design
background (I made a lot of money writing compilers for scripting
languages), and I've had to deal with this problem in other languages.
YARV looks like beautiful work, but I have a gut-feeling that it won't
solve the "speed" problem all by itself.

I won't wade into the whole "is Ruby fast enough" question because
defining "enough" really deserves its own thread. It's an important
question though, especially as regards Ruby in corporate IT.

7

Hokay some concrete suggestions on speeding up the here and now....

A Good Thing would be to have a Performance test suite. You can't
optimize unless you have a realistic measure of performance.

That would be a Good Thing for someone to create.

Ruby is based on gcc, gcc has grown some nifty ways of speeding things
up.

Much of ruby was written _before_ gcc grew these features but it is
really worth considering now.

Simple way number one.

Currently Ruby is compiled -O2 usually for generic i386

Obvious answers
  - Compile -O3, gcc, especially since version 4.0 has grown some really nifty
    optimizations. Almost the whole point of version gcc 4

  - Compile -march=pentium4, or -march=k7 or whatever the machine it's
    actually running on is. This gives you...
    - Tuned for the pipeline of that CPU.
    - Uses the new fancy instructions for that CPU.

  - gcc has / is growing profile based branch prediction. This is something
    ideal for an interpreted language with a largish test suite!

Not quite so easy answers...

Use gprof to get some profile data.

* Use inline functions. C++ and STL have forced all compilers today to
   be very very Good with inline functions. In several respects inlines are
   better than macros.

* const and pure attributes can speed things up too.

Ruby is _not_ gcc warning clean. Some gcc warnings are actually performance related.

Especially those to do with...
   * Alignment.
   * Strict aliasing.
   * Pointer casts.

A community "lets all clean up get Ruby -Wall -W clean" would be A Good Thing.

The register / cache / memory hierarchy speed differences are becoming
really really huge in modern machines.

Some thought as to keeping things "in cache" especially in things with
tight inner loops like Ruby can make a Big difference.

Thus a Good Thing to try would be to feed Ruby to cachegrind (a tool in
the valgrind.org suite)

John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong later."

From this principle, all of life and physics may be deduced.

···

On Mon, 22 May 2006, cremes.devlist@mac.com wrote:

I’ll be honest right away though and say that Ruby is slow. The Ruby community has been ignoring the huge “performance” elephant standing in the room and they need to start talking about it so it goes away. Elephants hate being talked about. There are a few efforts to make Ruby faster, but I see a lot less action than is needed to solve the problem.

8

YARV and Ruby's poor computational performance. Zed said, and I quote:

I?ll be honest right away though and say that Ruby is slow. The
Ruby community has been ignoring the huge ?performance? elephant
standing in the room and they need to start talking about it so it
goes away. Elephants hate being talked about. There are a few
efforts to make Ruby faster, but I see a lot less action than is
needed to solve the problem. One solution in the works is a real
virtual machine called Rite (or YARV depending on who you talk to)
which is showing some real promise and seems to be speed
competitive with the fastest Java implementations.

Yeah I have come to the sad conclusion that, at least with the 1.8.x
world, if speedy execution is your goal you should probably choose
another scripting language [python+psycho comes to mind]. Especially
when considered in conjunction with rails. There I have mentioned the
elephant :slight_smile:
That being said, thankfully 95% of web pages and scripts don't really
care about getting hammered, since they never will be. But the other 5%
will suffer until this gets figured out. And I'm not volunteering.
Hopefully this will improve in the near future. Until then back to my
coding of some poorly performing, elegantly written web pages.
-R

···

--
Posted via http://www.ruby-forum.com/\.

9

http://wiki.rubygarden.org/Ruby/page/show/VirtualMachineOptions

Cardinal
A re-implementation of Ruby on top of the [Parrot] virtual machine
This is the virtual machine that's being developed to support Perl 6
and other languages.

Availability
Parrot is currently in development. The Cardinal project has it's
homepage at [RubyForge]. However it has been dormant since August
2004.

In [languages/ruby] of the Parrot SVN repository there is an earlier
attempt at targetting Ruby on Parrot. The last real update of that was
in August 2002.

Advantages
   1. Access to large Perl development community and resources
   2. Will be able to use modules developed in and/or for Perl and
other Parrot-based languages
   3. The Ruby interpreter could be written in Ruby
   4. Native threads
   5. Precompiled bytecode
   6. JIT
   7. Possible speed improvement (but note that Parrot and Cardinal
are still in development)

Disadvantages
   1. Different C interface (But Parrot's C interface is also clean)
   2. Relies on Parrot, a large work-in-progress

Notes
   1. Not an "official" VM; when asked about VM Matz himself answered
he prefers to develop his own VM

···

On 5/22/06, Peter Hickman <peter@semantico.com> wrote:

One question I have to ask is "Wouldn't it make more sense to piggy back
on parrot (the Perl v6) engine?" and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Is there a reason why we are reinventing the wheel?

10

Unfortunately, not all of us are in your boat. I really like Ruby, but
if performance improvement is not going to be in Ruby's near future,
my management is not going to buy into Ruby!

In most of the corporate environments I have worked in, many managers
tend to take IT decisions based on 'feelings'; 'speed' is a word that
creates good 'feelings'. So, when the issue of performance is bought
up, Ruby doesn't win (doesn't matter if the application requires it or
not).

IMO, if Ruby's vision is to penetrate the corporate environment, Ruby
should address its performance issues among other things (e.g. support
for apps -- Ruby on Rails).

-Madan.

···

On 5/22/06, benjohn@fysh.org <benjohn@fysh.org> wrote:

I just want to point out why it doesn't (pragmatically) matter to me
personally atm.

11

Pretty much the same for me. If Ruby is not fast enough then I convert it to C, nothing less than C seems to have the performance if performance is an issue (ok in extreme cases there is assembler but nothing I have ever written *needed* to be written in assembler - besides it's not as much fun since the m68k went south). The pain of going from Ruby to Java is not going to realise the performance improvement of going from Ruby to C for, what appears to me to be, not quite as much pain.

12

*snip*

So what is being done to get YARV the attention it deserves? Why
aren't we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby's
poor computational performance?

There vertainly are perfectly good "excuses" (I didn't know I needed
one! :slight_smile: ) for not focussing on Ruby's performance - it's performance is
only a concern for one (poor or otherwise) if it's actually causing one
a problem.

In my previous incarnation as a signal processing and volume rendering
coder, I imagine Ruby's performance would have been unnaceptable. In my
current role as a testing framework programmer, it's plenty fast thanks,
and jolly expressive. I've no reason (at the moment) to require more
performance from it. When I switch back to doing graphics again, I'll be
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it'll probably be fast enough for that too.

Not to single you out (though I guess I am) but you are the first one in the thread to write-in with "it's good enough for me because of <insert reasons having nothing to do with computation speed here>."

I think we all know this. Most of us on this list are here because we find the language expressive, clean, fast enough for whatever, yada yada yada.

This thread is *specifically* about getting more computational performance for Ruby. Let's not throw it off track by reiterating all the *other* reasons the language is "good enough" for us.

All that said - I'll crack open a drink to Ruby's performance improving.

Good! Me too!

I just want to point out why it doesn't (pragmatically) matter to me
personally atm.

Oops! Wrong thread! :slight_smile:

cr

···

On May 22, 2006, at 10:07 AM, benjohn@fysh.org wrote:

13

What I found most striking about both interviews was their mention of
YARV and Ruby's poor computational performance. Zed said, and I quote:

> I'll be honest right away though and say that Ruby is slow. The
> Ruby community has been ignoring the huge "performance" elephant
> standing in the room and they need to start talking about it so it
> goes away. Elephants hate being talked about. There are a few
> efforts to make Ruby faster, but I see a lot less action than is
> needed to solve the problem. One solution in the works is a real
> virtual machine called Rite (or YARV depending on who you talk to)
> which is showing some real promise and seems to be speed
> competitive with the fastest Java implementations.

I think you're quoting selectively here. There are some counter points in
both interviews.

Of course I engaged in selective quoting. That's 90% of the fun! :slight_smile:

So what is being done to get YARV the attention it deserves? Why
aren't we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby's
poor computational performance?

Are you on the Ruby Core mailing list? Are you posting bugs and
feature requests to the trackers? Have you downloaded and built
YARV so that you can test it on your system? All of these are ways
you can get involved and help out.

I am on ruby-core but mostly just so I can bask in the glow of the smart people on that list. Plus, I *have* submitted at least one bug report to that least (only to have it turn out to be clumsy handling if IPv4 and IPv6 on OSX).

I have not yet downloaded and built YARV, but I will in the very near future. I am not a great programmer, but as a result I do find neat little test cases that can try the patience of even the most liberal parser & compiler!

cr

···

On May 22, 2006, at 11:59 AM, pat eyler wrote:

On 5/22/06, cremes.devlist@mac.com <cremes.devlist@mac.com> wrote:

14

One question I have to ask is "Wouldn't it make more sense to piggy back
on parrot (the Perl v6) engine?" and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Are there? Parrot has always seemed like a dead Parrot, to me, with
far too much trying to be done in too poorly a way.

Is there a reason why we are reinventing the wheel?

Yes. Ask ko1 and matz why for specifics, but essentially Ruby is more
dynamic than other VMs expect it to be.

-austin

···

On 5/22/06, Peter Hickman <peter@semantico.com> wrote:
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

15

Peter Hickman wrote:

One question I have to ask is "Wouldn't it make more sense to piggy back on parrot (the Perl v6) engine?" and get all the benefit of the performance improvements that parrot will pick up and the fact that there are many, many very clever people working on parrot whose work we could make use of.

Is there a reason why we are reinventing the wheel?

There is indeed a project to piggyback on Parrot (called Cardinal),
probably a little less mature than Parrot itself, which is saying
something.

The wheel is being reinvented because, while Ruby can be written
for Parrot, Parrot isn't written (entirely) for Ruby.

We want a nice, clean impedance match between the language and
the bytecode compiler. Also complete control over its specification
is handy.

Finally, I wouldn't be surprised if YARV ends up outracing Parrot
(both in maturity and in running speed). But we'll see.

Hal

16

Francis Cianfrocca wrote:

I've always had a sense that some of what makes Ruby so beautiful is
_precisely_ what makes it slow. Some of this is a matter of
implementation (if anyone remembers the early implementations of ML,
you know what good tail-recursion can do for you. Plus Ruby seems to
get super-linearly worse as its working set grows), but some of it is
fundamental. Metaprogramming is very hard to do without depending on a
lot of string comparing at runtime. I've got a language design
background (I made a lot of money writing compilers for scripting
languages), and I've had to deal with this problem in other languages.
YARV looks like beautiful work, but I have a gut-feeling that it won't
solve the "speed" problem all by itself.

Maybe you're the one to ask then. With all the talk of VMs, everyone seems to make the comparison to Java and Python. Surely Lisp is a better comparison? Lisps can be stupid fast (as mentioned in another response to this thread), so what does Lisp do right that Ruby doesn't?

···

--
Alex

17

Ruby is based on gcc, gcc has grown some nifty ways of speeding things
up.

No. Ruby is *not* based on gcc. Ruby is based on ANSI C. Any
GCC-specific performanced boosts would reduce Ruby's portability.

Currently Ruby is compiled -O2 usually for generic i386

Obvious answers
  - Compile -O3, gcc, especially since version 4.0 has grown some really nifty
    optimizations. Almost the whole point of version gcc 4

Be careful that those optimizations don't omit stack frames. You'll
get an unusable Ruby.

Ruby is _not_ gcc warning clean. Some gcc warnings are actually performance related.

Ruby should try to be clean on all compilers that don't try to give
you warnings about compiler-specific things (like Visual Studio's new
warnings about deprecated functions like printf).

All that said, the best thing one can do to improve the performance of
one's Ruby script is to have hard numbers about where your script is
performing poorly and being absolutely *certain* that it's Ruby and
not your algorithm. (That is, I know exactly why PDF::Writer is as
dramatically slow as it is; I just don't have a fix for it, yet.)

-austin

···

On 5/22/06, John Carter <john.carter@tait.co.nz> wrote:
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

18

John Carter <john.carter@tait.co.nz> writes:

Hokay some concrete suggestions on speeding up the here and now....

A Good Thing would be to have a Performance test suite. You can't
optimize unless you have a realistic measure of performance.

That would be a Good Thing for someone to create.

Ruby is based on gcc, gcc has grown some nifty ways of speeding things
up.

Ruby is in no way based on GCC.

Much of ruby was written _before_ gcc grew these features but it is
really worth considering now.

That would be a really horrible idea. I get a nearly 10% faster[1]
Ruby by compiling with something other than GCC.

A community "lets all clean up get Ruby -Wall -W clean" would be A Good Thing.

When writing C and compiling with GCC, I use -ansi -Wall -Werror. I
never bother to test code that emits warnings. When building large
software distributions, most warnings come from platform issues and
probably don't appear on the core developers' platforms. Matz applies
patches for these as they are brought to his attention.

The register / cache / memory hierarchy speed differences are becoming
really really huge in modern machines.

Some thought as to keeping things "in cache" especially in things with
tight inner loops like Ruby can make a Big difference.

Thus a Good Thing to try would be to feed Ruby to cachegrind (a tool in
the valgrind.org suite)

It would also be a shame to depend overly on single-platform tools
that "optimize" for a single platform.

Steve

[1] If there is conspiracy, it's the enormous pushback that people who
might otherwise be helpful are getting from Austin et. al. Any
readers who are members of the performance hit squad should kindly
replace the word "faster" with "happier" and move along.

···

John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong later."

From this principle, all of life and physics may be deduced.

19

...

There are big problems with this approach. None of gcc's optimizations
beyond -O2 come for free. They usually enlarge the binary, make
debugging really hard, impact thread-safeness or even correct
functionality (omit frame pointer).

The reason those switches are not standard is because they are not
beneficial in all cases, and sometimes decidedly worse than -O2. Since
especially distributors don't know the exact circumstances ruby will
be deployed in, it is better for them to play safe.

Reworking ruby for better alignment of memory accesses might be
useful. After determining how much of a problem it is though.

Jürgen

···

On Tue, May 23, 2006 at 10:18:10AM +0900, John Carter wrote:

Simple way number one.

Currently Ruby is compiled -O2 usually for generic i386

Obvious answers
- Compile -O3, gcc, especially since version 4.0 has grown some really
nifty
   optimizations. Almost the whole point of version gcc 4

- Compile -march=pentium4, or -march=k7 or whatever the machine it's
   actually running on is. This gives you...
   - Tuned for the pipeline of that CPU.
   - Uses the new fancy instructions for that CPU.

- gcc has / is growing profile based branch prediction. This is something
   ideal for an interpreted language with a largish test suite!

Not quite so easy answers...

Use gprof to get some profile data.

* Use inline functions. C++ and STL have forced all compilers today to
  be very very Good with inline functions. In several respects inlines are
  better than macros.

* const and pure attributes can speed things up too.

--
The box said it requires Windows 95 or better so I installed Linux

20

Agreed, but hasn't this dead horse been beaten enough?

···

On Mon, Jun 23, 2008 at 7:09 AM, Roger Pack <rogerpack2005@gmail.com> wrote:

Yeah I have come to the sad conclusion that, at least with the 1.8.x
world, if speedy execution is your goal you should probably choose
another scripting language [python+psycho comes to mind]. Especially
when considered in conjunction with rails. There I have mentioned the
elephant :slight_smile:
That being said, thankfully 95% of web pages and scripts don't really
care about getting hammered, since they never will be. But the other 5%
will suffer until this gets figured out. And I'm not volunteering.
Hopefully this will improve in the near future. Until then back to my
coding of some poorly performing, elegantly written web pages.
-R
--
Posted via http://www.ruby-forum.com/\.