# Python/ruby benchmark

**URL:** <https://rubytalk.org/t/python-ruby-benchmark/18873>\
**Category:** ruby-talk\
**Created:** [9 June 2005 15:16 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873 "2005-06-09T15:16:24Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![\_script](https://avatars.discourse-cdn.com/v4/letter/_/4da419/32.png) [@\_script](https://rubytalk.org/u/_script)\
**Post date:** [9 June 2005 15:16 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/1 "2005-06-09T15:16:24Z")

</div>

I took a look at  
[http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
and found the tests on ruby disapointing :(.  
So what I'm asking for is a link to some ruby-specific optimisation tips.

> **···**
>
> --  
> Regards, Groleo!
> 
> # touch universe  
> # chmod +x universe  
> # ./universe

---

<div class="post-metadata">

**Author:** ![gabriele\_renzi5](https://avatars.discourse-cdn.com/v4/letter/g/e79b87/32.png) [@gabriele\_renzi5](https://rubytalk.org/u/gabriele_renzi5)\
**Post date:** [9 June 2005 15:30 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/2 "2005-06-09T15:30:28Z")

</div>

"\</script\> ha scritto:

> I took a look at  
> [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
> and found the tests on ruby disapointing :(.  
> So what I'm asking for is a link to some ruby-specific optimisation tips.

a simple thing: do the most normal thing instead of writing code in ruby that is innatural, i.e. instead of rewritine an heapsort by hand, use the builtin sort/sort!/sort\_by routines. Instead of writing a random function by hand, use the builtin rand().

If you really feel ruby (or python) is slow you can act later optimizing it, not doing it before you ever need to be faster. "premature optimization is the root of all evil", people say.

---

<div class="post-metadata">

**Author:** ![Glenn\_Parker1](https://avatars.discourse-cdn.com/v4/letter/g/ba8739/32.png) [@Glenn\_Parker1](https://rubytalk.org/u/Glenn_Parker1)\
**Post date:** [9 June 2005 15:42 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/3 "2005-06-09T15:42:45Z")

</div>

> So what I'm asking for is a link to some ruby-specific optimisation tips.

Avoid creating new objects, e.g. re-use strings and arrays, or modify them in place when possible, especially large ones.

Use built-in iterators as much as possible.

Pray for YARV! 🙂

> **···**
>
> --  
> Glenn Parker | glenn.parker-AT-comcast.net | \<[http://www.tetrafoil.com/&gt](http://www.tetrafoil.com/&gt);

---

<div class="post-metadata">

**Author:** ![Vincent\_Foley](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@Vincent\_Foley](https://rubytalk.org/u/Vincent_Foley)\
**Post date:** [9 June 2005 16:00 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/4 "2005-06-09T16:00:28Z")

</div>

As Gabriele mentionned, they implement a lot of stuff that is done in C  
in Ruby. Also, the tests must use the exact same algorithm, so this  
disadvantages Ruby is some situations. I say the test should compare  
same algorithms, but also the one best suited to the language.

---

<div class="post-metadata">

**Author:** ![Alex\_the\_mindstorm\_P](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Alex\_the\_mindstorm\_P](https://rubytalk.org/u/Alex_the_mindstorm_P)\
**Post date:** [11 June 2005 13:07 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/5 "2005-06-11T13:07:56Z")

</div>

I've run through this thread and i am finally writting this because i am in a way astonished.

Afaik Python uses bytecodes while Ruby is completely interpretative, so imo there is not possible to  
drive conclusions from this benchmark.

A lot of \_comparisons\_ are made in this list (I see everywhere Ruby code has fewer lines than X,  
Ruby dev is better than Y, etc); finally you may expect these other kinds of comparisons to take  
place. As far as the 1st comparison was made, I see no benefit from hidding behind the finger and  
disregarding from the start the possibility to look at other types of comparisons.  
I see here (n.b. on the ML) lots of ideas and open minds while speaking of ruby (\_and just ruby\_)  
and I appreciate this. But everytime, somebody/something is bringing into attention another  
tool/language something bad/sad happens and all this openminded is vanishing away.

While looking at Python and Ruby as both being oo scripting languages, I agree I would like to  
hear/see/read comparisons: from all possible points of view. And imo one of the most important is  
the runtime performance. Maybe the development is easier and cheaper, but if the performance is bad  
i will not do it. Also, even if I have to write fewer lines in X, but the tools for Y are offering  
much more I will certanly go for Y (disclaimer: I am not speaking from the pov of writting a command  
line parser utility or some other little toy that parse the log of my server, but from the pov of  
developing a medium/large application).

A few months ago - not very many - I have started looking at Ruby. And I am enjoying its ideas a  
lot. But, after some time, I am still not sure I will start working with it. I know, you may say:  
`maybe Ruby is not for you´ (and maybe, I can agree with this). I must sadly confess that I haven't  
got the same impression while looking at Python (and from the sizes of communities I can deduce many  
others haven't felt the same).

Finally, please consider the above not a violent complain, but a sad objection I'm having.

:alex |.::the\_mindstorm::.|

---

<div class="post-metadata">

**Author:** ![Ryan\_Leavengood2](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@Ryan\_Leavengood2](https://rubytalk.org/u/Ryan_Leavengood2)\
**Post date:** [9 June 2005 15:54 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/6 "2005-06-09T15:54:51Z")

</div>

gabriele renzi said:

> If you really feel ruby (or python) is slow you can act later optimizing  
> it, not doing it before you ever need to be faster. "premature  
> optimization is the root of all evil", people say.

I heartily agree with this, and it doesn't just apply to Ruby and Python,  
but every language. I used to find myself wasting a lot of time trying to  
come up with some clever "fast" solution, when the reality is in almost  
every conceivable case the "dumb" solution does the job very fast on  
modern machines.

But Python does seem to be faster in most cases, and this is because the  
Python interpreter uses bytecode while the current Ruby interpreter just  
interprets the AST tree after parsing. But there is currently a lot of  
impressive work going into a Ruby bytecode interpreter, called YARV:  
[http://www.atdot.net/yarv/](http://www.atdot.net/yarv/)

You can expect the Ruby interpreter that uses YARV to be 3 or 4 times  
faster (I've compiled it and ran the benchmarks.)

Ryan

---

<div class="post-metadata">

**Author:** ![Lothar\_Scholz2](https://avatars.discourse-cdn.com/v4/letter/l/f14d63/32.png) [@Lothar\_Scholz2](https://rubytalk.org/u/Lothar_Scholz2)\
**Post date:** [9 June 2005 16:23 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/7 "2005-06-09T16:23:26Z")

</div>

Hello gabriele,

> "\</script\> ha scritto:
> 
> > I took a look at  
> > [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
> > and found the tests on ruby disapointing :(.  
> > So what I'm asking for is a link to some ruby-specific optimisation tips.

> a simple thing: do the most normal thing instead of writing code in ruby  
> that is innatural, i.e. instead of rewritine an heapsort by hand, use  
> the builtin sort/sort!/sort\_by routines. Instead of writing a random  
> function by hand, use the builtin rand().

Sorry you don't understand this benchmark. It is there to do this  
algorithms to compare the basic data operations.

The benchmark is not useable to compare compiled languages with  
interpretered, but comparing python with ruby is a valid comparision.

And that ruby is so worse then python is a real problem. If you still  
try to deny this, then you have a real mental problem.

If ruby is 9 times slower then python in for example function calling intensive  
code that means that ruby will slow down your development time a lot  
as it forces you to use optimized code or even C code much more often  
then python.

> **···**
>
> --  
> Best regards, emailto: scholz at scriptolutions dot com  
> Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com)  
> CTO Scriptolutions Ruby, PHP, Python IDE 's

---

<div class="post-metadata">

**Author:** ![Isaac\_Gouy1](https://avatars.discourse-cdn.com/v4/letter/i/4bbf92/32.png) [@Isaac\_Gouy1](https://rubytalk.org/u/Isaac_Gouy1)\
**Post date:** [10 June 2005 17:15 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/8 "2005-06-10T17:15:28Z")

</div>

Vincent Foley wrote:

> As Gabriele mentionned, they implement a lot of stuff that is done in C  
> in Ruby.

Specifically?

> Also, the tests must use the exact same algorithm, so this  
> disadvantages Ruby is some situations. I say the test should compare  
> same algorithms, but also the one best suited to the language.

Do this disadvantage Ruby more than PHP?  
[http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=php&sort=fullcpu](http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=php&sort=fullcpu)

Or more than Tcl, or more than GNU Smalltalk?  
[http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=gst&sort=fullcpu](http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=gst&sort=fullcpu)

---

<div class="post-metadata">

**Author:** ![Austin\_Ziegler5](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/austin_ziegler5/32/1985_2.png) [@Austin\_Ziegler5](https://rubytalk.org/u/Austin_Ziegler5)\
**Post date:** [11 June 2005 13:48 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/9 "2005-06-11T13:48:09Z")

</div>

> I've run through this thread and i am finally writting this  
> because i am in a way astonished.
> 
> Afaik Python uses bytecodes while Ruby is completely  
> interpretative, so imo there is not possible to drive conclusions  
> from this benchmark.

Which is one of the many reasons that I have a problem with the  
alioth shootout. Ultimately, it tries to treat compiled, semi-  
compiled, and interpreted languages equally. They're not. Compiled  
languages will be faster than semi-compiled (bytecode), which will  
be faster than interpreted. The level of speed difference is a  
matter of scale, and that scale also matters on development time and  
clarity.

> A lot of \_comparisons\_ are made in this list (I see everywhere  
> Ruby code has fewer lines than X, Ruby dev is better than Y, etc);  
> finally you may expect these other kinds of comparisons to take  
> place. As far as the 1st comparison was made, I see no benefit  
> from hidding behind the finger and disregarding from the start the  
> possibility to look at other types of comparisons. I see here  
> (n.b. on the ML) lots of ideas and open minds while speaking of  
> ruby (\_and just ruby\_) and I appreciate this. But everytime,  
> somebody/something is bringing into attention another  
> tool/language something bad/sad happens and all this openminded is  
> vanishing away.

Sorry, but that's just not true. People \*are\* open-minded on this  
list with respect to new ideas. On the other hand, many of us don't  
want to import those ideas which we feel will change the fundamental  
nature of Ruby. Most of us don't \*quite\* agree on what that  
fundamental nature is, but hey, that's the nature of individuals 😉

No one has said that Ruby can't afford to be faster, or that other  
languages ARE faster. What we've said is that, in most cases, Ruby  
is Fast Enough...and that it gives us pleasure to work with it and  
we feel that we have a better grasp on our problem domain and that  
we have a more powerful solution for less development time. All of  
these items work together. You know what? Python developers  
generally feel the same about their programs.

> While looking at Python and Ruby as both being oo scripting  
> languages, I agree I would like to hear/see/read comparisons: from  
> all possible points of view. And imo one of the most important is  
> the runtime performance. Maybe the development is easier and  
> cheaper, but if the performance is bad i will not do it.

See, the problem is that you can't really know what your performance  
will be like until you've solved at least part of your problem  
domain. The use of benchmarks -- especially brainless ones like  
you'll find at the alioth shootout -- won't help you. Unless, of  
course, your problem domain is writing benchmarks.

Look, there are people doing OpenGL work with Ruby. It's obviously  
fast enough for \*them\*. ARPA uses Ruby to drive a large-scale Java  
application. Rich Kilmer is working on making Ruby drive a Flash  
application (what's the status on that, Rich?). Ara Howard uses Ruby  
to do NOAA work on large weather pictures and to drive a cluster.  
These are all things that require significant computing power. These  
people have chosen to use Ruby. Why? Because it gives them pleasure,  
and Ruby is Fast Enough in real world tests.

Will they complain if Ruby gets faster? Not at all. They'll be  
ecstatic. But they're not complaining about Ruby's speed right now.

> Also, even if I have to write fewer lines in X, but the tools for  
> Y are offering much more I will certanly go for Y (disclaimer: I  
> am not speaking from the pov of writting a command line parser  
> utility or some other little toy that parse the log of my server,  
> but from the pov of developing a medium/large application).

IMO, this is a little short-sighted way to look at it. Sure,  
VisualStudio gives you great tools to develop .NET applications.  
\*Damn\* but does it ever give you great tools. (I've seen a preview  
of what's coming in November. Damn, I say, Damn, that's nice.) But  
you have to do a lot more with the languages behind VisualStudio to  
get things done. In most cases, the Ruby program will be done or  
deployable long before the .NET program (and \*way\* before a J2EE  
program). Real world results will be measured. Real optimisations  
can be performed.

> A few months ago - not very many - I have started looking at Ruby.  
> And I am enjoying its ideas a lot. But, after some time, I am  
> still not sure I will start working with it. I know, you may say:  
> `maybe Ruby is not for you´ (and maybe, I can agree with this). I  
> must sadly confess that I haven't got the same impression while  
> looking at Python (and from the sizes of communities I can deduce  
> many others haven't felt the same).

You know, I'm not sure whether Ruby is for you or not. Have you done  
development in it? Has it shown itself to be Too Slow for your  
problem domain(s)? Does it matter if Ruby finished the job one  
second slower than Python? Five seconds? Five minutes? Ultimately,  
it depends on the job as to how that can be answered.

Just as another note, I think that the RAA is pretty active. It's  
run on WEBrick through an Apache proxy. Wow, huh? WEBrick is a  
pretty performant web server written in Ruby.

-austin

> **···**
>
> On 6/11/05, Alexandru Popescu \<the\_mindstorm@evolva.ro\> wrote:  
> --  
> Austin Ziegler \* halostatue@gmail.com  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\* Alternate: austin@halostatue.ca

---

<div class="post-metadata">

**Author:** ![Kent\_Sibilev1](https://avatars.discourse-cdn.com/v4/letter/k/9d8465/32.png) [@Kent\_Sibilev1](https://rubytalk.org/u/Kent_Sibilev1)\
**Post date:** [9 June 2005 16:41 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/10 "2005-06-09T16:41:26Z")

</div>

Java is an order of magnitude faster than Ruby. The development of a  
web app using Rails is an order of magnitude faster than the  
development of a similar web app using Struts/Hibernate frameworks.  
There are other factors that can slow down your development time  
besides the speed of your language of choice.

Cheers,  
Kent  
Just another guy with huge mental problems.

> **···**
>
> On 6/9/05, Lothar Scholz \<mailinglists@scriptolutions.com\> wrote:
> 
> > Hello gabriele,
> > 
> > \> "\</script\> ha scritto:  
> > \>\> I took a look at  
> > \>\> [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
> > \>\> and found the tests on ruby disapointing :(.  
> > \>\> So what I'm asking for is a link to some ruby-specific optimisation tips.
> > 
> > \> a simple thing: do the most normal thing instead of writing code in ruby  
> > \> that is innatural, i.e. instead of rewritine an heapsort by hand, use  
> > \> the builtin sort/sort!/sort\_by routines. Instead of writing a random  
> > \> function by hand, use the builtin rand().
> > 
> > Sorry you don't understand this benchmark. It is there to do this  
> > algorithms to compare the basic data operations.
> > 
> > The benchmark is not useable to compare compiled languages with  
> > interpretered, but comparing python with ruby is a valid comparision.
> > 
> > And that ruby is so worse then python is a real problem. If you still  
> > try to deny this, then you have a real mental problem.
> > 
> > If ruby is 9 times slower then python in for example function calling intensive  
> > code that means that ruby will slow down your development time a lot  
> > as it forces you to use optimized code or even C code much more often  
> > then python.
> > 
> > --  
> > Best regards, emailto: scholz at scriptolutions dot com  
> > Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com)  
> > CTO Scriptolutions Ruby, PHP, Python IDE 's

---

<div class="post-metadata">

**Author:** ![gabriele\_renzi5](https://avatars.discourse-cdn.com/v4/letter/g/e79b87/32.png) [@gabriele\_renzi5](https://rubytalk.org/u/gabriele_renzi5)\
**Post date:** [10 June 2005 00:35 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/11 "2005-06-10T00:35:28Z")

</div>

Lothar Scholz ha scritto:

> Hello gabriele,
> 
> \> "\</script\> ha scritto:
> 
> > > I took a look at  
> > > [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
> > > and found the tests on ruby disapointing :(.  
> > > So what I'm asking for is a link to some ruby-specific optimisation tips.
> 
> \> a simple thing: do the most normal thing instead of writing code in ruby  
> \> that is innatural, i.e. instead of rewritine an heapsort by hand, use  
> \> the builtin sort/sort!/sort\_by routines. Instead of writing a random  
> \> function by hand, use the builtin rand().
> 
> Sorry you don't understand this benchmark. It is there to do this  
> algorithms to compare the basic data operations.

I understand the benchmark.  
I'm just saying that it shows "you should not implement X in ruby" not that "X takes a lot of time in ruby" since you may not need to implement X (i.e. because it is already there or because you would not actually ever need a recursive floating point function).

> The benchmark is not useable to compare compiled languages with  
> interpretered, but comparing python with ruby is a valid comparision.

\>

> And that ruby is so worse then python is a real problem. If you still  
> try to deny this, then you have a real mental problem.

I don't think I did

> If ruby is 9 times slower then python in for example function calling intensive  
> code that means that ruby will slow down your development time a lot  
> as it forces you to use optimized code or even C code much more often  
> then python.

I agree, It would be nice if ruby was faster. Luckily,  
people are working on this.

---

<div class="post-metadata">

**Author:** ![Austin\_Ziegler5](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/austin_ziegler5/32/1985_2.png) [@Austin\_Ziegler5](https://rubytalk.org/u/Austin_Ziegler5)\
**Post date:** [10 June 2005 18:02 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/12 "2005-06-10T18:02:48Z")

</div>

When the algorithm, as entered, won't complete, yes. When compared  
against a language that has tail recursion optimisation, or a possible  
situation where optimisation of the compiler deals with recursion  
intelligently? Yes.

The definition of Ackermann numbers is, I presume, well known. Because  
some platforms can't modify their stack level (Windows), and because  
the stack level isn't modified in the runs for the benchmark, there's  
going to be an issue with the deep recursion. In this case, it would  
be appropriate to reduce the level of recursion. But given the rules  
mentioned, this isn't acceptable.

What's the \*purpose\* of something like the Ackermann test? Is it to  
see how quickly the environment winds and unwinds the stack? How much  
state it carries around?

I personally think that most of the shootout items are bogus in just  
about every way. Don't get me wrong -- I'm not at all suggesting that  
Ruby couldn't be faster. I'm just saying that benchmarks, like  
statistics, are lies.

-austin

> **···**
>
> On 6/10/05, Isaac Gouy \<igouy@yahoo.com\> wrote:
> 
> > \> Also, the tests must use the exact same algorithm, so this  
> > \> disadvantages Ruby is some situations. I say the test should compare  
> > \> same algorithms, but also the one best suited to the language.  
> > Do this disadvantage Ruby more than PHP?  
> > Or more than Tcl, or more than GNU Smalltalk?
> 
> --  
> Austin Ziegler \* halostatue@gmail.com  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\* Alternate: austin@halostatue.ca

---

<div class="post-metadata">

**Author:** ![Isaac\_Gouy1](https://avatars.discourse-cdn.com/v4/letter/i/4bbf92/32.png) [@Isaac\_Gouy1](https://rubytalk.org/u/Isaac_Gouy1)\
**Post date:** [11 June 2005 16:10 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/13 "2005-06-11T16:10:29Z")

</div>

Austin Ziegler wrote:

> **···**
>
> > On 6/11/05, Alexandru Popescu \<the\_mindstorm@evolva.ro\> wrote:  
> > \> I've run through this thread and i am finally writting this  
> > \> because i am in a way astonished.  
> > \>  
> > \> Afaik Python uses bytecodes while Ruby is completely  
> > \> interpretative, so imo there is not possible to drive conclusions  
> > \> from this benchmark.
> > 
> > Which is one of the many reasons that I have a problem with the  
> > alioth shootout. Ultimately, it tries to treat compiled, semi-  
> > compiled, and interpreted languages equally. They're not. Compiled  
> > languages will be faster than semi-compiled (bytecode), which will  
> > be faster than interpreted.
> 
> Saying "Compiled languages will be faster than..." treats them all  
> equally!
> 
> There are compiled, bytecode, interpreter, ... /implementations/ of  
> programming languages - there are interpreters for C as-well-as  
> compilers.

---

<div class="post-metadata">

**Author:** ![Ptkwt](https://avatars.discourse-cdn.com/v4/letter/p/f14d63/32.png) [@Ptkwt](https://rubytalk.org/u/Ptkwt)\
**Post date:** [12 June 2005 07:20 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/14 "2005-06-12T07:20:31Z")

</div>

In article \<9e7db91105061106485b68d629@mail.gmail.com\>,

> > I've run through this thread and i am finally writting this  
> > because i am in a way astonished.  
> > =20  
> > Afaik Python uses bytecodes while Ruby is completely  
> > interpretative, so imo there is not possible to drive conclusions  
> > from this benchmark.
> 
> Which is one of the many reasons that I have a problem with the  
> alioth shootout. Ultimately, it tries to treat compiled, semi-  
> compiled, and interpreted languages equally. They're not. Compiled  
> languages will be faster than semi-compiled (bytecode), which will  
> be faster than interpreted. The level of speed difference is a  
> matter of scale, and that scale also matters on development time and  
> clarity.

I have to say that I tend to agree with the astonishment.

Here's what I mean:  
Someone comes along and asks about python vs. Ruby performance and  
references the alioth benchmarks. Several folks jump on the alioth  
benchmarks and call them bogus. I see this as blaming the messanger.

The fact is that these benchmarks exist and they cover lots of different  
types of algorithms. They're on the web and people will look at them -  
no way to stop that from happening.

Someone mentioned the Ackermann benchmark so I had a look and found that  
gawk did much better than Ruby - that's pathetic folks. Now, I know that  
if you could do this in gawk it should be possible to do it in Ruby as well.  
There aren't even mumbers for N=8 or N=9 for the Ruby version. Maybe it's  
because the Ruby version was written poorly. Maybe it's because of some  
issue within the Ruby interpreter itself - I don't know, I didn't really  
take time to look into it just now. Either way, wouldn't it be valuable  
to try to improve our score on that benchmark? If it's because it's  
poorly written, then it's gravy. If it's an issue with the interpreter  
then maybe it would be good to know about the issue for the YARV design.

Now sure you can say that the benchmarks are bogus (and who knows, maybe  
they are) and then take all your marbles and go home, but the benchmarks  
remain and other languages communities are trying to hone their numbers.  
If we totally sit it out, well, that's not going to look too good.

Sure these benchmarks don't tell the whole story about how nice it is to  
develop in Ruby vs. gawk, but performance \_is\_ an issue for some people  
and they will use this set of benchmarks (or another) as a factor in  
deciding which langauge to use. If Ruby consistently shows up down  
towards the bottom of the list... well, it can give a false impression.

If we tell them that they really shouldn't worry about performance (and  
who knows, maybe we would be right in doing so) it just sounds like so  
much excuse-making.

I think that as a community we should be trying to compete in these  
benchmarks just like other langauge communities are doing. Perhaps the  
results can help us as we do move to a VM - if nothing else to show  
that we're making improvements (look at them as a set of unit tests for  
performance). We know that Ruby needs help in the  
performance area. If we keep telling ourselves that Ruby is 'fast  
enough' for our application (and it may well be) are we going to be  
sitting still while other languages improve performance? Have we been  
the hare for the last few years sitting around while the tortise (TCL,  
for example, once considered a lot slower than Ruby) is passing us up  
(by creating or improving their VMs, for example).

Of course we know that we can do all sorts of things to speed things up  
by writing C extension code, for example - that's a given, and the fact  
that it's so easy to do in Ruby makes it possible to get into a lot of  
areas where you might normally need to consider a compiled langauge,  
however, a lot of people out there want to see how Ruby performs when  
you just write Ruby code (C is a compiled language, of course). In some  
sense marrying Ruby and C might seem like cheating to them, I suppose (try  
to see it from their perspective).

If it looks like some of the benchmarks are misleading or stacked against  
us, maybe we can propose other benchmarks where Ruby shines (BTW: why  
does it say that the Object Methods benchmark is being deprecated?),  
after all, it appears to be an open source process.

Bottom line: Ruby needs help in the performance area. Let's admit that  
and work on improving in that area instead of shooting the messenger -  
it'll make us look like whiners if we keep doing the latter. I'd really  
rather not have the Ruby community perceived that way from the outside.

In the meantime, perhaps we could have some Ruby quizzes that focus on  
choosing a benchmark and optimizing it. I think we would be  
better off as a community if we participate in the process rather than  
boycott it.

....just my 2cents.

Phil

> **···**
>
> Austin Ziegler \<halostatue@gmail.com\> wrote:
> 
> > On 6/11/05, Alexandru Popescu \<the\_mindstorm@evolva.ro\> wrote:

---

<div class="post-metadata">

**Author:** ![Dibya\_Prakash1](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Dibya\_Prakash1](https://rubytalk.org/u/Dibya_Prakash1)\
**Post date:** [9 June 2005 18:03 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/15 "2005-06-09T18:03:12Z")

</div>

Hi,  
I definitely agree.Only feature of a particular programming language  
cannot determine the speed of software.Other factors are also  
crucially important.  
I didnot know about YARV,but gr8 that I came to know it thru u  
people.I'll surely check it out.

Bye  
Dibya Prakash  
Happy coding

> **···**
>
> On 6/9/05, Kent Sibilev \<ksruby@gmail.com\> wrote:
> 
> > Java is an order of magnitude faster than Ruby. The development of a  
> > web app using Rails is an order of magnitude faster than the  
> > development of a similar web app using Struts/Hibernate frameworks.  
> > There are other factors that can slow down your development time  
> > besides the speed of your language of choice.
> > 
> > Cheers,  
> > Kent  
> > Just another guy with huge mental problems.
> > 
> > On 6/9/05, Lothar Scholz \<mailinglists@scriptolutions.com\> wrote:  
> > \> Hello gabriele,  
> > \>  
> > \> \> "\</script\> ha scritto:  
> > \> \>\> I took a look at  
> > \> \>\> [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu](http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu)  
> > \> \>\> and found the tests on ruby disapointing :(.  
> > \> \>\> So what I'm asking for is a link to some ruby-specific optimisation tips.  
> > \>  
> > \> \> a simple thing: do the most normal thing instead of writing code in ruby  
> > \> \> that is innatural, i.e. instead of rewritine an heapsort by hand, use  
> > \> \> the builtin sort/sort!/sort\_by routines. Instead of writing a random  
> > \> \> function by hand, use the builtin rand().  
> > \>  
> > \> Sorry you don't understand this benchmark. It is there to do this  
> > \> algorithms to compare the basic data operations.  
> > \>  
> > \> The benchmark is not useable to compare compiled languages with  
> > \> interpretered, but comparing python with ruby is a valid comparision.  
> > \>  
> > \> And that ruby is so worse then python is a real problem. If you still  
> > \> try to deny this, then you have a real mental problem.  
> > \>  
> > \> If ruby is 9 times slower then python in for example function calling intensive  
> > \> code that means that ruby will slow down your development time a lot  
> > \> as it forces you to use optimized code or even C code much more often  
> > \> then python.  
> > \>  
> > \>  
> > \> --  
> > \> Best regards, emailto: scholz at scriptolutions dot com  
> > \> Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com)  
> > \> CTO Scriptolutions Ruby, PHP, Python IDE 's  
> > \>  
> > \>  
> > \>  
> > \>

---

<div class="post-metadata">

**Author:** ![Lothar\_Scholz2](https://avatars.discourse-cdn.com/v4/letter/l/f14d63/32.png) [@Lothar\_Scholz2](https://rubytalk.org/u/Lothar_Scholz2)\
**Post date:** [9 June 2005 18:06 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/16 "2005-06-09T18:06:32Z")

</div>

Hello Kent,

> Java is an order of magnitude faster than Ruby. The development of a  
> web app using Rails is an order of magnitude faster than the  
> development of a similar web app using Struts/Hibernate frameworks.  
> There are other factors that can slow down your development time  
> besides the speed of your language of choice.

If i can't use the language because it is to slow to solve the problem  
then the development time doesn't matter. There are just too many  
problem domains left where you can't ignore it.

But we all hope and pray for YARV.

> **···**
>
> --  
> Best regards, emailto: scholz at scriptolutions dot com  
> Lothar Scholz [http://www.ruby-ide.com](http://www.ruby-ide.com)  
> CTO Scriptolutions Ruby, PHP, Python IDE 's

---

<div class="post-metadata">

**Author:** ![Jim\_Freeze2](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@Jim\_Freeze2](https://rubytalk.org/u/Jim_Freeze2)\
**Post date:** [10 June 2005 02:39 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/17 "2005-06-10T02:39:14Z")

</div>

\* gabriele renzi \<surrender\_it@remove-yahoo.it\> [2005-06-10 09:35:28 +0900]:

> I agree, It would be nice if ruby was faster. Luckily,  
> people are working on this.

Don't go away thinking that X is faster than Ruby. Do  
your own tests. We tested Perl against Ruby in a real  
world problem (netlist parsing) and Ruby was faster  
than Perl by 30%. When we did a C implementation  
(which could now be done automatically with optimize)  
it was 100% faster (ie, took half the time.)

These statements that X language is faster than  
Ruby is like saying that Einstein is smarter than  
Mozart. In what?

> **···**
>
> --  
> Jim Freeze  
> Theory and practice are the same, in theory. -- Ryan Davis

---

<div class="post-metadata">

**Author:** ![Isaac\_Gouy1](https://avatars.discourse-cdn.com/v4/letter/i/4bbf92/32.png) [@Isaac\_Gouy1](https://rubytalk.org/u/Isaac_Gouy1)\
**Post date:** [10 June 2005 18:50 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/18 "2005-06-10T18:50:29Z")

</div>

Austin Ziegler wrote:

> \> \> Also, the tests must use the exact same algorithm, so this  
> \> \> disadvantages Ruby is some situations. I say the test should compare  
> \> \> same algorithms, but also the one best suited to the language.  
> \> Do this disadvantage Ruby more than PHP?  
> \> Or more than Tcl, or more than GNU Smalltalk?
> 
> When the algorithm, as entered, won't complete, yes. When compared  
> against a language that has tail recursion optimisation, or a possible  
> situation where optimisation of the compiler deals with recursion  
> intelligently? Yes.

Ruby completes ackermann for N=7  
[http://shootout.alioth.debian.org/great/fulldata.php?test=ackermann&p1=ocaml-0&p2=clean-0&p3=gcc-3&p4=gcc-0&sort=fullcpu#cputable](http://shootout.alioth.debian.org/great/fulldata.php?test=ackermann&p1=ocaml-0&p2=clean-0&p3=gcc-3&p4=gcc-0&sort=fullcpu#cputable)

Which is how there can be a comparison between Ruby and Tcl on  
ackermann  
[http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=tcl&sort=fullcpu#ratio](http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=tcl&sort=fullcpu#ratio)

> The definition of Ackermann numbers is, I presume, well known. Because  
> some platforms can't modify their stack level (Windows), and because  
> the stack level isn't modified in the runs for the benchmark, there's  
> going to be an issue with the deep recursion.

We use Debian Linux, so tell us how to modify the stack level for Ruby.

-snip-

> I personally think that most of the shootout items are bogus in just  
> about every way. Don't get me wrong -- I'm not at all suggesting that  
> Ruby couldn't be faster. I'm just saying that benchmarks, like  
> statistics, are lies.

Sounds like me in editorial mode.  
[http://shootout.alioth.debian.org/great/miscfile.php?sort=fullcpu&file=benchmarking&title=Flawed%20Benchmarks](http://shootout.alioth.debian.org/great/miscfile.php?sort=fullcpu&file=benchmarking&title=Flawed%20Benchmarks)

> **···**
>
> > On 6/10/05, Isaac Gouy \<igouy@yahoo.com\> wrote:

---

<div class="post-metadata">

**Author:** ![Trey\_Campbell1](https://avatars.discourse-cdn.com/v4/letter/t/a9a28c/32.png) [@Trey\_Campbell1](https://rubytalk.org/u/Trey_Campbell1)\
**Post date:** [10 June 2005 20:02 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/19 "2005-06-10T20:02:04Z")

</div>

The most succinct way that I've heard it put is that there are "lies, damn lies, and benchmarks".

Trey Campbell  
treycampbell@mac.com

> **···**
>
> On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
> 
> > I personally think that most of the shootout items are bogus in just  
> > about every way. Don't get me wrong -- I'm not at all suggesting that  
> > Ruby couldn't be faster. I'm just saying that benchmarks, like  
> > statistics, are lies.
> > 
> > -austin  
> > -- Austin Ziegler \* halostatue@gmail.com  
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\* Alternate: austin@halostatue.ca

---

<div class="post-metadata">

**Author:** ![Alex\_the\_mindstorm\_P](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Alex\_the\_mindstorm\_P](https://rubytalk.org/u/Alex_the_mindstorm_P)\
**Post date:** [12 June 2005 09:34 UTC](https://rubytalk.org/t/python-ruby-benchmark/18873/20 "2005-06-12T09:34:07Z")

</div>

#: the mind was \*winged\* after Phil Tomson said on 6/12/2005 9:20 AM :#

> In article \<9e7db91105061106485b68d629@mail.gmail.com\>,
> 
> > > I've run through this thread and i am finally writting this  
> > > because i am in a way astonished.  
> > > =20  
> > > Afaik Python uses bytecodes while Ruby is completely  
> > > interpretative, so imo there is not possible to drive conclusions  
> > > from this benchmark.
> > 
> > Which is one of the many reasons that I have a problem with the  
> > alioth shootout. Ultimately, it tries to treat compiled, semi-  
> > compiled, and interpreted languages equally. They're not. Compiled  
> > languages will be faster than semi-compiled (bytecode), which will  
> > be faster than interpreted. The level of speed difference is a  
> > matter of scale, and that scale also matters on development time and  
> > clarity.
> 
> I have to say that I tend to agree with the astonishment.
> 
> Here's what I mean:  
> Someone comes along and asks about python vs. Ruby performance and  
> references the alioth benchmarks. Several folks jump on the alioth  
> benchmarks and call them bogus. I see this as blaming the messanger.
> 
> The fact is that these benchmarks exist and they cover lots of different  
> types of algorithms. They're on the web and people will look at them -  
> no way to stop that from happening.
> 
> Someone mentioned the Ackermann benchmark so I had a look and found that  
> gawk did much better than Ruby - that's pathetic folks. Now, I know that  
> if you could do this in gawk it should be possible to do it in Ruby as well.  
> There aren't even mumbers for N=8 or N=9 for the Ruby version. Maybe it's  
> because the Ruby version was written poorly. Maybe it's because of some  
> issue within the Ruby interpreter itself - I don't know, I didn't really  
> take time to look into it just now. Either way, wouldn't it be valuable  
> to try to improve our score on that benchmark? If it's because it's  
> poorly written, then it's gravy. If it's an issue with the interpreter  
> then maybe it would be good to know about the issue for the YARV design.
> 
> Now sure you can say that the benchmarks are bogus (and who knows, maybe  
> they are) and then take all your marbles and go home, but the benchmarks  
> remain and other languages communities are trying to hone their numbers.  
> If we totally sit it out, well, that's not going to look too good.
> 
> Sure these benchmarks don't tell the whole story about how nice it is to  
> develop in Ruby vs. gawk, but performance \_is\_ an issue for some people  
> and they will use this set of benchmarks (or another) as a factor in  
> deciding which langauge to use. If Ruby consistently shows up down  
> towards the bottom of the list... well, it can give a false impression.
> 
> If we tell them that they really shouldn't worry about performance (and  
> who knows, maybe we would be right in doing so) it just sounds like so  
> much excuse-making.
> 
> I think that as a community we should be trying to compete in these  
> benchmarks just like other langauge communities are doing. Perhaps the  
> results can help us as we do move to a VM - if nothing else to show  
> that we're making improvements (look at them as a set of unit tests for  
> performance). We know that Ruby needs help in the  
> performance area. If we keep telling ourselves that Ruby is 'fast  
> enough' for our application (and it may well be) are we going to be  
> sitting still while other languages improve performance? Have we been  
> the hare for the last few years sitting around while the tortise (TCL,  
> for example, once considered a lot slower than Ruby) is passing us up  
> (by creating or improving their VMs, for example).
> 
> Of course we know that we can do all sorts of things to speed things up  
> by writing C extension code, for example - that's a given, and the fact  
> that it's so easy to do in Ruby makes it possible to get into a lot of  
> areas where you might normally need to consider a compiled langauge,  
> however, a lot of people out there want to see how Ruby performs when  
> you just write Ruby code (C is a compiled language, of course). In some  
> sense marrying Ruby and C might seem like cheating to them, I suppose (try  
> to see it from their perspective).
> 
> If it looks like some of the benchmarks are misleading or stacked against  
> us, maybe we can propose other benchmarks where Ruby shines (BTW: why  
> does it say that the Object Methods benchmark is being deprecated?),  
> after all, it appears to be an open source process.
> 
> Bottom line: Ruby needs help in the performance area. Let's admit that  
> and work on improving in that area instead of shooting the messenger -  
> it'll make us look like whiners if we keep doing the latter. I'd really  
> rather not have the Ruby community perceived that way from the outside.
> 
> In the meantime, perhaps we could have some Ruby quizzes that focus on  
> choosing a benchmark and optimizing it. I think we would be  
> better off as a community if we participate in the process rather than  
> boycott it.
> 
> ....just my 2cents.
> 
> Phil

Lots of thanks Phil for better expressing my thoughts and feelings. That is exactly what I wanted to  
say - unfortunately the feelings just cut off my vocabulary.

And just as a confirmation of my last paragraph:  
[stupid-quoting-myself]  
I know, you may say:  
`maybe Ruby is not for you´ (and maybe, I can agree with this). I must sadly confess that I haven't  
got the same impression while looking at Python (and from the sizes of communities I can deduce many  
others haven't felt the same).  
[/stupid-quoting-myself]

I have received offline messages suggesting to go with other solutions and some solid arguments why  
I should do it.

I have to agree that it is no pleasure to find your invention `hammered´ by a (possible wrong)  
benchmark; but I think when this happens I would go out shouting my benchmarks where i am kicking ass.  
I remember that Java community did this a lot while the first c++ vs java benchmarks have been  
published.  
I remember that a few years ago when such a benchmark (driven by `paid´ money by MS), was proving  
that .NET was surclassing J2EE, finally even IBM joined the war and proved it completely wrong.

:alex |.::the\_mindstorm::.|

> **···**
>
> > Austin Ziegler \<halostatue@gmail.com\> wrote:
> > 
> > > On 6/11/05, Alexandru Popescu \<the\_mindstorm@evolva.ro\> wrote:

[Next page](https://rubytalk.org/t/python-ruby-benchmark/18873.md?page=2)
