The new great computer language shootout

FYI: The great computer language shootout is now being hosted by one of
the Debian groups. It's been updated to use the latest
compilers/interpreters (Ruby 1.8.1 is in there now).

You can find it here:
http://shootout.alioth.debian.org/index.php

It's interesting that Python has now moved ahead of Perl in performance -
in the older shootout Python was still slower than Perl. Ruby is
somewhere near the middle of the scale ( slower than both Perl and
Python).

Phil

Part of the speedup appears to be from their use of the Psyco JIT
compiler for the Python code -- the links to source code for Python
end in '.psyco'.

I would guess that some of the Ruby code needs to be optimized, as
well; for example, in the Array Access test, Ruby places dead last.
That's rediculous, and I don't think it's indicative of the array
performance for general cases.

Lennon

This is because of the use of Numeric#step. The implementation is
flawed.

···

Lennon Day-Reynolds (rcoder@gmail.com) wrote:

I would guess that some of the Ruby code needs to be optimized, as
well; for example, in the Array Access test, Ruby places dead last.
That's rediculous, and I don't think it's indicative of the array
performance for general cases.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

Never mind, I am smoking crack.

The real problem appears to be iteration in general.

···

Eric Hodel (drbrain@segment7.net) wrote:

This is because of the use of Numeric#step. The implementation is
flawed.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

Eric Hodel <drbrain@segment7.net> wrote in message news:<20040617223524.GE96045@segment7.net>...

> I would guess that some of the Ruby code needs to be optimized, as
> well; for example, in the Array Access test, Ruby places dead last.
> That's rediculous, and I don't think it's indicative of the array
> performance for general cases.

This is because of the use of Numeric#step. The implementation is
flawed.

Huh? In what way is it flawed? While there you can use slicing for
some operations, other array operations need iteration thru the array
as showed.
I do find the shootout is pretty accurate in terms of real ruby
performance overall.
The new web page is pretty bad in that most links are broken due to
typos and what not, so it is not clear to me the python tests are
based on psyco. There does appear to be a separate psyco benchmark.

···

Lennon Day-Reynolds (rcoder@gmail.com) wrote:

To be honest, some of the numbers seem highly suspect to me. For
example, on my P4/Fedora Core 1 box at least, the Sieve of
Eratosthenes example runs 20-30% *faster* in Ruby than it does in
Python, while his dataset shows is taking almost twice as long.

Lennon

surprisingly, using += on line 18 give a 12% speedup on my box.

cheers,
Mark

···

On Jun 17, 2004, at 4:03 PM, Eric Hodel wrote:

Eric Hodel (drbrain@segment7.net) wrote:

This is because of the use of Numeric#step. The implementation is
flawed.

Never mind, I am smoking crack.

The real problem appears to be iteration in general.

Eric Hodel <drbrain@segment7.net> wrote in message
news:<20040617223524.GE96045@segment7.net>...

> > I would guess that some of the Ruby code needs to be optimized, as
> > well; for example, in the Array Access test, Ruby places dead last.
> > That's rediculous, and I don't think it's indicative of the array
> > performance for general cases.
>
> This is because of the use of Numeric#step. The implementation is
> flawed.

Huh? In what way is it flawed? While there you can use slicing for

In what way is it flawed? The whole thing is flawed. It is not fair to judge
the languages like this. Especially when they are compiled with GNU gcc, the
optimizations are really poor. It would be fair if someone used an optimized
compiler to compile ruby and the others then compete. As you can see Java
apears as first on some of the lists. I will state the fact that Java is not
compiled by GCC. They are probably using a optimized comiler. which makes
everything faster. I would like to see someone compiler ruby by Intel's
Compiler ICC. I'm sure Ruby would then be better than java in tests. Java has
optimized code and assembly. theres no possible way much of anything can be
better when they are the ones using optimized comiler and tossing out the
binaries. ---David Ross

···

On Friday 18 June 2004 01:03 am, GGarramuno wrote:

> Lennon Day-Reynolds (rcoder@gmail.com) wrote:

some operations, other array operations need iteration thru the array
as showed.
I do find the shootout is pretty accurate in terms of real ruby
performance overall.
The new web page is pretty bad in that most links are broken due to
typos and what not, so it is not clear to me the python tests are
based on psyco. There does appear to be a separate psyco benchmark.

See ruby-talk:103992

···

GGarramuno (GGarramuno@aol.com) wrote:

Eric Hodel <drbrain@segment7.net> wrote in message news:<20040617223524.GE96045@segment7.net>...
>
> This is because of the use of Numeric#step. The implementation is
> flawed.

Huh? In what way is it flawed?

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

Are you using psyco? As has been mentioned, it looks as if the python
stuff has been done using that.

···

On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds <rcoder@gmail.com> wrote:

To be honest, some of the numbers seem highly suspect to me. For
example, on my P4/Fedora Core 1 box at least, the Sieve of
Eratosthenes example runs 20-30% *faster* in Ruby than it does in
Python, while his dataset shows is taking almost twice as long.

Mark Hubbart <discord@mac.com> wrote in message news:<80B1FAE0-C0B5-11D8-82C7-000A95D2DFAE@mac.com>...

···

On Jun 17, 2004, at 4:03 PM, Eric Hodel wrote:

> Eric Hodel (drbrain@segment7.net) wrote:
>
>> This is because of the use of Numeric#step. The implementation is
>> flawed.
>
> Never mind, I am smoking crack.
>
> The real problem appears to be iteration in general.

surprisingly, using += on line 18 give a 12% speedup on my box.

cheers,
Mark

It is not so surprising += is always faster in Ruby (from my
experience).This change would place us between mzscheme and mawk.
Note that many other languages (perl, icon lua...) uses += so it
should be allowed.

Ciao

Denis

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

Psyco does some great things for the performance of certain
algorithms, but it's not part of the core Python distribution for any
number of reasons: only works on x86, breaks several standard
introspection and debugging libraries, and perhaps most importantly,
prevents redefinition of methods at runtime on optimized objects.

In short, it's almost more like a specialized reimplementation of
Python (ala Stackless) than an add-on library. I don't think it's
significantly more appropriate to use for this kind of benchmarking
than, say, Ruby or Perl scripts rewritten to use one of the Inline
libraries to do the heavy listing in C...especially when it's not
explicitly mentioned on the benchmark site.

Lennon

In article <811f2f1c040617163272c240a9@mail.gmail.com>,

···

Michael Campbell <michael.campbell@gmail.com> wrote:

On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds ><rcoder@gmail.com> wrote:

To be honest, some of the numbers seem highly suspect to me. For
example, on my P4/Fedora Core 1 box at least, the Sieve of
Eratosthenes example runs 20-30% *faster* in Ruby than it does in
Python, while his dataset shows is taking almost twice as long.

Are you using psyco? As has been mentioned, it looks as if the python
stuff has been done using that.

Isn't psycho kind of a special varient of Python? Is it fair to compare
psycho Python in the language shootout (they should probalby use the more
widely used PythonC).

Phil

hmmm... actually, 12% speed increase (actually a 12.7% time decrease, to be specific) would just kick us up one notch, putting us ahead of gawk (on my box, anyway, YMMV). But at least we wouldn't be last :slight_smile:

cheers,
Mark

···

On Jun 18, 2004, at 3:43 AM, denis wrote:

Mark Hubbart <discord@mac.com> wrote in message news:<80B1FAE0-C0B5-11D8-82C7-000A95D2DFAE@mac.com>...

On Jun 17, 2004, at 4:03 PM, Eric Hodel wrote:

Eric Hodel (drbrain@segment7.net) wrote:

This is because of the use of Numeric#step. The implementation is
flawed.

Never mind, I am smoking crack.

The real problem appears to be iteration in general.

surprisingly, using += on line 18 give a 12% speedup on my box.

cheers,
Mark

It is not so surprising += is always faster in Ruby (from my
experience).This change would place us between mzscheme and mawk.
Note that many other languages (perl, icon lua...) uses += so it
should be allowed.

...

No disputes from me, I was just mentioning I think that's what they
did, and that it might explain the discrepancy.

···

On Fri, 18 Jun 2004 08:57:59 +0900, Lennon Day-Reynolds <rcoder@gmail.com> wrote:

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

I agree that they really should stick strictly to Python code if they
are comparing languages. We don't want to know how well C performs
with a Python outer shell, we want to know how well Python performs.

Carl

···

On Fri, 18 Jun 2004 09:53:24 +0900, Phil Tomson <ptkwt@aracnet.com> wrote:

In article <811f2f1c040617163272c240a9@mail.gmail.com>,

Michael Campbell <michael.campbell@gmail.com> wrote:
>On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds > ><rcoder@gmail.com> wrote:
>>
>> To be honest, some of the numbers seem highly suspect to me. For
>> example, on my P4/Fedora Core 1 box at least, the Sieve of
>> Eratosthenes example runs 20-30% *faster* in Ruby than it does in
>> Python, while his dataset shows is taking almost twice as long.
>
>Are you using psyco? As has been mentioned, it looks as if the python
>stuff has been done using that.
>

Isn't psycho kind of a special varient of Python? Is it fair to compare
psycho Python in the language shootout (they should probalby use the more
widely used PythonC).

Phil

Phil Tomson wrote:

Isn't psycho kind of a special varient of Python? Is it fair
to compare psycho Python in the language shootout (they
should probalby use the more widely used PythonC).

You may be thinking of Pyrex, a Python dialect that compiles to C and lets
you mix C and Python data types:

http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

Pyrex is really quite nifty, especially for writing C extensions to Python
and for wrapping existing C libraries. You can still code in Python or close
to it, and let Pyrex write all the boilerplate C code for you. For
performance critical algorithms, if you're using C types the generated code
should be as fast as any other C code.

Psyco, OTOH, works with standard Python code. It's essentially a Just In
Time compiler for Python:

http://psyco.sourceforge.net/introduction.html

Psyco isn't part of the standard Python distribution--you need to install
the Psyco module and add this to the top of your source file:

import psyco; psyco.full()

But that's all there is to it. There's nothing special about the
language--it's standard Python.

Psyco is only for x86 at the moment, but it can really pep up Python code on
the x86.

What's fair for performance comparisons? Well, a thorough language benchmark
would measure Psyco, standard cPython, and Jython times for x86, plus
cPython and Jython for other popular processors.

These are all Open Source, so don't let anybody stop you from porting them
to Ruby... :slight_smile:

-Mike

Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
optimizing integer/floating point operations) would become possible, at
least theoretically, without breaking code or loosing introspection at
the overall level. I might be wrong, of course :slight_smile:

Regards,

  Michael

···

On Fri, Jun 18, 2004 at 08:57:59AM +0900, Lennon Day-Reynolds wrote:

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

Psyco does some great things for the performance of certain
algorithms, but it's not part of the core Python distribution for any
number of reasons: only works on x86, breaks several standard
introspection and debugging libraries, and perhaps most importantly,
prevents redefinition of methods at runtime on optimized objects.

did you submitted the proposed change to the authors ?
while you're at it you could ask for the removal of all thst ugly
'then' in conditionals, and looking at except.ruby there are useless
'return self' in the initialize method, then
def some_function(num)
    begin
  hi_function(num)
    rescue
        print "We shouldn't get here, exception is: #{$!.type}\n"
    end
end

should be rewritten as:

def func
other_func
rescue
  print..
end

no need of explicit begin end.

I'd do this myself but I can't test fixed implementations ATM (no ruby
here). And about the hashes test: why we have to use string
conversion? we can safely use any object as an hash key..

···

il Sat, 19 Jun 2004 03:08:06 +0900, Mark Hubbart <discord@mac.com> ha scritto::

hmmm... actually, 12% speed increase (actually a 12.7% time decrease,
to be specific) would just kick us up one notch, putting us ahead of
gawk (on my box, anyway, YMMV). But at least we wouldn't be last :slight_smile:

wrote (more or less):
..

hmmm... actually, 12% speed increase (actually a 12.7% time decrease,
to be specific) would just kick us up one notch, putting us ahead of
gawk (on my box, anyway, YMMV). But at least we wouldn't be last :slight_smile:

Is there a new definition of speed that's not as a linear function of
time?

···

On Sat, 19 Jun 2004 03:08:06 +0900, Mark Hubbart <discord@mac.com>

--
Cheers,
   Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk