Performance Ruby

ruby-talk
21

Austin Ziegler wrote:

>>> Goel wrote:
>>>> Is there any benchmark of the performance of Ruby vs. C, C++, Java
>>>> etc?
>>>> I couln't find it on the ruby website.
>>> There's the Great Computer Language Shootout.
>>> <URL:http://shootout.alioth.debian.org/&gt;
>> Which, as I keep reminding people, isn't worth the paper it isn't
>> printed on. It's a waste of time, effort, and credibility to keep
>> promoting it as if the people who created it know what they're doing.
>>
>> Not all benchmarks are bad, but the GCLS is the least useful.
> I always had the impression that the GCLS was presented as "purely for
> entertainment purposes only," and people who take them too seriously
> were to be scoffed at by sane people everywhere as well as the
> founders of the project.
>
> That's just my perception, but this is one of the few cases where I
> don't really care whether or my perception is correct.

Okay, to be completely fair: yes, the GCLS is presented as "purely for
entertainment purposes only." At least that's what is said on the GCLS
website, which makes it the operating theory, at least.

In practice, though, the Alioth shootout is heavily promoted by the
people what run it and others, and there are comparisons between
different languages and little is done to make sure that the various
languages don't cheat (I found a cheat in the Perl implementation of the
Ackermann and a sort-of-cheat in the Python implementation).

The FAQ instructions direct you to this bug report page
https://alioth.debian.org/tracker/?atid=411002&group_id=30402&func=browse

There's a
whole veneer of respectability about this particular set of tests,
complete with the encouragement to "make your language perform better."
In other words, for something that's "for entertainment purposes only,"
there's a lot of time spent making it look "legitimate."

Is this quoted text "make your language perform better" actually on the
shootout website? What's the URL to the page that contains this text?

When the people who run it are confronted with this, they fall back on
the "it's not serious" line ... while very shortly after doing something
that suggests that it is, indeed, serious.

The Alioth shootout is dishonest in its presentation and purpose. It
does *more* than place "performance" numbers on the screen; it offers an
interpretation of those numbers ... all the while pretending not to
offer said interpretation.

What's the URL to the page that "offers an interpretation of those
numbers?

···

On 8/12/05, Brian Wisti <brian.wisti@gmail.com> wrote:
> On 8/12/05, Austin Ziegler <halostatue@gmail.com> wrote:
>> On 8/11/05, mathew <meta@pobox.com> wrote:

Real world performance numbers are more useful, and when Ara Howard
tells me that Ruby performs fast enough for his image crunching, or Rich
Kilmer tells me that Ruby made Cougaar possible and fast, then I believe
them. These people (and others) are pushing Ruby further than anyone
else, including people who use Ruby on Rails. And yet, I also believe
DHH when he says that Rails scalability isn't a big issue, in that it
scales the same way that web applications have scaled for a while, and
the same way that Kuro5hin and Slashdot scaled -- more machines with
load balancing and separating the database server from the application
server.

If you want to see an *intense* Ruby application, look at my own
PDF::Writer. The layout engine is about as computationally intensive as
one will get in pure Ruby, and would merit dropping to C in some places
(except that I have deliberately made it a goal to keep the code pure
Ruby). On my VERY SLOW LAPTOP, it takes no more than ten minutes to
generate the 90+ page manual. It takes a matter of seconds to generate
each demo (and most of *that* is in the startup).

On my 3Ghz work machine, it's about 2 1/2 - 3 minutes to do the same
manual. I suspect that with the report generator that's being talked
about, there will be some time involved with the PDF generation for
output, but it will still be fast enough to be usable from Rails.

I'm getting ready to add SVG support to PDF::Writer, and my current code
is whipping through complex SVGs pretty quickly. For some things, it
takes enough processing time that I'll recommend people do partial
rendering beforehand -- which is a good idea in any case for many
things and not just for performance reasons. I'll be documenting the
technique, too.

I'm not saying anything new here, not really.

-austin
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

22

Only problem with that one is that it is SO old it's using ruby 1.6.7

j.

···

On 8/11/05, Marek Kubica <pythonmailing@web.de> wrote:

Hello!

On Fri, 12 Aug 2005 01:04:31 +0900 Jeff Wood wrote:

> There is an online comparison of MANY computer languages ...
>
> http://shootout.alioth.debian.org/
>
> .. there are some programs that don't seem to have any ruby solution
> yet... if any of you feel up to it, ( or if you want to look at the
> existing solutions that are there and performance tweak them ) ... It
> would be a "good thing"

Heres another version of that shootout
The Great Win32 Computer Language Shootout

Brian Schröder put it imho right:
"Only optimize if it is too slow"

greets,
Marek

--
"So long, and thanks for all the fish"

Jeff Wood

23

Austin Ziegler wrote:

> There is an online comparison of MANY computer languages ...
>
> http://shootout.alioth.debian.org/
>
> ... there are some programs that don't seem to have any ruby solution
> yet... if any of you feel up to it, ( or if you want to look at the
> existing solutions that are there and performance tweak them ) ... It
> would be a "good thing"
>
> Anyways, I hope that provides the informatoin that the original post
> was looking for.

The best option for this shootout is to NOT implement anything for it.

The people running it don't know the first thing about what they're
doing and won't take responsibility for the errors in methodology and
the fact that their presentation encourages bad interpretation. The
Alioth shootout is dishonest to its core.

This is baseless abuse.

Lothar Scholz is demonstrably correct - some pure Ruby programs will be
hundreds of times slower than optimized compiled code, and some
programs will be as little as 1.5x slower.

Whether that matters or not depends on what we're trying to accomplish.

···

On 8/11/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

-austin
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

24

Austin Ziegler wrote:

There is an online comparison of MANY computer languages ...
http://shootout.alioth.debian.org/
... there are some programs that don't seem to have any ruby
solution yet... if any of you feel up to it, ( or if you want to
look at the existing solutions that are there and performance tweak
them ) ... It would be a "good thing"

Anyways, I hope that provides the informatoin that the original post
was looking for.

The best option for this shootout is to NOT implement anything for
it.

The people running it don't know the first thing about what they're
doing and won't take responsibility for the errors in methodology and
the fact that their presentation encourages bad interpretation. The
Alioth shootout is dishonest to its core.

This is baseless abuse.

Incorrect. This is not baseless abuse. It is accurate based on numerous
discussions in which you have been involved. You refuse to acknowledge
that the shootout encourages bad interpretation (e.g., the whole "lang
v. lang" comparisons, the graphs, etc.), and you refuse to modify the
way that the whole idiotic enterprise works to discourage such bad
interpretation.

Lothar Scholz is demonstrably correct - some pure Ruby programs will
be hundreds of times slower than optimized compiled code, and some
programs will be as little as 1.5x slower.

That's actually an irrelevancy and, itself, is a bad interpretation.
Hand-tuned assembler will often be much faster than even optimized
compiled code. Do you want to write everything in assembler? I thought
not.

Whether that matters or not depends on what we're trying to
accomplish.

And what I'm trying to accomplish is to convince people to stop using
your stupid project, because your project is based on dishonest
pseudo-statistical comparisons and idiotic "benchmarks." At least have
the honesty and decency to start admitting you don't know what you're
doing and start *removing* some things from the web pages that encourage
bad interpretation.

As to your other post, I haven't submitted a bug because I think that
the whole project is a waste. The only bug report that I'd submit would
be "shut the whole damned things down." Of course, you'll ignore it.
There is also the point that while your pages may not explicitly state
that one should improve the performance of one's own language, there is
strong implicit encouragement toward such, and the blind acceptance of
tests that are on the verge of cheating (e.g., the Perl Ackermann; the
Python Ackermann isn't far off, either, given that it won't run at all
without one particular non-algorithmic line, while not running the Ruby
Ackermann with OS configurations that *will* let it run) indicate that
you really are as clueless as I think you are.

-austin

···

On 8/15/05, Isaac Gouy <igouy@yahoo.com> wrote:

On 8/11/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

25

Austin Ziegler wrote:

> Austin Ziegler wrote:
>>> There is an online comparison of MANY computer languages ...
>>> http://shootout.alioth.debian.org/
>>> ... there are some programs that don't seem to have any ruby
>>> solution yet... if any of you feel up to it, ( or if you want to
>>> look at the existing solutions that are there and performance tweak
>>> them ) ... It would be a "good thing"
>>>
>>> Anyways, I hope that provides the informatoin that the original post
>>> was looking for.
>> The best option for this shootout is to NOT implement anything for
>> it.
>>
>> The people running it don't know the first thing about what they're
>> doing and won't take responsibility for the errors in methodology and
>> the fact that their presentation encourages bad interpretation. The
>> Alioth shootout is dishonest to its core.
> This is baseless abuse.

Incorrect. This is not baseless abuse. It is accurate based on numerous
discussions in which you have been involved. You refuse to acknowledge
that the shootout encourages bad interpretation (e.g., the whole "lang
v. lang" comparisons, the graphs, etc.), and you refuse to modify the
way that the whole idiotic enterprise works to discourage such bad
interpretation.

> Lothar Scholz is demonstrably correct - some pure Ruby programs will
> be hundreds of times slower than optimized compiled code, and some
> programs will be as little as 1.5x slower.

That's actually an irrelevancy and, itself, is a bad interpretation.
Hand-tuned assembler will often be much faster than even optimized
compiled code. Do you want to write everything in assembler? I thought
not.

> Whether that matters or not depends on what we're trying to
> accomplish.

And what I'm trying to accomplish is to convince people to stop using
your stupid project, because your project is based on dishonest
pseudo-statistical comparisons and idiotic "benchmarks." At least have
the honesty and decency to start admitting you don't know what you're
doing and start *removing* some things from the web pages that encourage
bad interpretation.

As to your other post,

http://groups.google.com/group/comp.lang.ruby/msg/696a3b04c450815c?hl=en&

I haven't submitted a bug because I think that
the whole project is a waste. The only bug report that I'd submit would
be "shut the whole damned things down." Of course, you'll ignore it.

There is also the point that while your pages may not explicitly state
that one should improve the performance of one's own language,

You quoted the Shootout website as saying "make your language perform
better"

Do you now agree that no such text appears on the Shootout website?

You also claimed that
"The Alioth shootout is dishonest in its presentation and purpose. It
does *more* than place "performance" numbers on the screen; it offers
an interpretation of those numbers ... all the while pretending not to
offer said interpretation."

Do you now agree that no such text appears on the Shootout website?

there is
strong implicit encouragement toward such, and the blind acceptance of
tests that are on the verge of cheating (e.g., the Perl Ackermann; the
Python Ackermann isn't far off, either, given that it won't run at all
without one particular non-algorithmic line, while not running the Ruby
Ackermann with OS configurations that *will* let it run) indicate that
you really are as clueless as I think you are.

Personal abuse never makes a good argument.

Ruby Ackermann does run for N=7 and it's slower than Awk.

···

On 8/15/05, Isaac Gouy <igouy@yahoo.com> wrote:
>> On 8/11/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

-austin
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

26

Austin Ziegler wrote:

I haven't submitted a bug because I think that the whole project is a
waste. The only bug report that I'd submit would be "shut the whole
damned things down." Of course, you'll ignore it.

There is also the point that while your pages may not explicitly
state that one should improve the performance of one's own language,

You quoted the Shootout website as saying "make your language perform
better"

That wording specifically? But what about:

    Compare programming language performance on a few dozen flawed
    benchmarks and contribute faster more elegant programs.

I don't know about you, but "contribute faster more elegant programs" is
a strong endorsement to "make your [language of choice] perform better."
The link for "contribute...", by the way, leads to this text:

    Fix slow broken benchmark programs - read How should I implement…?
    and then contribute a fixed program.

Again, that's an endorsement. So no, I do *not* agree that no such text
appears. I'm not reading between the lines, either. Remove the "faster"
and "slow" from this and then you might have a leg to stand on. Keep it,
and my charge stands.

You also claimed that
"The Alioth shootout is dishonest in its presentation and purpose. It
does *more* than place "performance" numbers on the screen; it offers
an interpretation of those numbers ... all the while pretending not to
offer said interpretation."

Do you now agree that no such text appears on the Shootout website?

Nope. The very comparison between languages provides interpretation.
Minimal interpretation, mind you, but interpretation nonetheless. You
*cannot* meaningfully compare some algorithms in various languages.

there is strong implicit encouragement toward such, and the blind
acceptance of tests that are on the verge of cheating (e.g., the Perl
Ackermann; the Python Ackermann isn't far off, either, given that it
won't run at all without one particular non-algorithmic line, while
not running the Ruby Ackermann with OS configurations that *will* let
it run) indicate that you really are as clueless as I think you are.

Personal abuse never makes a good argument.

Ruby Ackermann does run for N=7 and it's slower than Awk.

Ack(3,9): 4093 # ack.rb
real 0m28.750s
user 0m28.670s
sys 0m0.090s

Ack(3,9): 4093 # ack.py
real 0m20.595s
user 0m13.930s
sys 0m0.000s

If I remove the "setrecursionlimit" lines from the Python code, I get:
  RuntimeError: maximum recursion depth exceeded

By the way, I got the ack.rb running by doing:

  ulimit -s 32768
  time ack.rb

Further, unless you're looking at an update that I haven't seen, there's
no awk implementation on the shootout.

The flaw with the Perl Ackermann I've pointed out in the past (namely,
there's two implementations, and one of them is done purely for the
purpose of doing better in the results).

If Ruby had a way to modify ulimit for its own process the way that
Python appears to do, then it wouldn't be an issue for Ackermann. But
because it doesn't, and you won't run Ackermann with this, there's going
to be a "won't run". It's more than that, but I stand by this: the
Alioth shootout is dishonest, and you haven't done anything to make it
honest.

I'm not sure you *can* make it completely honest, but you can certainly
decrease how seriously people take this stinking pile of crap. Yet you
refuse to do so. I wonder why.

-austin

···

On 8/15/05, Isaac Gouy <igouy@yahoo.com> wrote:
--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca