The Java code was broken in many different ways, such that any numbers
generated using this code are badly skewed against Java. You can't run
benchmarks against Java, to prove Java's slow, and then write code that's
obviously crippling it. I'll only fix the blatant mistakes here...others may
do a deeper cleanup of the Java code if they wish. I'll post my code if
requested, but any Java programmer will understand the optimizations as I
list them below. They're pretty obvious.
And I don't doubt that C will probably be faster, even with those
optimizations...but it won't be faster by much and certainly not by orders
of magnitude. Algorithms where Java does especially well are any that
involve memory allocation, which doesn't come into play here but which is
applicable to almost all real-world code. The point of this thread is that
using the underlying platform code--C for Ruby, Java for JRuby--will often
help many algorithms...and this much is true. But don't venture into
comparing C against Java if you're going to make blanket statements using
flawed tests.
First, some notes on benchmarking:
- NEVER include IO when benchmarking a numeric algorithm; IO speeds vary
greatly from system to system and can vary from run to run depending on what
else is happening
- Do not include initialization code in benchmarks, especially in this case
where you're manually tweaking a gigantic array
- If you're building up a large chunk of strings, write to an internal
buffer and then write out at the end; don't write for every little tiny
operation. At the very least, use a buffer per-line, rather than a separate
write for every element on that line.
- Make sure you're actually testing the same thing on all systems; in this
case, the Java code was severely crippled on a number of levels
- I have not changed the algorithm in any way, but an iterative algorithm
would perform a lot better on all platforms.
So I made some mild optimizations:
- You do not need to compare boolean values to "true" or "false"; just use
them directly as the test condition.
- Write strings to an internal buffer or do not write them at all; to
support Unicode across platforms, Java normalizes text to the base
platform's preferred encoding, and so incurs extra overhead for this
benchmark than the other versions. If you want to make this a better test,
have the C version use wide char strings internally and normalize to ASCII
on write.
- I moved the initialization of the Compared array to a separate function
and excluded it from the test. I clear out and reinit the string buffer and
the compared array for each test run. The C code loads a static array into
memory in probably microseconds, so including this initialization for the
Java test totally skews results.
- I had the test benchmark just the call to addRow, since the Java platform
overhead is a fixed cost outside of this test. If you want to figure that
cost in, you're welcome to...I've left the timings as in the original.
- I ran the algorithm six times per test to allow the JVM to optimize it.
Note how quickly the speed improves once HotSpot gets to it.
And one caveat: I don't have perl set up properly on this machine, so I
wasn't able to generate the header or run the C code. When I do I'll post
those numbers for comparison.
Ubuntu Linux 6 (64-bit), current supported kernel (something 2.6.15ish)
Opteron 150, 2.6GHz, 2GB RAM
All Java versions are AMD64.
Java 5, client vm, no string creation/buffering:
headius@opteron:~/latin_in_java$ time java Latin
Took 1082 ms
Took 645 ms
Took 388 ms
Took 385 ms
Took 551 ms
Took 385 ms
real 0m3.667s
user 0m3.436s
sys 0m0.032s
Java 5, client vm, write strings to internal string buffer:
headius@opteron:~/latin_in_java$ time java Latin
Took 631 ms
Took 599 ms
Took 492 ms
Took 496 ms
Took 492 ms
Took 499 ms
real 0m3.340s
user 0m3.080s
sys 0m0.116s
Java 6, client vm, no strings:
headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java
Latin
Took 400 ms
Took 395 ms
Took 408 ms
Took 369 ms
Took 367 ms
real 0m2.459s
user 0m2.368s
sys 0m0.032s
Java 6, client vm, write strings to internal buffer:
headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java
Latin
Took 531 ms
Took 497 ms
Took 478 ms
Took 486 ms
Took 494 ms
real 0m3.172s
user 0m2.940s
sys 0m0.104s
···
On 8/1/06, Charles O Nutter <headius@headius.com> wrote:
Ok, so there's a bunch of problems with the Java version.
- In addition to the addRow run and the Java startup time you're also
benchmarking over 5200 array modifications to set Compared values to true
- Your variable naming is entirely contrary to every Java coding
convention
published (not a benchmark thing, but it sets off any Java devs warning
flags)
- Almost all of the time spent running is spent building and printing
strings
Benchmarking just the algorithm run itself with no gross string creation
and
printing, I'm getting in the neighborhood of 370ms per invocation once
HotSpot has optimized the code. I'll have more detailed numbers shortly.
--
Contribute to RubySpec! @ Welcome to headius.com
Charles Oliver Nutter @ headius.blogspot.com
Ruby User @ ruby.mn
JRuby Developer @ www.jruby.org
Application Architect @ www.ventera.com
--
Contribute to RubySpec! @ Welcome to headius.com
Charles Oliver Nutter @ headius.blogspot.com
Ruby User @ ruby.mn
JRuby Developer @ www.jruby.org
Application Architect @ www.ventera.com