Comparison Ruby, Python, Php, Groovy ecc

Are those executables compiled with identical compilers + compile flags?

Marco Mastrodonato wrote:

···

Comparison script languages for the fractal geometry, these are the
languages i tested:

Java
Lua 5.1.4
Php 5.3.0
Python 2.6.2
Python 3.1.1
Jython 2.5.0
Groovy 1.6.3
Jruby 1.3.1
Ruby 1.9.1 p129
Ruby 1.8.6 p368
Ruby 1.8.6 p111
IronRuby 0.9.0
IronPython 2.0.2
Perl 5.10.0

Let me know yours comment

http://mastrodonato.info/index.php/2009/08/comparison-script-languages-for-the-fractal-geometry/?lang=en

Two things I notice.

It seems ruby 1.9 indeed managed to keep up with the python versions
more than the older ruby versions. And it seems to be (almost) as fast
as perl for that test.

The other thing, which is very strange, is that the ironruby
implementation is significantly slower than ironpython, whereas the
other versions arent by as much. What is wrong here?

IronRuby 0.9.0 6,038 39x
IronPyhon 2.0.2 0,978 6x

vs

Ruby 1.9.1 p129 2,688 18x
Python 3.1.1 1,566 10x

···

--
Posted via http://www.ruby-forum.com/.

Urabe Shyouhei <shyouhei@ruby-lang.org> writes:

Are those executables compiled with identical compilers + compile flags?

The question is understandable, but these implementation might very
well be written in different languages, so it doesn't really matter.
We can assume that they are as much as possible, if they come from a
common distribution.

Marco Mastrodonato wrote:

Comparison script languages for the fractal geometry, these are the
languages i tested:

Java
Lua 5.1.4
Php 5.3.0
Python 2.6.2
Python 3.1.1
Jython 2.5.0
Groovy 1.6.3
Jruby 1.3.1
Ruby 1.9.1 p129
Ruby 1.8.6 p368
Ruby 1.8.6 p111
IronRuby 0.9.0
IronPython 2.0.2
Perl 5.10.0

Let me know yours comment

http://mastrodonato.info/index.php/2009/08/comparison-script-languages-for-the-fractal-geometry/?lang=en

I added a comment to the web site, but I'm not sure it was taken into
account (I didn't got the same feed back as for a second shorter
comment). So here it is again:

For completeness, could you please try Common Lisp too?

You could use sbcl 1.0.29 (MS-Windows port in progress) at:
http://prdownloads.sourceforge.net/sbcl/sbcl-1.0.29-x86-windows-binary.msi

-------(bench1.lisp)----------------------------------------------------
(declaim (optimize (speed 3) (space 2) (debug 0) (safety 0)))
(declaim (ftype (function (single-float single-float) fixnum) iterate))

(defparameter *bailout* 16.0)
(defparameter *max-iterations* 1000)

(defun bench1 ()
  (format t "Rendering...~%") (force-output)
  (loop :for y fixnum :from -39 to 39 :do
     (terpri)
     (loop :for x fixnum :from -39 to 39 :do
        (princ (if (zerop (iterate (the single-float (/ x 40.0))
                                   (the single-float (/ y 40.0))))
                   "*"
                   " "))))
  (finish-output))

(defun iterate (x y)
  (declare (single-float x y))
  (loop
     :with cr single-float = (- y 0.5)
     :with ci single-float = x
     :with zi single-float = 0.0
     :with zr single-float = 0.0
     :for i fixnum :from 0 :below *max-iterations*
     :do (let ((temp (* zr zi))
               (zr2 (* zr zr))
               (zi2 (* zi zi)))
           (declare (single-float temp zr2 zi2))
           (setf zr (+ (- zr2 zi2) cr)
                 zi (+ temp temp ci))
           (when (< (the single-float *bailout*) (the single-float (+ zi2 zr2)))
             (return-from iterate i)))
     :finally (return-from iterate 0)))

(time (bench1))

···

------------------------------------------------------------------------

Run the test with:

    sbcl --no-userinit --eval '(load (compile-file "bench1.lisp"))' --eval '(quit)'

--
__Pascal Bourguignon__

Urabe Shyouhei wrote:

Are those executables compiled with identical compilers + compile flags?

Ruby p111 is mswin32 (onclick installer) the others are mingw32
downloaded from http://rubyinstaller.org/downloads/
Iron and java version, Python, groovy, php ...was been downloaded from
main site. The perl's exe is strawberry's installation. Java was been
compiled by netbeans.

···

--
Posted via http://www.ruby-forum.com/.

Pascal J. Bourguignon wrote:

Urabe Shyouhei <shyouhei@ruby-lang.org> writes:

Are those executables compiled with identical compilers + compile flags?

The question is understandable, but these implementation might very
well be written in different languages, so it doesn't really matter.
We can assume that they are as much as possible, if they come from a
common distribution.

It does matter very much. At least identical ones should be used for each
underlying languages to write them. The report says that test was held on
Windows XP, so I suspect there is no such thing as "a common distribution" on it.

Marc Heiler wrote:

Two things I notice.

It seems ruby 1.9 indeed managed to keep up with the python versions
more than the older ruby versions. And it seems to be (almost) as fast
as perl for that test.

The other thing, which is very strange, is that the ironruby
implementation is significantly slower than ironpython, whereas the
other versions arent by as much. What is wrong here?

1. Python is still faster, but version 1.9.1 goes very well catching up
good results, goes better than perl, its script was been very optimized
to get that result, 2.7s against 4s of the normal version.

2. I think there aren't correlation between these versions: Ironruby,
ironpython, ruby and python are different projects and with different
development.

···

--
Posted via http://www.ruby-forum.com/.

Marco Mastrodonato wrote:

Urabe Shyouhei wrote:

Are those executables compiled with identical compilers + compile flags?

Ruby p111 is mswin32 (onclick installer) the others are mingw32
downloaded from http://rubyinstaller.org/downloads/
Iron and java version, Python, groovy, php ...was been downloaded from
main site. The perl's exe is strawberry's installation. Java was been
compiled by netbeans.

So the next thing you should do is to recompile by yourself to uniform
compilation environment among them. Fairness is the most essential part when
you want to do an emotional Yo-Yo on a benchmark like that.

And luckily, all implementations nominated are open sourced.

Pascal J. Bourguignon wrote:

I added a comment to the web site, but I'm not sure it was taken into
account (I didn't got the same feed back as for a second shorter
comment). So here it is again:

For completeness, could you please try Common Lisp too?

I got only the second comment, anyway, i'll add lisp asap and thanks for
your work

···

--
Posted via http://www.ruby-forum.com/.

Urabe Shyouhei wrote:

So the next thing you should do is to recompile by yourself to uniform
compilation environment among them. cut ...

Yes i could do that but will be a different comparison. I don't think
there are many people who compile themself from the sources, on windows

···

--
Posted via http://www.ruby-forum.com/.

Another thing to consider when running the benchmark runs to not write
the program output to the console (e.g. ./program > /dev/null or
program.exe > NUL). Presumably the intention is to compare the speed
of the implementations calculating the result and not test how fast
the desktop environment can display the result. While it may not in
all cases make a significant difference in the results, it is best to
remove that variable from the test.

For example, in one case I saw a benchmark that claimed something to
the effect of "look BRANDX is not slow: this program in BRANDX is only
slightly slower than C!". But, on closer examination it took the C
program less than 1 second to calculate the result and it took the
BRANDX program a few seconds to calculate the result; but, in both
cases, it took cmd.exe many seconds to display the result.

Using your Java program* (10 runs):
   Min. 1st Qu. Median Mean 3rd Qu. Max. Std-dev
  158.0 178.8 196.5 195.6 205.5 249.0 27.37882

and, sending out to to /dev/null:
   Min. 1st Qu. Median Mean 3rd Qu. Max. Std-dev
  106.0 108.2 109.0 109.0 110.0 112.0 1.699673

[*] Modified to send the time output to standard error

···

On Mon, Aug 24, 2009 at 8:47 AM, Marco Mastrodonato<m.mastrodonato@gmail.com> wrote:

Urabe Shyouhei wrote:

So the next thing you should do is to recompile by yourself to uniform
compilation environment among them. cut ...

Yes i could do that but will be a different comparison. I don't think
there are many people who compile themself from the sources, on windows

Marco Mastrodonato wrote:

Urabe Shyouhei wrote:

So the next thing you should do is to recompile by yourself to uniform
compilation environment among them. cut ...

Yes i could do that but will be a different comparison. I don't think
there are many people who compile themself from the sources, on windows

If you really need speed recompiling is the easiest way to achieve it. Does it
really worth comparing those executables? Their "slowness" might be sourced
from a bad compilation. I don't know about other language but 1.8.6-p111
versus 1.8.6-p368 case is (I believe) due to difference of their compilers.
What is your point on that article, then? Are you really comparing languages?
not compilers behind them?

unknown wrote:

Another thing to consider when running the benchmark runs to not write
the program output to the console

unknown, thanks for your advice. I agree with you, i didn't care to test
the stdout write speed. Honestly i tried to write into a variable and
send to output only at the end... i saw there was no difference (ruby's
script) and i went back to the original, more nice to use because you
can early feel the speed. In your test, instead, there's a discrete
difference

···

--
Posted via http://www.ruby-forum.com/.

Urabe Shyouhei wrote:

If you really need speed recompiling is the easiest way to achieve it.

The aim is a languages's comparison, using the downlodable package
...without the need to compile every interpreter. I could be agree with
you about a "real comparison", but this is the practise done by the
99,5% of the windows users.

@Pascal
Sorry, i'm having some trouble with lisp, have i to use the txt format?
Or have i to compile it first? Take a look:

C:\Lavoro\Progetti\Test\Bench\multilanguage>sbcl --no-userinit --eval
'(load (compile-file "bench1.lisp"))' --eval '(quit)'
This is SBCL 1.0.29, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.

This is experimental prerelease support for the Windows platform: use
at your own risk. "Your Kitten of Death awaits!"

debugger invoked on a END-OF-FILE:
  end of file on #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}>

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Ignore runtime option --eval "'(load".
  1: [ABORT ] Skip rest of --eval and --load options.
  2: Skip to toplevel READ/EVAL/PRINT loop.
  3: [QUIT ] Quit SBCL (calling #'QUIT, killing the process).

(SB-IMPL::STRING-INCH #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}> T NIL)
0]

···

--
Posted via http://www.ruby-forum.com/.

I just ran a few of these benchmarks on my machine. I got different results
Linux 2.6.28-11-generic GNU/Linux
Intel core 2 duo 2GHz, 2MB L2 cache, 3GB ram DDR

         Language Time for 100 iterations times slower than java with
-server java1.6 –server 0.18 1 Ruby1.8 7.78 44.07 Ruby1.9.2 4.2 23.78
Jruby 2.5 14.16 Jruby1.3.1—sever 2.31 13.1 java1.6 -client
0.18 1.01 python 2.6.2
3.04 17.21

Jruby is the fastest here with 13 times slower than java. I am sure there
are other command line options which may allow jruby to perform faster, that
I am not aware of. Also shows ruby1.9.2 is slower than python 2.6.2

···

On Mon, Aug 24, 2009 at 10:43 PM, Marco Mastrodonato < m.mastrodonato@gmail.com> wrote:

Urabe Shyouhei wrote:
> If you really need speed recompiling is the easiest way to achieve it.

The aim is a languages's comparison, using the downlodable package
...without the need to compile every interpreter. I could be agree with
you about a "real comparison", but this is the practise done by the
99,5% of the windows users.

@Pascal
Sorry, i'm having some trouble with lisp, have i to use the txt format?
Or have i to compile it first? Take a look:

C:\Lavoro\Progetti\Test\Bench\multilanguage>sbcl --no-userinit --eval
'(load (compile-file "bench1.lisp"))' --eval '(quit)'
This is SBCL 1.0.29, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.

This is experimental prerelease support for the Windows platform: use
at your own risk. "Your Kitten of Death awaits!"

debugger invoked on a END-OF-FILE:
end of file on #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}>

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
0: [CONTINUE] Ignore runtime option --eval "'(load".
1: [ABORT ] Skip rest of --eval and --load options.
2: Skip to toplevel READ/EVAL/PRINT loop.
3: [QUIT ] Quit SBCL (calling #'QUIT, killing the process).

(SB-IMPL::STRING-INCH #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}> T NIL)
0]
--
Posted via http://www.ruby-forum.com/.

Okay that table got completely screwed. I will try once more or you
may get these results here http://pastie.org/594583

Language Time for 100 iterations Times slower
than java with -server
java1.6 –server 0.18 1
Ruby1.8.7 7.78 44.07
Ruby1.9.2 4.2 23.78
Jruby 2.5 14.16
Jruby1.3.1—sever 2.31 13.1
java1.6 -client 0.18 1.01
python 2.6.2 3.04 17.21

Jruby is the fastest here with 13 times slower than java. I am sure there
are other command line options which may allow jruby to perform faster, that
I am not aware of. Also shows ruby1.9.2 is slower than python 2.6.2

···

On Wed, Aug 26, 2009 at 1:54 AM, Piyush Ranjan <piyush.pr@gmail.com> wrote:

I just ran a few of these benchmarks on my machine. I got different results
Linux 2.6.28-11-generic GNU/Linux
Intel core 2 duo 2GHz, 2MB L2 cache, 3GB ram DDR

    Language Time for 100 iterations times slower than java with

-server java1.6 –server 0.18 1 Ruby1.8 7.78 44.07 Ruby1.9.2 4.2 23.78
Jruby 2.5 14.16 Jruby1.3.1—sever 2.31 13.1 java1.6 -client
0.18 1.01 python 2.6.2
3.04 17.21

Jruby is the fastest here with 13 times slower than java. I am sure there
are other command line options which may allow jruby to perform faster, that
I am not aware of. Also shows ruby1.9.2 is slower than python 2.6.2

On Mon, Aug 24, 2009 at 10:43 PM, Marco Mastrodonato < > m.mastrodonato@gmail.com> wrote:

> Urabe Shyouhei wrote:
> > If you really need speed recompiling is the easiest way to achieve it.
>
> The aim is a languages's comparison, using the downlodable package
> ...without the need to compile every interpreter. I could be agree with
> you about a "real comparison", but this is the practise done by the
> 99,5% of the windows users.
>
> @Pascal
> Sorry, i'm having some trouble with lisp, have i to use the txt format?
> Or have i to compile it first? Take a look:
>
> C:\Lavoro\Progetti\Test\Bench\multilanguage>sbcl --no-userinit --eval
> '(load (compile-file "bench1.lisp"))' --eval '(quit)'
> This is SBCL 1.0.29, an implementation of ANSI Common Lisp.
> More information about SBCL is available at <http://www.sbcl.org/>.
>
> SBCL is free software, provided as is, with absolutely no warranty.
> It is mostly in the public domain; some portions are provided under
> BSD-style licenses. See the CREDITS and COPYING files in the
> distribution for more information.
>
> This is experimental prerelease support for the Windows platform: use
> at your own risk. "Your Kitten of Death awaits!"
>
> debugger invoked on a END-OF-FILE:
> end of file on #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}>
>
> Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
>
> restarts (invokable by number or by possibly-abbreviated name):
> 0: [CONTINUE] Ignore runtime option --eval "'(load".
> 1: [ABORT ] Skip rest of --eval and --load options.
> 2: Skip to toplevel READ/EVAL/PRINT loop.
> 3: [QUIT ] Quit SBCL (calling #'QUIT, killing the process).
>
> (SB-IMPL::STRING-INCH #<SB-IMPL::STRING-INPUT-STREAM {23B59BD1}> T NIL)
> 0]
> --
> Posted via http://www.ruby-forum.com/.
>
>