How to speed up ruby and make it as fast as possible

[Ben_Johnson1 (Ben Johnson)]

I did some profiling on my code and the real bottle neck is using the Net::HTTP library. I'm using it to grab HTML from a website. Our server is located in Boston and after doing a digg I came to found out the server we are communicating with is located in LA. I realize the communication between the 2 servers is something I can not speed up, as our server is in a very nice data center with 6 T3s. But I would still like to speed up Ruby as much as possible. Am I wrong here?

Lastly, I read in another thread that the Net::HTTP library was actually slow and using something like curl is faster. Is this correct?

Anyways, going back to my original post. I am new to ruby and I just wanted to make sure that I was getting the most out of it. I was uncertain if ruby had a configuration file or any kind of configuration that would keep it from using ALL of your computer's resources.

Thanks for the help.

Thank You,
Ben Johnson
E: bjohnson@contuitive.com

···

On Jul 16, 2006, at 8:43 AM, Alexandru Popescu wrote

I would say that in the world I am come from, or maybe in an ideal
world I would say:
- have you identified which parts are slow for you? I mean what is
performing bad from your business perspective?
- if not, than I would ask how can I figure out which parts are the
slowest for the scenarios I want more speed
- if yes, I would point my finger to the problem and ask if there is
any knowledge on how to speed up those fragment. If not, than what
alternatives I can take.

[Dominik_Bathon]

Here they are:

diff -ruw lib_orig/pdf/writer/fontmetrics.rb lib/pdf/writer/fontmetrics.rb
--- lib_orig/pdf/writer/fontmetrics.rb 2005-06-16 06:28:25.000000000 +0200
+++ lib/pdf/writer/fontmetrics.rb 2006-07-08 01:33:40.000000000 +0200
@@ -57,6 +57,7 @@
      font = nil
      afm = nil

+ resss = nil
      metrics_path.each do |path|
        afm_file = File.join(path, "#{name}.afm").gsub(/\.afm\.afm$/o, ".afm")
        rfm_file = "#{afm_file}.rfm"
@@ -67,7 +68,8 @@
          if File.exists?(rfm_file)
            data = File.open(rfm_file, "rb") { |file| file.read }
            font = Marshal.load(data)
- return font
+ resss = font
+ break
          end
        rescue
          nil
@@ -188,6 +190,7 @@
        end rescue nil # Ignore file errors
        break unless font.nil?
      end
+ return resss if resss

      raise ArgumentError, "Font #{font_name} not found." if font.nil?
      font

This is because return from inside a block is not supported.

diff -ruw lib_orig/pdf/writer.rb lib/pdf/writer.rb
--- lib_orig/pdf/writer.rb 2005-09-07 19:01:14.000000000 +0200
+++ lib/pdf/writer.rb 2006-07-08 01:34:32.000000000 +0200
@@ -1687,7 +1687,7 @@
      tw = width / size.to_f * 1000

      pos = -1
- loop do
+ while true
        pos += 1
        break if pos == text.size
        font_change = true

Again because return from inside a block is not supported (so I just changed the block to a while loop).

diff -ruw lib_orig/transaction/simple.rb lib/transaction/simple.rb
--- lib_orig/transaction/simple.rb 2005-05-05 18:16:49.000000000 +0200
+++ lib/transaction/simple.rb 2006-07-08 01:33:00.000000000 +0200
@@ -650,7 +650,8 @@
          if respond_to?(:instance_variable_get)
            instance_variable_set(vv, rr.instance_variable_get(vv))
          else
- instance_eval(%q|#{vv} = rr.instance_eval("#{vv}")|)
+ $__________xxxxxxxx___rr = rr
+ instance_eval(%q|#{vv} = $__________xxxxxxxx___rr.instance_eval("#{vv}")|)
          end
        end

This is because the instance_eval wouldn't see the local variable rr.

That's all that I changed to get it to compile and the resulting C extension seems to work correctly with the demos.

Dominik

···

On Sun, 16 Jul 2006 09:40:47 +0200, Austin Ziegler <halostatue@gmail.com> wrote:

On 7/14/06, Dominik Bathon <dbatml@gmx.de> wrote:

Ruby2CExtension doesn't support 100% of Ruby's features, but it supports
enough to handle real world libraries. It can compile Austin Ziegler's
PDF::Writer for example (with some small changes). The resulting C
extension is about 33% faster (i.e. 3s instead of 4s) than the Ruby code.

I'd love to hear what those changes are.

[Alex_Young]

M. Edward (Ed) Borasky wrote:

Yukihiro Matsumoto wrote:

Hi,

>My suggestion to anyone new or old that finds Ruby slow is this...
>
>It will never get better by this group.

Agreed. Saying it's slow is far easier to make it fast. I know it's
slow. We are working for it. What can I say more.

Blaming, beating, or even insulting us would never help performance.

                            matz.

One thing that would really help is a "standard Ruby benchmark suite" that one could use to quickly compare alternative implementations. Since it's a little unclear to me what the "target market focus" is for the language, I'll invite the members of this list to post what they think should be in such a suite. Although it's an obvious "answer", I think I'd prefer to leave Rails out of the mix, because Rails performance depends on both a web server and a database directly, and the server OS, client/browser and network bandwidth indirectly.

So ... what should be in a "standard Ruby benchmark suite?" Does such a thing exist already?

I've never heard of one, but I can't think of anyone better than a professional performance consultant to drive one

If they were to be chosen as the focal point for performance improvement, the things I'd benefit most from being benchmarked are:

- REXML
- Mechanize
- WEBrick
- Test::Unit

I've pointed out higher-level libraries rather than core because they exercise more of Ruby with fewer data points. Is that the right approach? I realise that one downfall of this perspective is that the library authors may have avoided certain techniques precisely because they are too slow in the current Ruby implementation, so those techniques would be completely sidelined. Continuations are one that instantly spring to mind - I can only find callcc used in one file in the stdlib.

An alternative would be to use rubicon's full run time as the benchmark - that way we know that all the language features are being flexed, but some might get too much weighting.

Thoughts?

···

In message "Re: How to speed up ruby and make it as fast as possible" >> on Sun, 16 Jul 2006 18:16:23 +0900, Reggie Mr <buppcpp@yahoo.com> >> writes:

--
Alex

[Alex_Young]

Ben Johnson wrote:
<snip>

I did some profiling on my code and the real bottle neck is using the Net::HTTP library. I'm using it to grab HTML from a website. Our server is located in Boston and after doing a digg I came to found out the server we are communicating with is located in LA. I realize the communication between the 2 servers is something I can not speed up, as our server is in a very nice data center with 6 T3s. But I would still like to speed up Ruby as much as possible. Am I wrong here?

Lastly, I read in another thread that the Net::HTTP library was actually slow and using something like curl is faster. Is this correct?

Yes, but if you have a look through that thread, there was a specific change mentioned that you can make in the library to speed up Net::HTTP. From the top of my head (you may wish to check this), it was changing the buffer size in BufferedIO#rbuf_fill from 1024 to 8192.

Why that's a constant, I don't know...

···

--
Alex

[M_Edward_Ed_Borasky1 (M. Edward (Ed) Borasky)]

Alex Young wrote:

I've never heard of one, but I can't think of anyone better than a professional performance consultant to drive one

Well, I'm on the YARV mailing list, and I just saw a message go by on a similar subject. They do have a benchmark they use, which looks like a mix of "traditional" small benchmarks like Tower of Hanoi and some benchmarks designed to deliberately exercise Ruby internals. I'm going to see if I can get a copy and have a look at it. No sense re-inventing a wheel, especially if four of them make YARV go faster.

If they were to be chosen as the focal point for performance improvement, the things I'd benefit most from being benchmarked are:

- REXML
- Mechanize
- WEBrick
- Test::Unit

Yeah, those look like "typical Ruby tasks" to me.

I've pointed out higher-level libraries rather than core because they exercise more of Ruby with fewer data points. Is that the right approach? I realise that one downfall of this perspective is that the library authors may have avoided certain techniques precisely because they are too slow in the current Ruby implementation, so those techniques would be completely sidelined. Continuations are one that instantly spring to mind - I can only find callcc used in one file in the stdlib.

An alternative would be to use rubicon's full run time as the benchmark - that way we know that all the language features are being flexed, but some might get too much weighting.

Thoughts?

A benchmark is little different from an automated unit test. The "assertions" are simply replaced with (hopefully accurate) timings. It's a little more complicate than that, but that's the general idea. Once you have a suite of benchmarks, the interesting thing is that some tunings to the "compiler" or the runtime environment will make tests uniformly better, some uniformly worse and some will induce "tradeoffs". That's where the statistics comes in -- reducing a list of relative performance numbers to a single "figure of merit", usually with some weighting based on known usage patterns.