How to speed up ruby and make it as fast as possible

Hi,

Sorry for such a noob question but I could not find anything on Google discussing this.

I have a server that is dedicated solely for running a single ruby application. Is there a ruby configuration file that I can change to help speed ruby up and take as much resources as it needs? Basically make sure there is no memory limit, processing limit, and give it the highest priority process as possible. Also is there anything else I can do to get the most speed out of ruby?

Thanks for your help.

Thank You,
Ben Johnson
E: bjohnson@contuitive.com

Maybe you can try rubyjit, a ruby just-in-time compiler

http://easter.kuee.kyoto-u.ac.jp/~hiwada/ruby/rubyjit/

Ben Johnson 写道:

···

Hi,

Sorry for such a noob question but I could not find anything on Google discussing this.

I have a server that is dedicated solely for running a single ruby application. Is there a ruby configuration file that I can change to help speed ruby up and take as much resources as it needs? Basically make sure there is no memory limit, processing limit, and give it the highest priority process as possible. Also is there anything else I can do to get the most speed out of ruby?

Thanks for your help.

Thank You,
Ben Johnson
E: bjohnson@contuitive.com

Write it C.

···

--
Posted via http://www.ruby-forum.com/.

Profile your code, see where the bottlenecks are and refine the algorithms. There are no real tricks, it all comes down to good coding that is aware of how the language is best used and how your application is structured to handle the commonest usage.

If this is a critical production thing then buy better hardware, it is invariably cheaper in the long term than trying to squeeze a few percentage points out of your code.

I also rewrite all my performance critical code in C, that is the whole application gets rewritten in C once I have used Ruby to prototype the code.

Asking your question in such a general way is bound to produce more
arguments (as you can see) then actual help. If you post some
information about what your application does, and how it's performing
now, you will probably get more useful answers. You also might want
to start a new thread with a more specific subject.

Chris

···

On 7/13/06, Ben Johnson <bjohnson@contuitive.com> wrote:

Hi,

Sorry for such a noob question but I could not find anything on
Google discussing this.

I have a server that is dedicated solely for running a single ruby
application. Is there a ruby configuration file that I can change to
help speed ruby up and take as much resources as it needs? Basically
make sure there is no memory limit, processing limit, and give it the
highest priority process as possible. Also is there anything else I
can do to get the most speed out of ruby?

Thanks for your help.

Wow! You seem to hit some grumpy folks in the chatter spot. Even after
all that I'm not sure any of them actually read or answered your
question!

Ah well. I'll have a bash.

By Server, do you mean Web Server? ie. If you are doing a CGI script
that is getting hammered heavily you might want to try mod_ruby
  modruby.net

The "take the resources it needs" bit probably has more to do with the
OS than with Ruby.

You didn't say which OS, so I'm going to assume Linuxy / Unixy type OS....

"man setrusage" may help there.

Say "help ulimit" for more info on that, and "man mlock" may be helpful
too.

The Ruby syscall function may be your friend here.

For "Highest priority", you need super user privileges to do that and the
"nice" or "renice" command.

Take care, don't make it too high otherwise it competes with vital
daemons.

I have been accumulating optimization hints and tips at...
   http://wiki.rubygarden.org/Ruby/page/show/RubyOptimization
Please contribute anything useful you find to that.

Here is a "worked example" of doing a syscall (not one you want here,
but at least shows the way...)

···

On Fri, 14 Jul 2006, Ben Johnson wrote:

I have a server that is dedicated solely for running a single ruby application. Is there a ruby configuration file that I can change to help speed ruby up and take as much resources as it needs? Basically make sure there is no memory limit, processing limit, and give it the highest priority process as possible. Also is there anything else I can do to get the most speed out of ruby?

======================================================================
=begin rdoc

This is the kernel statfs64 structure definition only here for
documentation purposes.

__extension__ typedef unsigned long long int __u_quad_t;
__extension__ typedef __u_quad_t __fsblkcnt64_t;
__extension__ typedef __u_quad_t __fsfilcnt64_t;
__extension__ typedef struct { int __val[2]; } __fsid_t;

struct statfs64
   {
     int f_type;
     int f_bsize;
     __fsblkcnt64_t f_blocks;
     __fsblkcnt64_t f_bfree;
     __fsblkcnt64_t f_bavail;
     __fsfilcnt64_t f_files;
     __fsfilcnt64_t f_ffree;
     __fsid_t f_fsid;
     int f_namelen;
     int f_frsize;
     int f_spare[5];
   };

#define __NR_statfs64 268
#define __NR_fstatfs64 269

This is runs a little .c program to verify it all works... (Don't forget
this is all commented out for now..)

tmp = "/tmp/struct.c"
open(tmp, 'w') do |c|
   c.puts "
#define _GNU_SOURCE 1
#include <sys/syscall.h>
#include <sys/statfs.h>

struct statfs64 s;

"
end

puts `gcc -E #{tmp}`

=end

require 'pp'

class StatFs64
   STRUCT_STATFS64 = [
                      [0, 'i', 'f_type'],
                      [0, 'i', 'f_bsize'],
                      [0, 'Q', 'f_blocks'],
                      [0, 'Q', 'f_bfree'],
                      [0, 'Q', 'f_bavail'],
                      [0, 'Q', 'f_files'],
                      [0, 'Q', 'f_ffree'],
                      [0, 'i', 'f_fsid0'],
                      [0, 'i', 'f_fsid1'],
                      [0, 'i', 'f_namelen'],
                      [0, 'i', 'f_frsize'],
                      [0, 'i', 'f_spare0'],
                      [0, 'i', 'f_spare1'],
                      [0, 'i', 'f_spare2'],
                      [0, 'i', 'f_spare3'],
                      [0, 'i', 'f_spare4'],
                     ]

   def syscall_pack( triplets)
     template = ''
     default =
     triplets.each do |value, directive, name|
       default << value
       template += directive
     end
     default.pack( template)
   end

   def syscall_unpack( triplets, string)
     template = ''
     names =
     triplets.each do |value, directive, name|
       names << name
       template += directive
     end

     result = {}
     values = string.unpack( template)
     names.each_with_index do |name,i|
       value = values[i]
       result[name] = value
     end

     result
   end

   def initialize( path)
     string = syscall_pack( STRUCT_STATFS64)
     result = syscall( 268, path, string.size, string)
     raise "Unexpected return value '#{result}'" unless
       result == 0

     @statfs64 = syscall_unpack( STRUCT_STATFS64, string)
   end

   # Returns number of free megabytes available
   def free_megabytes
     (@statfs64['f_bavail'] * @statfs64['f_bsize']) / (1024 * 1024.0)
   end

   # Returns number of free gigabytes available
   def free_gigabytes
     free_megabytes / 1024.0
   end

   attr_reader :statfs64
end

if $0 == __FILE__ then
   require 'test/unit'

   class TC_StatFs < Test::Unit::TestCase
     def test_statfs
       s = StatFs64.new( "/")
       puts "Free #{s.free_megabytes}mb"
       puts "Free #{s.free_gigabytes}gb"#
       puts s.statfs64.keys.sort.collect{|k| "#{k}=#{s.statfs64[k]}" }.join("\n")
     end
   end
end

======================================================================

John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter@tait.co.nz
New Zealand

Carter's Clarification of Murphy's Law.

"Things only ever go right so that they may go more spectacularly wrong later."

From this principle, all of life and physics may be deduced.

Peter,
I found your last paragraph intresting. Can you give us some ballpark
comparisons between a profiled/selectively C extensionized app and a
completely re-written one? I would think it would be almost as
effective to render all of the significant bottlenecks in C and leave
the rest in ruby. Have you seen significant speed improvements by
taking this last step of replacing the whole thing with C?

thanks,
jp

Peter Hickman wrote:

···

Profile your code, see where the bottlenecks are and refine the
algorithms. There are no real tricks, it all comes down to good coding
that is aware of how the language is best used and how your application
is structured to handle the commonest usage.

If this is a critical production thing then buy better hardware, it is
invariably cheaper in the long term than trying to squeeze a few
percentage points out of your code.

I also rewrite all my performance critical code in C, that is the whole
application gets rewritten in C once I have used Ruby to prototype the
code.

--
Posted via http://www.ruby-forum.com/\.

The problem is that there really is no answer for the question. Any answers are specific to one's operating system, and those OS tuning answers don't really have anything specifically to do with Ruby.

Kirk Haines

···

On Sat, 15 Jul 2006, snacktime wrote:

On 7/13/06, Ben Johnson <bjohnson@contuitive.com> wrote:

I have a server that is dedicated solely for running a single ruby
application. Is there a ruby configuration file that I can change to
help speed ruby up and take as much resources as it needs? Basically
make sure there is no memory limit, processing limit, and give it the
highest priority process as possible. Also is there anything else I
can do to get the most speed out of ruby?

Thanks for your help.

Asking your question in such a general way is bound to produce more
arguments (as you can see) then actual help. If you post some
information about what your application does, and how it's performing
now, you will probably get more useful answers. You also might want
to start a new thread with a more specific subject.

And first place for the most useless and un-helpful comment goes to...

···

On Fri, Jul 14, 2006 at 07:55:31 +0900, Reggie Mr wrote:

Write it C.

Given a program written from scratch. Profiling and reworking the algorithms will speed up the run time by about 50% to 80%. Even then rewriting it in C results in a massive improvement.

I had a program that read an image, calculated metrics from the image and wrote the data to a database (for a subsequent search application). A sort of vector space search for images.

The first shot at the code took around 15 minutes per image.
Profiling and reflecting on the code got this down to 2 to 4 minutes per image.
Once the code was worked out I wrote it in C, that came down to 5 seconds an image.

I had nearly 500,000 images to process. So the Ruby version was never a starter for this project but the ease with which I could develop the algorithms and tune the various parameters in Ruby saved me countless days of C development. I no longer develop code in C from scratch, I always start with either Ruby or Perl or Python so that I can test and adjust the design.

I am not too sure what the advantages of moving code into C libraries is as the program I just described was using the same libraries in C that were being used in Ruby (GD and SQLite), unless the bridge between the C libraries and Ruby is really crap - which I don't believe.

My latest project involves a Perl program that writes the C program to do the work.

http://blog.zenspider.com/archives/2005/04/space_vs_time.html

···

On Jul 14, 2006, at 8:24 AM, Jeff Pritchard wrote:

Peter Hickman wrote:

I also rewrite all my performance critical code in C, that is the whole
application gets rewritten in C once I have used Ruby to prototype the
code.

I found your last paragraph intresting. Can you give us some ballpark
comparisons between a profiled/selectively C extensionized app and a
completely re-written one? I would think it would be almost as
effective to render all of the significant bottlenecks in C and leave
the rest in ruby. Have you seen significant speed improvements by
taking this last step of replacing the whole thing with C?

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

There could be any number of things he could do to improve performance
that have everything to do with how he coded it, which would be ruby
specific. Two applications written in the same language to perform
the same function can have very different performance. And not to
pick on the OP, but how the question was asked leads me to believe
that he might not have a lot of programming experience, and thus it's
even more likely that writing his application differently could
increase performance.

Chris

···

On 7/14/06, khaines@enigo.com <khaines@enigo.com> wrote:

On Sat, 15 Jul 2006, snacktime wrote:

> On 7/13/06, Ben Johnson <bjohnson@contuitive.com> wrote:
>>
>> I have a server that is dedicated solely for running a single ruby
>> application. Is there a ruby configuration file that I can change to
>> help speed ruby up and take as much resources as it needs? Basically
>> make sure there is no memory limit, processing limit, and give it the
>> highest priority process as possible. Also is there anything else I
>> can do to get the most speed out of ruby?
>>
>> Thanks for your help.
>
> Asking your question in such a general way is bound to produce more
> arguments (as you can see) then actual help. If you post some
> information about what your application does, and how it's performing
> now, you will probably get more useful answers. You also might want
> to start a new thread with a more specific subject.

The problem is that there really is no answer for the question. Any
answers are specific to one's operating system, and those OS tuning
answers don't really have anything specifically to do with Ruby.

Kirk Haines

And first place for the slowest responce goes to... :wink:

···

-----Original Message-----
From: Phil Jackson [mailto:phil@shellarchive.co.uk]
Sent: Wednesday, July 19, 2006 5:34 AM
To: ruby-talk ML
Subject: Re: How to speed up ruby and make it as fast as possible

On Fri, Jul 14, 2006 at 07:55:31 +0900, Reggie Mr wrote:

> Write it C.

And first place for the most useless and un-helpful comment goes to...

My original post was to get a simple yes or no answer to my questions. I did not expect 30 responses to it. Anyways, it was just a question to see if there was anything I could do.

To correct Chris, I have plenty of programming experience. I am new to ruby and this is my first application I have written in ruby, which is why I was asking if there was a ruby configuration file. Many languages have configuration files and I wasn't sure if ruby had the same.

My application is a very complex application, since it is so complex my company and I chose rails to do this in, because the MVC design pattern seemed to fit best with the problem we were trying to solve. Doing complex applications in rails doesn't seem so complex and we ended up completing the application in 1/4 of the allotted time.

Lastly, the reason I asked this question was because I have a pretty intense algorithm that takes about .7 seconds to complete. .7 seconds is not that long, but with this application every millisecond is precious. Therefore, I simply was not sure if there was something I could do to make it a little faster. That's all.

···

On Jul 14, 2006, at 2:59 PM, snacktime wrote:

On 7/14/06, khaines@enigo.com <khaines@enigo.com> wrote:

On Sat, 15 Jul 2006, snacktime wrote:

> On 7/13/06, Ben Johnson <bjohnson@contuitive.com> wrote:
>>
>> I have a server that is dedicated solely for running a single ruby
>> application. Is there a ruby configuration file that I can change to
>> help speed ruby up and take as much resources as it needs? Basically
>> make sure there is no memory limit, processing limit, and give it the
>> highest priority process as possible. Also is there anything else I
>> can do to get the most speed out of ruby?
>>
>> Thanks for your help.
>
> Asking your question in such a general way is bound to produce more
> arguments (as you can see) then actual help. If you post some
> information about what your application does, and how it's performing
> now, you will probably get more useful answers. You also might want
> to start a new thread with a more specific subject.

The problem is that there really is no answer for the question. Any
answers are specific to one's operating system, and those OS tuning
answers don't really have anything specifically to do with Ruby.

Kirk Haines

There could be any number of things he could do to improve performance
that have everything to do with how he coded it, which would be ruby
specific. Two applications written in the same language to perform
the same function can have very different performance. And not to
pick on the OP, but how the question was asked leads me to believe
that he might not have a lot of programming experience, and thus it's
even more likely that writing his application differently could
increase performance.

Chris

I agree with the people who think the questions surrounding performance
are lacking. If you provide no meaningful context the best advice we
can offer is profile your application, find the bottlenecks, and try to
improve them. These are basic programming skills which do not require
any knowledge of C whatsoever. There are plenty of tools and tutorials
one click away from a Google search to help. Heck, you'll even find
help on this list if you ask the right question. Here's another general
pointer for anyone looking for such things: check out Stephen Kaes'
blog. He has some interesting posts on how to speed up Rails apps (I
believe the OP stated the project was a Rails app). Outside of all the
things mentioned above, there is not much more general advice to be
given out on this subject.

If you are already proficient with C, then there are additional
advanced tweaks you could make to improve the performance of your code.
But no one on this list can answer that question, because we have no
idea what your problem domain is, how fast your current app is, or what
your target performance metric is. Which brings me to my last point.

If you want more specific advice, please provide more specific context.
If you find people on this list short of patience with these questions,
it's because they take this as *understood*. If you show us a few lines
of code and say, "this seems really slow to me" you may get some
helpful pointers. Someone may write a faster version for you or offer
general advice like: use an array as a buffer or some such thing. In
the end, you will not be able to fix your performance problem without
measurement on *your* platform.

Most of this has nothing to do with the OP's initial question. He wants
to shave some time off a particular algorithm. My best advice to him:
post some code or else at least describe the general procedure ex: "I
create a jagged array, loop through it, sort it, flatten it, eliminate
the dupes" or whatever. It's quite difficult for any of us to be
helpful in the absence of data. Other than that, let me wish you the
best of luck with your project.

Lastly, the reason I asked this question was because I have a pretty intense algorithm that takes about .7 seconds to complete. .7 seconds is not that long, but with this application every millisecond is precious. Therefore, I simply was not sure if there was something I could do to make it a little faster. That's all.

Assuming the algorithm itself isn't proprietary, you could
post the code here. Or, submit it as a Ruby Quiz. :slight_smile:

Usually when folks have posted code here asking about ways
to speed it up, there have been a number of helpful responses
from experienced rubyists.

Regards,

Bill

···

From: "Ben Johnson" <bjohnson@contuitive.com>

My original post was to get a simple yes or no answer to my questions. I did not expect 30 responses to it. Anyways, it was just a question to see if there was anything I could do.

To correct Chris, I have plenty of programming experience. I am new to ruby and this is my first application I have written in ruby, which is why I was asking if there was a ruby configuration file. Many languages have configuration files and I wasn't sure if ruby had the same.

For Rails, I think there are some configuration options you can fiddle with - but as you know, twiddling the knobs on the front of the server (so to speak) is less likely to produce a significant gain compared to re-writing the algorithm in a lower-level language, or if the algorithm is wasteful, tuning that. I'd suggest looking at the RailsExpress [1] website - dedicated to tuning Rails apps, and/or posting time critical pieces of code here (if that option is available due to confidentiality restrictions). Many of the people here will have a lot more experience of ruby than yourself (as you mention that this is your first ruby application), and may see a way of squeezing some extra performance out of the code without a C re-write. I discovered when I first started ruby that there are many ways to achieve the same goal, but some of them use a hell of a lot less processing power than the others - an example was on this list just a couple of weeks ago regarding Blocks and Procs [2]. Hope these suggestions help.

Thanks

Kev

[1] - www.railsexpress.de/blog
[2] - http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/199634

···

--
"To be governed is to be watched over, inspected, spied on, directed, legislated..." - Pierre-Joseph Proudhon

I agree with the people who think the questions surrounding performance
are lacking. If you provide no meaningful context the best advice we
can offer is profile your application, find the bottlenecks, and try to
improve them. These are basic programming skills which do not require
any knowledge of C whatsoever. There are plenty of tools and tutorials
one click away from a Google search to help. Heck, you'll even find
help on this list if you ask the right question. Here's another general
pointer for anyone looking for such things: check out Stephen Kaes'
blog. He has some interesting posts on how to speed up Rails apps (I

Would it be possible to get a link to Stephen Kaes' blog? I've tried
searching Google and came up with nothing.

believe the OP stated the project was a Rails app). Outside of all the

···

On 7/20/06, xmlblog (Christian Romney) <xmlblog@gmail.com> wrote:

things mentioned above, there is not much more general advice to be
given out on this subject.

If you are already proficient with C, then there are additional
advanced tweaks you could make to improve the performance of your code.
But no one on this list can answer that question, because we have no
idea what your problem domain is, how fast your current app is, or what
your target performance metric is. Which brings me to my last point.

If you want more specific advice, please provide more specific context.
If you find people on this list short of patience with these questions,
it's because they take this as *understood*. If you show us a few lines
of code and say, "this seems really slow to me" you may get some
helpful pointers. Someone may write a faster version for you or offer
general advice like: use an array as a buffer or some such thing. In
the end, you will not be able to fix your performance problem without
measurement on *your* platform.

Most of this has nothing to do with the OP's initial question. He wants
to shave some time off a particular algorithm. My best advice to him:
post some code or else at least describe the general procedure ex: "I
create a jagged array, loop through it, sort it, flatten it, eliminate
the dupes" or whatever. It's quite difficult for any of us to be
helpful in the absence of data. Other than that, let me wish you the
best of luck with your project.

http://railsexpress.de/

google for stefan kaes or stefen kaes :wink:

J.

···

On 7/20/06, Jeff Avallone <jeff.avallone@gmail.com> wrote:

On 7/20/06, xmlblog (Christian Romney) <xmlblog@gmail.com> wrote:
>
> I agree with the people who think the questions surrounding performance
> are lacking. If you provide no meaningful context the best advice we
> can offer is profile your application, find the bottlenecks, and try to
> improve them. These are basic programming skills which do not require
> any knowledge of C whatsoever. There are plenty of tools and tutorials
> one click away from a Google search to help. Heck, you'll even find
> help on this list if you ask the right question. Here's another general
> pointer for anyone looking for such things: check out Stephen Kaes'
> blog. He has some interesting posts on how to speed up Rails apps (I

Would it be possible to get a link to Stephen Kaes' blog? I've tried
searching Google and came up with nothing.

believe the OP stated the project was a Rails app). Outside of all the
> things mentioned above, there is not much more general advice to be
> given out on this subject.
>
> If you are already proficient with C, then there are additional
> advanced tweaks you could make to improve the performance of your code.
> But no one on this list can answer that question, because we have no
> idea what your problem domain is, how fast your current app is, or what
> your target performance metric is. Which brings me to my last point.
>
> If you want more specific advice, please provide more specific context.
> If you find people on this list short of patience with these questions,
> it's because they take this as *understood*. If you show us a few lines
> of code and say, "this seems really slow to me" you may get some
> helpful pointers. Someone may write a faster version for you or offer
> general advice like: use an array as a buffer or some such thing. In
> the end, you will not be able to fix your performance problem without
> measurement on *your* platform.
>
> Most of this has nothing to do with the OP's initial question. He wants
> to shave some time off a particular algorithm. My best advice to him:
> post some code or else at least describe the general procedure ex: "I
> create a jagged array, loop through it, sort it, flatten it, eliminate
> the dupes" or whatever. It's quite difficult for any of us to be
> helpful in the absence of data. Other than that, let me wish you the
> best of luck with your project.
>

http://railsexpress.de/blog/

···

On 7/20/06, Jeff Avallone <jeff.avallone@gmail.com> wrote:

Would it be possible to get a link to Stephen Kaes' blog? I've tried
searching Google and came up with nothing.