Two Advanced Ruby Performance Questions

First, I am a Ruby newbie but am an experienced developer of highly
scalable applications.

I like the Ruby community because it is very friendly and helpful;
however, I wanted to give my background because the answer to this
question (despite being a newbie to Ruby) won't be your typical 80/20
optimize when you need to, watch your database first, or bandwidth is
the limiting factor type question.

Our application servers on our current application run in highly
optimized Coldfusion (sub 100ms page response times) has about a dozen
application servers attached to it with probably about a dozen more
supporting servers. We have dual load balancers, dual Firewalls, RAIDed
dbs on multiple servers, and a 100 Mbps connection (likely to be
upgraded). We anticipate this new app to run on up to 100 application
servers eventually but this is obviously dependant on app server and
code performance.

I know to optimize when it's important but I am concerned about the
overhead of a framework I want to place on top of Ruby. The RAILS
framework, unfortunately, is too limiting for the application we are
planning (at least the VC parts of MVC) and prefer the control and
performance understanding of having built most of the framework
ourselves anyways.

Thank you in advance for listening. If I can get the right answers to
these questions, we would like to launch possibly one of the more highly
scaled our Ruby web applications. I find Ruby a highly desirable
language to use but I find very little documentation or discussion on
how things work underneath.

QUESTION 1

Is there a way (or does it do this already) for the Classes of the
application to be cached such that it doesn't add performance overhead?
Because it is such a dynamic language, my understanding is that the
classes themselves are created at run-time and DO add overhead before
any object instantiation occurs. My guess is that this would still
happen under YARV too?

In other words, can I create a scope in the web application (say a scope
that lasts the lifetime of the web server) where I can store class
definitions and/or object instances themselves and then use them to
create instances in the page request scope?

Why I want to do this is so I can define many classes without having to
worry about the overhead of having them defined at runtime for every
page request. In this way, if I don't use the classes in a page request,
they won't add any extra to the execution time.

When I write Javascript, I know that there is overhead so I have to keep
my libraries short and sweet. In ColdFusion, I have created a framework
that stores shared classes and object instances in what ColdFusion calls
an "application" scope so that classes and objects are setup only once
at application startup (my framework is a little more complex than this
but you probably get the point). Because of this, I have many libraries
and they are wide and deep. This is very helpful because I can create
many helper libraries without worrying about performance overhead.

I want to know if this is possible in Ruby.

Overall, I'm not really sure what kind of persistence and
non-persistence there is between page requests when Ruby is attached a
web server.

Any thoughts or pointers to resources would be helpful.

QUESTION 2

I've found a lot of documentation on ERB but a lot less on eRuby. All
the documentation I have found on eRuby has it executing from the
command line or through a web server plugin, usually through Apache. Can
eRuby be called from inside Ruby to do parsing? I ask this because eRuby
seems like it would execute faster seeing it is built using C.

Using ERB is straightforward but I'd love to get the performance
benefits of using eRuby if I could; however, my framework would likely
requiring making calls from inside Ruby and not ONLY .rhtml files
directly.

I'm guessing we can use ERB to generate the Ruby code and saving the
generated code to a file and then executing the generated file. This
would improve performance since the parsing step only happens once;
however, I'd still like to know if eRuby can be used this way.

FINAL COMMENTS

Sorry for the monster large post. This is incredibly important for us
and will help us decide if we want to switch to Ruby for our new
application. We have a large amount of good code in ColdFusion but as an
agile company, I can see the benefits of Ruby down the line, especially
after a couple of years. Mostly, I love the clean syntax and the overall
design of the language.

Thanks for your input and I hope (beg) that somebody can help answer
these questions.

···

--
Posted via http://www.ruby-forum.com/.

Sunny Hirai wrote:

First, I am a Ruby newbie but am an experienced developer of highly
scalable applications.

I like the Ruby community because it is very friendly and helpful;
however, I wanted to give my background because the answer to this
question (despite being a newbie to Ruby) won't be your typical 80/20
optimize when you need to, watch your database first, or bandwidth is
the limiting factor type question.

Our application servers on our current application run in highly
optimized Coldfusion (sub 100ms page response times) has about a dozen
application servers attached to it with probably about a dozen more
supporting servers. We have dual load balancers, dual Firewalls, RAIDed
dbs on multiple servers, and a 100 Mbps connection (likely to be
upgraded). We anticipate this new app to run on up to 100 application
servers eventually but this is obviously dependant on app server and
code performance.

I know to optimize when it's important but I am concerned about the
overhead of a framework I want to place on top of Ruby. The RAILS
framework, unfortunately, is too limiting for the application we are
planning (at least the VC parts of MVC) and prefer the control and
performance understanding of having built most of the framework
ourselves anyways.
  

Have you looked at Nitro or IOWA? There are some extensive descriptions of them in Hal Fulton's second edition of "The Ruby Way". Nitro in particular seems to be quite flexible and may do what you want where Rails can't.

[snip]

FINAL COMMENTS

Sorry for the monster large post. This is incredibly important for us
and will help us decide if we want to switch to Ruby for our new
application. We have a large amount of good code in ColdFusion but as an
agile company, I can see the benefits of Ruby down the line, especially
after a couple of years. Mostly, I love the clean syntax and the overall
design of the language.
  

I don't know much about ColdFusion, but it hardly seems to me like it's going away any time soon. I would think that, aside from the "agility" of Ruby, the main motivator for switching from ColdFusion to Ruby would be to reduce software license costs. :slight_smile: In any event, despite your concerns about Ruby's performance, your decision needs to be made on economic grounds and not necessarily on technical ones. After all, you've demonstrated a willingness to throw hardware at your existing scalable ColdFusion applications. I would be more concerned about what would happen when a team highly experienced with ColdFusion and making scalable applications with it suddenly are "asked" to jump head first into Ruby.

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

This post may be stating the obvious, but here goes anyway... I hope I
am not preaching to the choir.

First of all, the most important part of getting high performance is a
performance-oriented software and hardware architecture. Second of all,
at the code level, the selection of appropriate algorithms is crucial.
Finally comes the low-level code tuning.

Given an algorithm in Ruby, and the same algorithm in C, the algorithm
will perform better in C if the Ruby code consists mostly of primitives
(e.g. a += 1, loops with many iterations, conditionals, object creation
and destruction, and so on). If the Ruby code is really just calling
high-level underlying C library code, then there will not be that much
difference.

That being said, my experience with getting serious performance out of
dynamic languages such as Ruby has been to create extensions in C that
do the heavy lifting, and to write code in the dynamic language that
consists mainly of calls to the extensions. The wrong thing to do is to
write algorithms that make heavy use of fine-grained methods in Ruby. As
with any interpreter, the overhead of interpreting the code and calling
the method can be a significant proportion of the entire operation.

I believe that a very common performance mistake is to design using the
wrong level of granularity. This happened a lot with distributed
computing (DCOM, CORBA, Web Services) when people would create a
fine-grained remote method without considering the overhead of executing
it. I formulated a heuristic stating that if the method being called did
not take at least 10-100 times longer than the time needed to actually
call it (e.g. marshaling, network overhead, etc), it was not
coarse-grained enough and should not be a remote method.

I think this rule of thumb (with a reasonable choice of constant
multiplier, maybe 1000x) could apply to dynamic (or interpreted)
languages too. To design a high-performance interpreted application,
partition the application appropriately between native code and
interpreted code. (The trick is deciding what should be native and what
should be Ruby).

In doing so, of course, you lose some (maybe a lot) of the portability
of the application, and probably maintainability, but that often happens
when you are aiming for extreme performance anyway.

This point of view may not sit well with Rubyists who want 100% pure
Ruby solutions, but as with anything in life, there are always
trade-offs. I take a pragmatic point of view, and do what is needed
based on priorities. If I can write something in the pure language, I
do. If that doesn't make the grade, I either use anther technology or
write an extension.

···

--
Posted via http://www.ruby-forum.com/.

Ruby classes are created/loaded once at run-time.

In ActiveRecord, the classes are loaded at the start of the process.
They aren't reloaded after that (unless you want them to be -- like in
"development" mode).

Joe

···

On 11/26/06, Sunny Hirai <sunny@citymax.com> wrote:

Is there a way (or does it do this already) for the Classes of the
application to be cached such that it doesn't add performance overhead?
Because it is such a dynamic language, my understanding is that the
classes themselves are created at run-time and DO add overhead before
any object instantiation occurs. My guess is that this would still
happen under YARV too?

Hello Sunny-

First, I am a Ruby newbie but am an experienced developer of highly
scalable applications.
<snip>

QUESTION 1

Is there a way (or does it do this already) for the Classes of the
application to be cached such that it doesn't add performance overhead?
Because it is such a dynamic language, my understanding is that the
classes themselves are created at run-time and DO add overhead before
any object instantiation occurs. My guess is that this would still
happen under YARV too?

In other words, can I create a scope in the web application (say a scope
that lasts the lifetime of the web server) where I can store class
definitions and/or object instances themselves and then use them to
create instances in the page request scope?

Why I want to do this is so I can define many classes without having to
worry about the overhead of having them defined at runtime for every
page request. In this way, if I don't use the classes in a page request,
they won't add any extra to the execution time.

When I write Javascript, I know that there is overhead so I have to keep
my libraries short and sweet. In ColdFusion, I have created a framework
that stores shared classes and object instances in what ColdFusion calls
an "application" scope so that classes and objects are setup only once
at application startup (my framework is a little more complex than this
but you probably get the point). Because of this, I have many libraries
and they are wide and deep. This is very helpful because I can create
many helper libraries without worrying about performance overhead.

I want to know if this is possible in Ruby.

Yes this is entirely possible in ruby. You can easily pre load all classes at server start time inastead of per request once you are ready for production environment. What I mean is that in a development mode it is very handy to have classes reload for every request. This allows you to make code changes and see them instantly reflected in the browser. But this is only good for when you are developing your applications. When you no longer are making changes to the source code then you can pre load all classes and not reload them per request. This does reduce the overhead for apps in production quite a bit but still can allow you to do dev in an easy way as well.

Overall, I'm not really sure what kind of persistence and
non-persistence there is between page requests when Ruby is attached a
web server.

This is entirely up to you. In something like rails you have the session around for state between requests. But you can also run a drb(distributed ruby) daemon to do longer tasks in an asyncronous way to increase speed. In effect offload any time consuming tasks to a background daemon and let the htp request return right away thru an xmlhttprequest. Then polling to check the status of jobs. These daemons can be avaiable to all your ruby processes running your application code.

The best way to obtain high throughput in ruby web applications is to add more processes behind a http or fcgi proxy. This is how rails and other frameworks scale. You add more processes to the cluster and they share state through the database or other means like memcached or drb.

Any thoughts or pointers to resources would be helpful.

QUESTION 2

I've found a lot of documentation on ERB but a lot less on eRuby. All
the documentation I have found on eRuby has it executing from the
command line or through a web server plugin, usually through Apache. Can
eRuby be called from inside Ruby to do parsing? I ask this because eRuby
seems like it would execute faster seeing it is built using C.

Using ERB is straightforward but I'd love to get the performance
benefits of using eRuby if I could; however, my framework would likely
requiring making calls from inside Ruby and not ONLY .rhtml files
directly.

I'm guessing we can use ERB to generate the Ruby code and saving the
generated code to a file and then executing the generated file. This
would improve performance since the parsing step only happens once;
however, I'd still like to know if eRuby can be used this way.

Ok this you are going to like. There is a erb compatible alternative that is 3 times faster then ERB and 10-15% faster then the C eruby and it is written in pure ruby. Its called erubis:

http://www.kuwata-lab.com/erubis/

I also want to mention a project I am working on. Its called Merb mongrel+erb:

http://merb.devjavu.com/
http://svn.devjavu.com/merb/README

Merb is faster lightweight replacement for ActionPack which is the VC layer for the rails MVC. Merb still uses ActiveRecord for database persistence. But it can also use Og or Mongoose(pure ruby db). It is integrated into mongrel for http serving and has its own controller and view abstraction with sessions filters and erb. It is just a lot smaller and closer to the metal then ActionPack. I wrote it mainly to use in conjusnction with rails applications. To have a small merb app stand in for performance sensative portions of an application.

ActionPack is not thread safe and requires a mutex around the entire dispatch to rails. This can cause problems with file uploads. Because each file upload blocks an entire rails app server for the duration of the upload. This means that if you have numerous users uploading large files all at once, you will need an app server instance for each concurrent upload(!). This was one of the original reasons I made merb. It has its own mime parser and does not use cgi.rb or anything else that makes actionpack non thread safe. So it can process many concurrent file uploads or requests at one time in one multi threaded app server mongrel process. Merb does use a mutex for parts of the request that can be calling out to ActiveRecord code because although ActiveRecord is thread safe, it does not perform better then single threaded mode and does cause some other problems. So all of the header and mime parsing is handled in thread safe sections of the code and only uses a mutex for sections of code that call the database. ActionPack has a mutex around all mime body parsing as well as everything else actionpack does to serve one request.

You mention you would rather build most of your own framework to be closer to the metal. But you may want to look at merb and see if you want to work on it with me. I plan on continuing its development and it is being used in heavy production already. Augmenting rails applications for faster response times and file uploads.

FINAL COMMENTS

Sorry for the monster large post. This is incredibly important for us
and will help us decide if we want to switch to Ruby for our new
application. We have a large amount of good code in ColdFusion but as an
agile company, I can see the benefits of Ruby down the line, especially
after a couple of years. Mostly, I love the clean syntax and the overall
design of the language.

Thanks for your input and I hope (beg) that somebody can help answer
these questions.

I also find that Xen virtualization works very well for scaling ruby applications. Scaling ruby apps usually means adding more application servers and maxing out your database servers. Also caching plays an important role as well. Anything that can be cached to static files or even partial caching or using memcached for expensive sections of code can yield big performance gains. Using a number of Xen virtual machines with a shared filesystem like gfs can make it easy to scale your ruby applications pretty much horizontally. You just end up pushing the persistence into the database, memcached or drb and trying to use the "shared nothing" approach for as many portions of the system as you can.

In an application stack like this adding nodes to the app server cluster is easy and gives you very good scalability up or down. Ruby is really a small part of a technology stack like this. There are lots of other places to optimize performance. We have built a custom Gentoo distribution that is tailored to running ruby application at optimal performance in Xen instances. I hope to release this distro as soon as I get some free time to package it up.

Cheers-

-- Ezra Zygmuntowicz-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

···

On Nov 26, 2006, at 1:10 PM, Sunny Hirai wrote:

[edited slightly]

I want to [...] define many classes without having to
worry about the overhead of having them defined at runtime for every
page request.

This is done using FastCGI, SCGI or similar with persistent ruby processes.

QUESTION 2

I've found a lot of documentation on ERB but a lot less on eRuby. All
the documentation I have found on eRuby has it executing from the
command line or through a web server plugin, usually through Apache. Can
eRuby be called from inside Ruby to do parsing? I ask this because eRuby
seems like it would execute faster seeing it is built using C.

ERB and eRuby both generate ruby code which is then executed. If you cache the generated code it won't make a difference (in the long run) which one you use.

···

On Nov 26, 2006, at 1310 , Sunny Hirai wrote:

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Our application servers on our current application run in highly
optimized Coldfusion (sub 100ms page response times) has about a dozen

It depends a LOT on what your are doing in that 100ms, but in general, once can do a lot in 100ms. Below is an extraction from the logging for a current IOWA based site. Timings are in the middle column; the first number is the amount of time taken to receive the request (which, for the mode of operation of this app, should always be 0.0), and the second is is the amount of time taken for the application to receive the request, generate the requested content, and pass it back up to the web server for delivery to the client. Bytes transmitted in the response is at the end of the line.

2006-11-29 11:18:22 :: (0.0/0.002295) :: /index.html "200 OK" 14276B
2006-11-29 11:18:22 :: (0.0/0.001249) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:25 :: (0.0/0.003757) :: /index.html "200 OK" 14282B
2006-11-29 11:18:25 :: (0.0/0.001513) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:27 :: (0.0/0.002254) :: /index.html "200 OK" 14282B
2006-11-29 11:18:28 :: (0.0/0.001244) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:32 :: (0.0/0.008826) :: /about.html "200 OK" 13434B
2006-11-29 11:18:32 :: (0.0/0.001643) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:33 :: (0.0/0.016653) :: /funds/xxx_plus_fund.html "200 OK" 18798B
2006-11-29 11:18:34 :: (0.0/0.018764) :: /about/management.html "200 OK" 18405B
2006-11-29 11:18:34 :: (0.0/0.001325) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:37 :: (0.0/0.018813) :: /about/investment.html "200 OK" 14089B
2006-11-29 11:18:37 :: (0.0/0.001263) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:40 :: (0.0/0.003241) :: /about.html "200 OK" 13434B
2006-11-29 11:18:40 :: (0.0/0.001491) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:40 :: (0.0/0.003661) :: / "200 OK" 14276B
2006-11-29 11:18:41 :: (0.0/0.002577) :: /about/management.html "200 OK" 18405B
2006-11-29 11:18:41 :: (0.0/0.001258) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:43 :: (0.0/0.001244) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:44 :: (0.0/0.018290) :: /about/sales.html "200 OK" 16254B
2006-11-29 11:18:45 :: (0.0/0.001241) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:47 :: (0.0/0.002186) :: /index.html "200 OK" 14276B
2006-11-29 11:18:47 :: (0.0/0.001641) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:19:20 :: (0.0/0.010662) :: /products.html "200 OK" 13570B
2006-11-29 11:19:20 :: (0.0/0.001643) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:25 :: (0.0/0.004319) :: /products.html "200 OK" 22589B
2006-11-29 11:19:25 :: (0.0/0.001242) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:31 :: (0.0/0.008743) :: /products.html "200 OK" 19099B
2006-11-29 11:19:32 :: (0.0/0.001484) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:37 :: (0.0/0.003661) :: /products.html "200 OK" 25243B
2006-11-29 11:19:37 :: (0.0/0.001220) :: /styles.css "304 Not Modified" 0B

Every one of these requests is dynamic. The pages all involve content pulled from a database, with navigation that is dynamically generated based on db contents, and the products pages all have tables of fund statistics queried from the database.

Now, admittedly, the app is caching information where it can, but these logs demonstrate that it's very reasonable to expect good performance from Ruby.

These are the logs from a single process, and when hammered, will handle as many as 200 requests per second, even with very high concurrency. The web server configuration fails due to high concurrency long before the application has a problem. The server itself is a very mundane, middle of the road 32 bit dual processor Athlon box, and is not dedicated to this one site. It has 60 IOWA based sites/applications on it, currently, in addition to other duties.

Overall, I'm not really sure what kind of persistence and
non-persistence there is between page requests when Ruby is attached a
web server.

This depends on the architecture of the framework. Generally, for frameworks that utilize a persistent process type backend, everything that is time consuming to deal with can persist from request to request.

application. We have a large amount of good code in ColdFusion but as an
agile company, I can see the benefits of Ruby down the line, especially
after a couple of years. Mostly, I love the clean syntax and the overall
design of the language.

Just an aside, but I spent some time converting ColdFusion pages for displaying mutual fund information to a Ruby application (a MUCH earlier version of IOWA) a few years ago, and the end result was both faster and cleaner.

Kirk Haines

···

On Mon, 27 Nov 2006, Sunny Hirai wrote:

Edwin Fine wrote:

Given an algorithm in Ruby, and the same algorithm in C, the algorithm
will perform better in C if the Ruby code consists mostly of primitives
(e.g. a += 1, loops with many iterations, conditionals, object creation
and destruction, and so on). If the Ruby code is really just calling
high-level underlying C library code, then there will not be that much
difference.

True, but the original poster indicated he's looking at a database
backed web application. In that kind of environment, in all likelihood
most of the execution time will be dictated by database speed and
network performance.

Of course you can easily kill performance for those kind of apps by
doing the wrong things too, but the odds that he'll need to resort to C
to get the performance he needs for something like that are minimal,
and in any case with the ease of writing C extensions for Ruby it would
be a severe case of premature optimization.

The original poster is looking in the right direction by looking for
options that minimize the per request overhead (i.e. avoiding parsing
the source and creating the object hierarchy per request). FastCGI for
instance would be a good starting point.

Vidar

Edwin Fine wrote:

This post may be stating the obvious, but here goes anyway... I hope I am not preaching to the choir.

First of all, the most important part of getting high performance is a performance-oriented software and hardware architecture. Second of all, at the code level, the selection of appropriate algorithms is crucial. Finally comes the low-level code tuning.
  

I'm not sure this is at all "stating the obvious". Still, the OP hasn't formally decided to switch from ColdFusion to Ruby, and is digging for low-level details on Ruby in general and web applications in particular. What this tells me is that 1 and 2 are already taken care of. As I noted in my post, I think the economic considerations are more important than the low-level details. How is a team with presumably many person-years of accumulated experience building scalable ColdFusion applications going to react when being asked to learn a whole new language and framework? Is the reduced cost of an open source platform over a commercial one enough of a motivation to put the team through that? I don't think you're preaching to the choir. I think what I'm saying is, "If it ain't broke, don't fix it!" :slight_smile:

In doing so, of course, you lose some (maybe a lot) of the portability of the application, and probably maintainability, but that often happens when you are aiming for extreme performance anyway.
  

In many cases portability is not a requirement. More fundamental requirements are total cost of ownership and usability of the application. Performance figures into both total cost of ownership and application usability.

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

:from => "Ezra Zygmuntowicz [mailto:ezmobius@gmail.com]"

# http://www.kuwata-lab.com/erubis/

···

#
# I also want to mention a project I am working on. Its called Merb
# mongrel+erb:
#
# http://merb.devjavu.com/
# http://svn.devjavu.com/merb/README

wow, this are cool projects.
thanks for the tip/gems, Ezra.
kind regards -botp

I have a few questions(just curious):

1) How many users will there be?
2) Will it be internal or external? (internet or intranet)

···

On 11/27/06, Eric Hodel <drbrain@segment7.net> wrote:

On Nov 26, 2006, at 1310 , Sunny Hirai wrote:

[edited slightly]

> I want to [...] define many classes without having to
> worry about the overhead of having them defined at runtime for every
> page request.

This is done using FastCGI, SCGI or similar with persistent ruby
processes.

> QUESTION 2
>
> I've found a lot of documentation on ERB but a lot less on eRuby. All
> the documentation I have found on eRuby has it executing from the
> command line or through a web server plugin, usually through
> Apache. Can
> eRuby be called from inside Ruby to do parsing? I ask this because
> eRuby
> seems like it would execute faster seeing it is built using C.

ERB and eRuby both generate ruby code which is then executed. If you
cache the generated code it won't make a difference (in the long run)
which one you use.

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Frederick, M. Edward, Ezra, J-Van, Eric

First of all, thanks for ALL of the fantastic replies. I must say, the
Ruby community is simply amazing.

Vrtwo,

Thought I'd address your question first since it might give some context
to the rest of the replies.

We anticipate the system should, at a minimum, support 1 million users
with the ability to scale larger as required. We built a similar
application about 6 years ago and it had around 250,000 users IIRC
before we shut it down (unfortunately, the model was not sustainable
back in 2000 though it is now). Given the growth in the Internet, I
anticipate hitting 1 million users would not be difficult today.

For a bit more background, my company currently has multiple website
builders targeting small businesses. We also have a "website builder
builder" or perhaps we could call it a "meta website builder." These
automatically talk to DNS servers, email servers, other services and we
have partner integrations (as well as marketing partnerships) with many
external companies like Google, eBay, PayPal, Yahoo!, etc. In other
words, the application is pretty complex.

However, this business to business builder is built on standard web
technology. We regularly review all of our competitors (in depth) and we
believe it is currently the best such website builder overall (though it
needs and we are currently working on a serious refresh of the home
pages and a refresher in design tools which are written but not yet
launched); however, it is not Web 2.0 as it was started in a different
era. I believe we actually had the first online website builder on the
Internet.

Our new project will target personal websites as a free service to go
into a MySpace like market but aim at a market a little older than that.
I will say that many of the new ideas that we've coded have never been
implemented on the web yet (that I've seen or heard) and are not trivial
things to implement. We had to write custom graphic manipulation
libraries to get the performance we wanted after having tested about a
dozen available libraries on the Internet and being unsatisfied with
them (it will blow Image Magick away in performance and power for
example). Currently, I have written all the code for the new project and
wrote all the base code for the business-to-business project though it
has been taken over for a couple years now with a team of coders.

Hi Ezra,

Let me say wow. I find there are few people who really understand
scaling issues well and allow me to say that it is fantastic information
you've provided. This is exactly the type of info I was looking for and,
quite frankly, I was worried there was still too little performance work
being done out there. It seems there is some serious work being done
however.

Actually, the file upload issue is something that ColdFusion also has a
problem with. In ColdFusion, the optimal number of maximum threads is a
multiple of the number of CPUs but that doesn't work when 10 threads can
be consumed by 10 concurrent uploads. Our solution would have been to
have a dedicated upload server with a higher maximum thread count just
to handle the uploads. In practice, the upload limit hasn't been
problematic (yet).

I love the idea of taking the upload handling and allowing that to scale
better. It is, frankly, a brilliant idea. I also really like the fact
that you are thread-conscious.

I will be looking at Merb tonight.

As an aside

···

-----
If we decide to go the Ruby road (which I'm feeling a little more
confident with now), I would consider contributing to merb and your
project and am flattered that you asked. That said, allow me to be
completely frank and say that I don't think I would be useful to the
project immediately. I would say (again frankly) that I am a very good
developer and that I think I could program in a way that is natural to
the Ruby style, but still, I have a lack of experience in Ruby. I have,
however, adopted many Ruby style programming constructs into ColdFusion
because I like them so much (things like array iterators) and have
created a pretty nice framework in ColdFusion. I was originally wooed
into Ruby through Prototype which I am very good at coding with however.
-----

I have to say that I'm not quite sure I understand why multiple Xen
instances would provide better performance than a single environment
running multiple Ruby instances. Not saying it isn't true. Just that I
don't understand why this would be.

Thank you for the info on ActionPack and MERB. I'm going to look into
this more.

As suggested, we do use a shared nothing approach in our applications
using the database servers for persistence. Since our db has enough RAM,
it basically acts like DRB.

As you mention, we do a lot of caching but because of the dynamic nature
of our application, we do most of it in the db. We basically do a lot of
front-end processing to get data from a bunch of different places in the
db, process it, then encode it into a single TEXT field, and drop it
into a table we designate for caching. When we need it, we retrieve it
then decode it for use. We currently encode/decode using WDDX but I now
have a simple custom library do it about 100x faster. If we edit any of
the underlying data, the edit action also deletes the related cache
record. Upon next retrieval, if the data isn't found in the cache it is
automatically recreated with the new data from the edits. Makes for a
really simple interface to refresh data and also makes it simple since
the DB is authoritative. It scales quite well.

Thank you Eric, Edward and Joe (J-Van) and Frederick.

The info on persistence and threading was useful. I do find it both cool
and troubling that essentially the same application scope is used for
every request though I understand there seems to be the ability to
execute in Anonymous Modules. Seems a little dangerous but also seems
like you can get some really cool performance tweeks by front loading
processing at Class creation time.

M. Edward,

I would be excited to see JRuby becoming a good implementation of Ruby.
With their team being adopted by Sun, this seems like a strong
possibility. Plus a lot of the integration libraries are written in
Java.

Didn't mean to preach to the choir. Just thought the last comment
suggested that I shouldn't worry so much about performance and just
wanted to explain why our situation may not be the typical Ruby
application and why I was concerned with performance so much.

Thanks for referring me to Zed Shaw. I'm not sure if I'm ready for him
just yet but he seems to be a good guy to talk to.

J-Van

Thanks for the ideas. By scaling with "processes" I'm assuming your
meaning to have multiple complete application scopes instead of one
application scope running concurrent threads? Just wanting to make sure
I understood that correctly.

Eric,

Thanks for the insight into ERB vs eRuby and caching.

Sunny Hirai
CEO, MeZine Inc.

--
Posted via http://www.ruby-forum.com/.

Hey Ezra, this is an awesome e-mail, are you going to blog it? Can I?
I just forwarded it to a client, and I would have sounded much more
intelligent in a phone interview the other day if I'd had this
knowledge at my disposal then. This is the most useful thing I've read
on Rails scaling in a while.

···

--
Giles Bowkett
http://www.gilesgoatboy.org

On 11/27/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:

Hello Sunny-

On Nov 26, 2006, at 1:10 PM, Sunny Hirai wrote:

> First, I am a Ruby newbie but am an experienced developer of highly
> scalable applications.
> <snip>
>
> QUESTION 1
>
> Is there a way (or does it do this already) for the Classes of the
> application to be cached such that it doesn't add performance
> overhead?
> Because it is such a dynamic language, my understanding is that the
> classes themselves are created at run-time and DO add overhead before
> any object instantiation occurs. My guess is that this would still
> happen under YARV too?
>
> In other words, can I create a scope in the web application (say a
> scope
> that lasts the lifetime of the web server) where I can store class
> definitions and/or object instances themselves and then use them to
> create instances in the page request scope?
>
> Why I want to do this is so I can define many classes without
> having to
> worry about the overhead of having them defined at runtime for every
> page request. In this way, if I don't use the classes in a page
> request,
> they won't add any extra to the execution time.
>
> When I write Javascript, I know that there is overhead so I have to
> keep
> my libraries short and sweet. In ColdFusion, I have created a
> framework
> that stores shared classes and object instances in what ColdFusion
> calls
> an "application" scope so that classes and objects are setup only once
> at application startup (my framework is a little more complex than
> this
> but you probably get the point). Because of this, I have many
> libraries
> and they are wide and deep. This is very helpful because I can create
> many helper libraries without worrying about performance overhead.
>
> I want to know if this is possible in Ruby.

Yes this is entirely possible in ruby. You can easily pre load all
classes at server start time inastead of per request once you are
ready for production environment. What I mean is that in a
development mode it is very handy to have classes reload for every
request. This allows you to make code changes and see them instantly
reflected in the browser. But this is only good for when you are
developing your applications. When you no longer are making changes
to the source code then you can pre load all classes and not reload
them per request. This does reduce the overhead for apps in
production quite a bit but still can allow you to do dev in an easy
way as well.
>
> Overall, I'm not really sure what kind of persistence and
> non-persistence there is between page requests when Ruby is attached a
> web server.

This is entirely up to you. In something like rails you have the
session around for state between requests. But you can also run a drb
(distributed ruby) daemon to do longer tasks in an asyncronous way to
increase speed. In effect offload any time consuming tasks to a
background daemon and let the htp request return right away thru an
xmlhttprequest. Then polling to check the status of jobs. These
daemons can be avaiable to all your ruby processes running your
application code.

The best way to obtain high throughput in ruby web applications is to
add more processes behind a http or fcgi proxy. This is how rails and
other frameworks scale. You add more processes to the cluster and
they share state through the database or other means like memcached
or drb.

>
> Any thoughts or pointers to resources would be helpful.
>
> QUESTION 2
>
> I've found a lot of documentation on ERB but a lot less on eRuby. All
> the documentation I have found on eRuby has it executing from the
> command line or through a web server plugin, usually through
> Apache. Can
> eRuby be called from inside Ruby to do parsing? I ask this because
> eRuby
> seems like it would execute faster seeing it is built using C.
>
> Using ERB is straightforward but I'd love to get the performance
> benefits of using eRuby if I could; however, my framework would likely
> requiring making calls from inside Ruby and not ONLY .rhtml files
> directly.
>
> I'm guessing we can use ERB to generate the Ruby code and saving the
> generated code to a file and then executing the generated file. This
> would improve performance since the parsing step only happens once;
> however, I'd still like to know if eRuby can be used this way.

Ok this you are going to like. There is a erb compatible alternative
that is 3 times faster then ERB and 10-15% faster then the C eruby
and it is written in pure ruby. Its called erubis:

http://www.kuwata-lab.com/erubis/

I also want to mention a project I am working on. Its called Merb
mongrel+erb:

http://merb.devjavu.com/
http://svn.devjavu.com/merb/README

Merb is faster lightweight replacement for ActionPack which is the VC
layer for the rails MVC. Merb still uses ActiveRecord for database
persistence. But it can also use Og or Mongoose(pure ruby db). It is
integrated into mongrel for http serving and has its own controller
and view abstraction with sessions filters and erb. It is just a lot
smaller and closer to the metal then ActionPack. I wrote it mainly to
use in conjusnction with rails applications. To have a small merb app
stand in for performance sensative portions of an application.

ActionPack is not thread safe and requires a mutex around the entire
dispatch to rails. This can cause problems with file uploads. Because
each file upload blocks an entire rails app server for the duration
of the upload. This means that if you have numerous users uploading
large files all at once, you will need an app server instance for
each concurrent upload(!). This was one of the original reasons I
made merb. It has its own mime parser and does not use cgi.rb or
anything else that makes actionpack non thread safe. So it can
process many concurrent file uploads or requests at one time in one
multi threaded app server mongrel process. Merb does use a mutex for
parts of the request that can be calling out to ActiveRecord code
because although ActiveRecord is thread safe, it does not perform
better then single threaded mode and does cause some other problems.
So all of the header and mime parsing is handled in thread safe
sections of the code and only uses a mutex for sections of code that
call the database. ActionPack has a mutex around all mime body
parsing as well as everything else actionpack does to serve one request.

You mention you would rather build most of your own framework to be
closer to the metal. But you may want to look at merb and see if you
want to work on it with me. I plan on continuing its development and
it is being used in heavy production already. Augmenting rails
applications for faster response times and file uploads.

>
> FINAL COMMENTS
>
> Sorry for the monster large post. This is incredibly important for us
> and will help us decide if we want to switch to Ruby for our new
> application. We have a large amount of good code in ColdFusion but
> as an
> agile company, I can see the benefits of Ruby down the line,
> especially
> after a couple of years. Mostly, I love the clean syntax and the
> overall
> design of the language.
>
> Thanks for your input and I hope (beg) that somebody can help answer
> these questions.

I also find that Xen virtualization works very well for scaling ruby
applications. Scaling ruby apps usually means adding more application
servers and maxing out your database servers. Also caching plays an
important role as well. Anything that can be cached to static files
or even partial caching or using memcached for expensive sections of
code can yield big performance gains. Using a number of Xen virtual
machines with a shared filesystem like gfs can make it easy to scale
your ruby applications pretty much horizontally. You just end up
pushing the persistence into the database, memcached or drb and
trying to use the "shared nothing" approach for as many portions of
the system as you can.

In an application stack like this adding nodes to the app server
cluster is easy and gives you very good scalability up or down. Ruby
is really a small part of a technology stack like this. There are
lots of other places to optimize performance. We have built a custom
Gentoo distribution that is tailored to running ruby application at
optimal performance in Xen instances. I hope to release this distro
as soon as I get some free time to package it up.

Cheers-

-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

I guess I gave the right answer to the wrong question :slight_smile:

I was really looking at the OP's words below when I posted:

I know to optimize when it's important but I am concerned about the
overhead of a framework I want to place on top of Ruby.

...

eRuby seems like it would execute faster seeing it is built using C.

Vidar Hokstad wrote:

True, but the original poster indicated he's looking at a database
backed web application. In that kind of environment, in all likelihood
most of the execution time will be dictated by database speed and
network performance.

Very likely, but you never know. It all depends on the application. I am
working on a "database-backed web application" that happens to be an
OLAP/data mart application, and in this, response time is dictated by
*everything*, including CPU.

I do agree with what you guys posted, especially that TCO and usability
should be the ultimate decision factors, especially the thought that
reskilling experienced ColdFusion developers in Ruby could be costly.

Still, I would also be interested in knowing what facilities exist in
Ruby app servers that are similar to the Java app server world (like app
scope, session scope, page scope etc.) I don't have any experience in
Ruby app servers. An earlier poster mentioned Nitro - maybe I should
read up on that.

I also remember reading a bit about "continuation" servers, which if I
recall correctly should address some of the OP's performance concerns.

···

--
Posted via http://www.ruby-forum.com/\.

Hi Sunny,

I know you don't intend to use Rails, but there's a
performance-specific blog dealing with Rails that might be of interest
to you:

http://railsexpress.de/blog/

Maybe you can get in touch with those guys and get them to share some
fo their experiences.

Cheers,
Max

Hey Giles-

  Feel free to do what you like with the post. I am writing a lot about this topic for my rails deployment book for the pragprogs. I hope to get it out in beta form soon.

Cheers-
-- Ezra Zygmuntowicz-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

···

On Nov 28, 2006, at 5:16 PM, Giles Bowkett wrote:

Hey Ezra, this is an awesome e-mail, are you going to blog it? Can I?
I just forwarded it to a client, and I would have sounded much more
intelligent in a phone interview the other day if I'd had this
knowledge at my disposal then. This is the most useful thing I've read
on Rails scaling in a while.

-- Giles Bowkett
http://www.gilesgoatboy.org
http://gilesbowkett.blogspot.com

Hi All and thanks for the responses.

Before I go on, I thought I should note that Vidar Hokstad is
understanding my particular problem the best.

Thank you for all the great responses but I would like to mention that I
have a good grasp of how we will probably need to scale our application
and the potential pitfalls. As I mentioned, we have already scaled one
application to a good scale. I am looking for specific answers to the
fill in the holes of my knowledge with Ruby. I am a Ruby newbie, but I
have read three books on Ruby and a couple of them multiple times. I'm
pretty much aware of most of what I want to know about the other
parameters that you have asked me to consider.

However, in no particular order, I'd like to address these issues to
show you that I'm not looking for the "typical" answers as I mentioned
in the OP.

1. This is a NEW application so many won't be switching. I will write
the majority of it to start. It IS expensive for me to switch, but I'm
thinking what each language will cost/save us 2-3 years down the road.
Furthermore, I wrote all of the original code in the current application
and will write all the core code for the new one. In other words, I will
bear the brunt of the complexity which I'm currently determining
whethere it is worht it. None of our developers except one actually knew
ColdFusion from the start and were all taught ColdFusion on the job. All
except two developers (not counting me) were hired less than a year ago
in a 6 person team which means they also had to learn both a language
and a framework. This is because it is difficult to find great
ColdFusion developers. Rather, we hired smart people and taught them a
new language. We will keep many developers with the old app and some
will come over to the new app team. We are hiring two new people who
will likely not know ColdFusion or Ruby anyways. I think ColdFusion is
easier to learn but Ruby has a better syntax and I personally like its
design philosophy better (though I like that of ColdFusion too). I am
actually counting on, in the long term, a stable version of a VM, but am
willing to wait a year to get it and just pack power against it in the
shorter term.

2. As a database backed app, dropping to C is not needed for most of our
application and having it split into two languages is costly from a
development point of view so I'd like to keep the main app in one
language only; however, there are performance oriented portions like a
Photoshop like image processor that I've written in C# and .Net as a web
service. It is Mono compatible but Mono seems to choke randomly. We used
C# because of its strong underlying graphic framework, its Java like
memory protection and still the ability to drop to UNSAFE mode where the
custom filters, merging and effects code need to execute very quickly.
We also have portions of interoperability in Java. Most likely we will
also have a Lucene search engine in Java. All of these operate (or will
operate) as web services.

3. I address the overhead of web services operations constantly. Even
when working within the SAME language, I benchmark overhead and test the
cost of each operation. For example, ColdFusion has a native WDDX (XML
like) conversion format that we used to use to fold multiple fields into
a single field for caching in a database. After testing, I wrote a
custom encoding/decoding format that executes about 1-2 orders of
magnitude faster if I remember correctly. I also weigh the cost of
calling methods. In fact, one of the major reasons for wanting a switch
to Ruby is that the object instantiation cost in ColdFusion is very
high. It takes about 1ms per instantiation. This means that if I need to
instantiate an object for say each row in a 50 row query for output, I
have added a 50ms overhead to our project (in which we are aiming at
under 100ms for total page execution times). And to cover the next
response, YES, I know I shouldn't be doing this in an environment where
object instantiation is expensive so I don't do this. Instead, I have
three ways to instantiate an object which I've created in the framework,
two ways which fake it in a manner that is 2 orders of magnitude faster;
however, the syntax is ugly. It also adds to development time greatly
because I need to decide every time which model I need to use and
sometimes, the best model changes over time. I could use the fastest
method (with the ugliest code) but it just introduces a layer of
complexity and potential for bugs to the code which goes against my
instincts. I constantly weigh app performance against developer
performance. Most likely, however, is that I don't use OOP at all and
inline the code but this results in bad reusability.

Okay, now, that I've (hopefully) convinced you I'm attacking the problem
at an atypical level, back to the regular program. :wink:

I found some of the information I wanted in not the eRuby or ERB pages
but in mod_ruby.

It suggests that ONE instance of Ruby executes to handle all the
threads; however, it doesn't go into too much detail about how this is
handled. So, for example, if I call

require 'somelibrary'

It is actually only included into the code once.

However, it doesn't say anything about where scopes begin and end. The
general feeling I'm getting is that there is very little documentation
on the guts of Ruby and I'd like to learn about them without having to
read the source code which I probably wouldn't understand anyways.

For example, I'd like to know how multiple threads are handled. It
appears that objects in the "global" scope are shared but objects in
normal scopes are not. I found this in the FAQ for mod_ruby here:

http://wiki.modruby.net/en/?FAQ#Why+are+changes+to+my+library+not+reflected+in+the+server%3F

http://wiki.modruby.net/en/?FAQ#How+do+I+keep+an+object+instance+between+invocations+of+a+page%2C+for+example%2C+a+persistent+database+handle%3F

But it is REALLY unclear how this all works. So for example, if I extend
the original Array object, this does NOT persist between requests. But
if I add the extension in a "require"d file, it then does persist but
does NOT reload. I find this contradictory. It probably has something to
do with this statement in the FAQ:

···

---
You can't override classes in your mod_ruby scripts directly. (Instead,
a new class will be defined.) Because mod_ruby scripts are loaded by
Kernel#load(filename, true).

If you have to override existing classes, please do it in a library,
then require it from your mod_ruby scripts.
---

But I'm not really sure how Kernal#load works underneath either.

I'd also like to know how to deal with locking global variables for
transactional use in a multi-threaded environment (e.g. web server) or
if this is even possible.

I still don't know whethere eRuby can be called from within Ruby or if
it has to be called from the command line or through some sort of
adapter.

I feel like Ruby needs a "High Performance Ruby" book. There is one for
MySQL and that is the only reason I had the confidence to make the
decision to switch out of MS SQL Server. Knowing what I'm up against
would help tremendously.

Thanks for your feedback. If anybody knows anything more about the guts
of mod_ruby and/or Ruby, please let me know.

All the best,

Sunny Hirai
CEO, MeZine Inc.

--
Posted via http://www.ruby-forum.com/.

Sweet! I think that book will be EXTREMELY popular by the way. It's
the question you hear asked over and over again and you seem to have
some very coherent and battle-tested answers.

I'm working on a site that involves a lot of file uploads, it sounds
like I should look into using Merb rather than Rails for that part of
the system?

···

--
Giles Bowkett
http://www.gilesgoatboy.org

On 11/28/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:

On Nov 28, 2006, at 5:16 PM, Giles Bowkett wrote:

> Hey Ezra, this is an awesome e-mail, are you going to blog it? Can I?
> I just forwarded it to a client, and I would have sounded much more
> intelligent in a phone interview the other day if I'd had this
> knowledge at my disposal then. This is the most useful thing I've read
> on Rails scaling in a while.
>
> --
> Giles Bowkett
> http://www.gilesgoatboy.org
> http://gilesbowkett.blogspot.com
>

Hey Giles-

        Feel free to do what you like with the post. I am writing a lot
about this topic for my rails deployment book for the pragprogs. I
hope to get it out in beta form soon.

Cheers-
-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

Max Muermann wrote:

I know you don't intend to use Rails, but there's a
performance-specific blog dealing with Rails that might be of interest
to you:

RailsExpress.blog

Maybe you can get in touch with those guys and get them to share some
fo their experiences.

Thanks for the recommendation. The website looks to be some more of what
I'm looking for.

Hi Ezra,

Could you comment on Xen instances yielding better performance then
instances of Ruby on a single server. Is this a mild improvement, a
significant one or for redundancy/easy of deployment?

By the way, the ideas in engineyard are fascinating. Offering a hosting
platform with scalability built in. Very Nice.

I'd be willing to pay for an early beta of your book. :slight_smile:

Sunny Hirai
CEO, MeZine Inc.

···

--
Posted via http://www.ruby-forum.com/\.

Sunny Hirai wrote:

Hi All and thanks for the responses.
  

[snip]

I am actually counting on, in the long term, a stable version of a VM, but am willing to wait a year to get it and just pack power against it in the shorter term.
  

There is one stable implementation at the moment, Ruby 1.8.5 "stable snapshot". It's not a VM -- it's pure C code, plus some Ruby once enough Ruby is built during the installation. A year out, there will be jRuby, built on the Java Virtual Machine and running 1.8.x syntax/semantics, probably YARV with different syntax and semantics ("Ruby 1.9.1"), and possibly something running 1.8.x on the CLR. If you get started building now, my guess is you'll be most likely going down the jRuby path if you want a VM.

[snip]

Okay, now, that I've (hopefully) convinced you I'm attacking the problem at an atypical level, back to the regular program. :wink:
  

I wouldn't call it atypical ... but you are definitely "challenging" the language. :slight_smile:

I found some of the information I wanted in not the eRuby or ERB pages but in mod_ruby.

It suggests that ONE instance of Ruby executes to handle all the threads; however, it doesn't go into too much detail about how this is handled. So, for example, if I call

require 'somelibrary'

It is actually only included into the code once.

However, it doesn't say anything about where scopes begin and end. The general feeling I'm getting is that there is very little documentation on the guts of Ruby and I'd like to learn about them without having to read the source code which I probably wouldn't understand anyways.
  

The best description of scoping in the Ruby language I've seen is in David A. Black's "Ruby for Rails". I've never used "mod_ruby", so I can't be of much help with it. For that matter, I stay as far away from Apache as I possibly can -- life is too short to know *everything* and so I've chosen to learn about Markov processes instead of Apache config files. :slight_smile:

As far as the "guts of Ruby" and reading the source code, it's actually quite readable once you've been through the Pickaxe book, understand the layout of objects, classes and pointers and know the material on interfacing C with Ruby. Ruby has a lot fewer ugly hacks than, say, Perl, or a modern finite element structural analysis code.

I feel like Ruby needs a "High Performance Ruby" book. There is one for MySQL and that is the only reason I had the confidence to make the decision to switch out of MS SQL Server. Knowing what I'm up against would help tremendously.
  

I think the jRuby team is writing a high-performance *Ruby* -- perhaps that would be better than a book.

One last comment -- from what the folks on this list tell me, programming in Ruby is supposed to be more fun than programming in other languages. Try not to get so bogged down in understanding how it all works that it stops being fun.

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.