# High ActiveRecord CPU Utilization **URL:** \ **Category:** ruby-talk\ **Created:** [7 March 2007 03:35 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939 "2007-03-07T03:35:08Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [7 March 2007 03:35 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/1 "2007-03-07T03:35:08Z")
When running a test that primarily involves loading up a few MySQL tables with ActiveRecord objects, I was surprised to see the Ruby CPU utilization at 93% and the MySQL CPU utilization at 7%. I would expect this workload to be heavier on MySQL than that. I would think inserts (particularly with updating several foreign key indices) would tax the database more than Ruby. Has this been other folks' experience? Is running in the test environment incredibly different than production with respect to CPU utilization? I suppose my next step is to run in production to see what kind of results I get. I'm running the test from the root of my Rails project via: ruby test/unit/foo.rb Here's part of the profiler output:   % time name 7.96 ActiveRecord::ConnectionAdapters::Quoting.quote 5.61 ActiveRecord::Base#read\_attribute 5.15 ActiveRecord::Base#column\_for\_attribute 4.25 ActiveRecord::Base#connection 3.74 Hash#[] 3.58 Array#each 3.30 ActiveRecord::ConnectionAdapters::MysqlAdapter#quote 3.16 ActiveRecord::ConnectionAdapters::Column#type\_cast 2.84 Module#=== 2.65 ActiveRecord::Base#clone\_attribute\_value 2.29 ActiveRecord::Base#write\_attribute 2.24 Kernel.class 2.22 Hash#each 2.08 String#to\_s 2.03 ActiveRecord::Base#quote\_value 1.59 Kernel.send 1.55 Array#include? 1.52 Kernel.== 1.48 ActiveRecord::Base#unserializable\_attribute? 1.39 Class#read\_inheritable\_attribute 1.34 Kernel.clone 1.29 ActiveRecord::Callbacks.notify 1.15 ActiveRecord::Base#method\_missing 1.08 ActiveRecord::Base#columns\_hash 1.08 ActiveRecord::Base#respond\_to? 1.08 ActiveRecord::Callbacks.callback 0.99 Kernel.eval 0.95 Symbol#=== 0.90 Observable.notify\_observers 0.88 ActiveRecord::ConnectionAdapters::Column#text? 0.85 Hash#[]= 0.85 Class#inheritable\_attributes 0.83 Kernel.kind\_of? 0.81 ActiveRecord::Base#convert\_number\_column\_value ... ---
**Author:** ![Jeremy\_Kemper1](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@Jeremy\_Kemper1](https://rubytalk.org/u/Jeremy_Kemper1)\ **Post date:** [7 March 2007 04:19 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/2 "2007-03-07T04:19:15Z")
Hey Brian, could you start this conversation on the Rails list? It's a pretty narrow topic. (In short: yes. Record instantiation and save are slow relative to the database. In the big picture, page rendering is typically slower still.) jeremy > **···** > > On 3/6/07, Brian Adkins \ wrote: > > > When running a test that primarily involves loading up a few MySQL > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > this workload to be heavier on MySQL than that. > > > > I would think inserts (particularly with updating several foreign key > > indices) would tax the database more than Ruby. > > > > Has this been other folks' experience? Is running in the test > > environment incredibly different than production with respect to CPU > > utilization? I suppose my next step is to run in production to see what > > kind of results I get. ---
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [7 March 2007 04:20 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/3 "2007-03-07T04:20:05Z")
Brian Adkins wrote: > When running a test that primarily involves loading up a few MySQL tables with ActiveRecord objects, I was surprised to see the Ruby CPU utilization at 93% and the MySQL CPU utilization at 7%. I would expect this workload to be heavier on MySQL than that. > [...] I just moved my test code into a controller and ran it via: mongrel\_rails start -e production Similar CPU characteristics except that Mongrel wasn't able to fully utilize my dual core CPU (I suppose because of the serialization of Rails code due to lack of thread safetyness). So the unit test (1093 records -\> table1, 1093 records -\> table2, 1 record -\> table3) took 5.5 seconds to complete and the identical test in a controller with Mongrel in production mode took 27.4 seconds! Yeah, I know I can have a cluster of Mongrel processes, and that's how I run for real, but I'm still a little bummed with these results ☹ I've switched my company's development from 100% Java to 100% Ruby, and I still believe that was a good decision because of productivity gains and joy, but I do miss some of the runtime performance of Java and the ease with which I could spin up a thread to do some background process. I'm glad BackgrounDRB has been provided, but it's not quite the same. Hopefully future versions of Ruby/Rails will provide some more runtime performance and concurrency - I'd be glad if I could just fork in Rails without trouble, but I don't think that's the case. For now, I don't have more customers than a Core 2 Duo can handle, so it's not exactly on the critical path for me yet 🙂 In fact, I'm glad MySQL isn't on the critical path because overcoming that seems much more difficult than having a bunch of Apache/Mongrel processes running. Brian ---
**Author:** ![Kirk\_Haines](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@Kirk\_Haines](https://rubytalk.org/u/Kirk_Haines)\ **Post date:** [7 March 2007 11:24 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/4 "2007-03-07T11:24:34Z")
No, that isn't surprising. Any ORM trades CPU utilization outside of the database for convenience in the data representation. AR is fairly heavyweight in that regard, so it's doing a lot of work to give you the API that it does. Latency to a database can be a significant bottleneck to some applications, but relatively speaking, CPU utilization by the db will usually be a small fraction of the CPU utilization of the ORM using application that is talking with the db. Kirk Haines > **···** > > On Wed, 7 Mar 2007, Brian Adkins wrote: > > > When running a test that primarily involves loading up a few MySQL tables with ActiveRecord objects, I was surprised to see the Ruby CPU utilization at 93% and the MySQL CPU utilization at 7%. I would expect this workload to be heavier on MySQL than that. > > > > I would think inserts (particularly with updating several foreign key indices) would tax the database more than Ruby. > > > > Has this been other folks' experience? Is running in the test environment incredibly different than production with respect to CPU utilization? I suppose my next step is to run in production to see what kind of results I get. ---
**Author:** ![Zach\_Dennis](https://avatars.discourse-cdn.com/v4/letter/z/3d9bf3/32.png) [@Zach\_Dennis](https://rubytalk.org/u/Zach_Dennis)\ **Post date:** [8 March 2007 01:40 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/5 "2007-03-08T01:40:11Z")
> When running a test that primarily involves loading up a few MySQL > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > this workload to be heavier on MySQL than that. What is your script doing? Can you post it? > Has this been other folks' experience? Is running in the test > environment incredibly different than production with respect to CPU > utilization? I think depends on more of what you're doing and how you're doing it. I've seen CPU and memory issues with AR before, but these have all been fixed by understanding how and when to do things in AR (and in ruby). This also depends on the hardware differences between your development, test and production environments. Disk speed, memory and CPU(s) can have alot to do with the change between the environments. > I suppose my next step is to run in production to see what > kind of results I get. > I'm running the test from the root of my Rails project via: What your program is doing and how it is doing it will give more insight into your issue then just profiler output. Have you isolated the problem to a particular block of code? Zach > **···** > > On Mar 6, 10:30 pm, Brian Adkins \ wrote: ---
**Author:** ![Sam\_Smoot](https://avatars.discourse-cdn.com/v4/letter/s/f17d59/32.png) [@Sam\_Smoot](https://rubytalk.org/u/Sam_Smoot)\ **Post date:** [7 March 2007 06:05 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/6 "2007-03-07T06:05:06Z")
Ruby is a slowish language right now, but this isn't really Ruby's fault. Rails is just incredibly slow, and the problem only seems to be getting worse. You could spend some time trying to speed it up, but there's some really broad design decisions that make that pretty difficult. If you can live with the performance, then I guess I'd just do so with the expectation that a future release will improve performance. If you can't, you might try looking into some of the alternatives? It's a tough choice... > **···** > > On Mar 6, 10:18 pm, Brian Adkins \ wrote: > > > Brian Adkins wrote: > > \> When running a test that primarily involves loading up a few MySQL > > \> tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > \> utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > \> this workload to be heavier on MySQL than that. > > \> [...] > > > > I just moved my test code into a controller and ran it via: > > > > mongrel\_rails start -e production > > > > Similar CPU characteristics except that Mongrel wasn't able to fully > > utilize my dual core CPU (I suppose because of the serialization of > > Rails code due to lack of thread safetyness). > > > > So the unit test (1093 records -\> table1, 1093 records -\> table2, 1 > > record -\> table3) took 5.5 seconds to complete and the identical test in > > a controller with Mongrel in production mode took 27.4 seconds! > > > > Yeah, I know I can have a cluster of Mongrel processes, and that's how I > > run for real, but I'm still a little bummed with these results ☹ > > > > I've switched my company's development from 100% Java to 100% Ruby, and > > I still believe that was a good decision because of productivity gains > > and joy, but I do miss some of the runtime performance of Java and the > > ease with which I could spin up a thread to do some background process. > > I'm glad BackgrounDRB has been provided, but it's not quite the same. > > > > Hopefully future versions of Ruby/Rails will provide some more runtime > > performance and concurrency - I'd be glad if I could just fork in Rails > > without trouble, but I don't think that's the case. > > > > For now, I don't have more customers than a Core 2 Duo can handle, so > > it's not exactly on the critical path for me yet 🙂 In fact, I'm glad > > MySQL isn't on the critical path because overcoming that seems much more > > difficult than having a bunch of Apache/Mongrel processes running. > > > > Brian ---
**Author:** ![Ben\_Bleything1](https://avatars.discourse-cdn.com/v4/letter/b/3e96dc/32.png) [@Ben\_Bleything1](https://rubytalk.org/u/Ben_Bleything1)\ **Post date:** [7 March 2007 15:20 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/7 "2007-03-07T15:20:51Z")
For what it's worth, I'm using AR extensively outside rails and would have missed this had it started on the Rails list. I like it when AR discussions happen here 😉 Ben > **···** > > On Wed, Mar 07, 2007, Jeremy Kemper wrote: > > > Hey Brian, could you start this conversation on the Rails list? It's a > > pretty narrow topic. ---
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [7 March 2007 16:35 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/8 "2007-03-07T16:35:05Z")
khaines@enigo.com wrote: > > When running a test that primarily involves loading up a few MySQL tables with ActiveRecord objects, I was surprised to see the Ruby CPU utilization at 93% and the MySQL CPU utilization at 7%. I would expect this workload to be heavier on MySQL than that. > > > > I would think inserts (particularly with updating several foreign key indices) would tax the database more than Ruby. > > > > Has this been other folks' experience? Is running in the test environment incredibly different than production with respect to CPU utilization? I suppose my next step is to run in production to see what kind of results I get. > > No, that isn't surprising. Any ORM trades CPU utilization outside of the database for convenience in the data representation. AR is fairly heavyweight in that regard, so it's doing a lot of work to give you the API that it does. Latency to a database can be a significant bottleneck to some applications, but relatively speaking, CPU utilization by the db will usually be a small fraction of the CPU utilization of the ORM using application that is talking with the db. I agree about ORMs trading CPU for convenience, but I've used other ORMs (such as Hibernate with Java), and I don't recall the CPU ratio to be quite so high. In fact, my prior experience has always been that a piece of code such as I'm using would be database bound, and not CPU bound. I mean, seriously, all the code is doing is a bunch of database inserts! Also, since I'm not using stored procedures with Rails, MySQL has to work harder to parse the statements, so if stored procedures were used, the Rails/MySQL CPU ratio would be even higher than 93/7. Frankly, it's not worth my time to whip up a Java/Hibernate example for comparison, but I'm pretty darn curious now - both at what the CPU ratio would be as well as the time for completion. I'm willing to live with it, but that doesn't mean I have to like it 🙂 > **···** > > > On Wed, 7 Mar 2007, Brian Adkins wrote: > > > Kirk Haines ---
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [8 March 2007 04:45 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/9 "2007-03-08T04:45:08Z")
zdennis wrote: > > When running a test that primarily involves loading up a few MySQL > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > this workload to be heavier on MySQL than that. > > What is your script doing? Can you post it? I created a smaller test that I could post that exhibits the same characteristics: class PerfTestController \< ApplicationController    def index      t1 = Time.now      3000.times do        member = Member.new        member.first\_name = 'Fred'        member.last\_name = 'Flintstone'        member.address1 = '123 High St.'        member.city = 'Reykjavik'        member.state = 'Michigan'        member.email = 'fred@flintstone.com'        member.save!      end      t2 = Time.now      puts "Time elapsed = #{t2-t1}"    end end That took 35.7 seconds (84 inserts per second) on a dual core 2 GHz AMD Opteron. It pegged Mongrel and MySQL didn't break a sweat. I just ran another test with a short ruby program inserting records directly using the mysql gem and it only took 1.6 seconds (1,875 inserts per second!), and the CPU utilization was as it should be - the MySQL CPU was ten times as much as Ruby. So it definitely appears that Rails/ActiveRecord is about 22 times as slow than a straight Ruby program - wow! This result makes me feel much better since the performance of Ruby seems fine. The fact that Rails/ActiveRecord is way slow isn't hurting me yet, and there is hope it can be sped up since it doesn't appear to be an inherent problem with Ruby. Here's the schema for Member: create table members (    id int not null auto\_increment,    created\_at datetime not null,    updated\_at datetime not null,    first\_name varchar(30) null,    last\_name varchar(30) null,    address1 varchar(50) null,    address2 varchar(50) null,    city varchar(30) null,    state varchar(5) null,    email varchar(100) null,    home\_phone varchar(25) null,    primary key(id) ) engine=InnoDB; > > Has this been other folks' experience? Is running in the test > > environment incredibly different than production with respect to CPU > > utilization? > > I think depends on more of what you're doing and how you're doing it. > I've seen CPU and memory issues with AR before, but these have all > been fixed by understanding how and when to do things in AR (and in > ruby). This also depends on the hardware differences between your > development, test and production environments. Disk speed, memory and > CPU(s) can have alot to do with the change between the environments. > > > I suppose my next step is to run in production to see what > > kind of results I get. > > > I'm running the test from the root of my Rails project via: > > What your program is doing and how it is doing it will give more > insight into your issue then just profiler output. Have you isolated > the problem to a particular block of code? I respectfully disagree. It was the profiler output that showed me where the time was being spent. I truncated the profiler output before it even got to my code - it was all Rails code. > **···** > > > On Mar 6, 10:30 pm, Brian Adkins \ wrote: > > > Zach ---
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [7 March 2007 08:40 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/10 "2007-03-07T08:40:06Z")
Sam Smoot wrote: > > Brian Adkins wrote: > > > > > When running a test that primarily involves loading up a few MySQL > > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > > this workload to be heavier on MySQL than that. > > > [...] > > > > I just moved my test code into a controller and ran it via: > > > > mongrel\_rails start -e production > > > > Similar CPU characteristics except that Mongrel wasn't able to fully > > utilize my dual core CPU (I suppose because of the serialization of > > Rails code due to lack of thread safetyness). > > > > So the unit test (1093 records -\> table1, 1093 records -\> table2, 1 > > record -\> table3) took 5.5 seconds to complete and the identical test in > > a controller with Mongrel in production mode took 27.4 seconds! > > > > Yeah, I know I can have a cluster of Mongrel processes, and that's how I > > run for real, but I'm still a little bummed with these results ☹ > > > > I've switched my company's development from 100% Java to 100% Ruby, and > > I still believe that was a good decision because of productivity gains > > and joy, but I do miss some of the runtime performance of Java and the > > ease with which I could spin up a thread to do some background process. > > I'm glad BackgrounDRB has been provided, but it's not quite the same. > > > > Hopefully future versions of Ruby/Rails will provide some more runtime > > performance and concurrency - I'd be glad if I could just fork in Rails > > without trouble, but I don't think that's the case. > > > > For now, I don't have more customers than a Core 2 Duo can handle, so > > it's not exactly on the critical path for me yet 🙂 In fact, I'm glad > > MySQL isn't on the critical path because overcoming that seems much more > > difficult than having a bunch of Apache/Mongrel processes running. > > > > Brian > > Ruby is a slowish language right now, but this isn't really Ruby's > fault. Yeah, I remember when I first saw the computer language shootout stats! I was bummed that my new favorite language performed so poorly, but then I recalled the early days of Java (I started with 1.02) and gained some perspective. > Rails is just incredibly slow, and the problem only seems to be > getting worse. You could spend some time trying to speed it up, but > there's some really broad design decisions that make that pretty > difficult. If you can live with the performance, then I guess I'd just > do so with the expectation that a future release will improve > performance. I'm pretty sure I can live with the performance - I think a few fast web server machines in front of a fast MySQL machine will do fine, and I'm not close to needing that yet. > If you can't, you might try looking into some of the > alternatives? It's a tough choice... When I switched from Java to Ruby, I knew I was giving up some runtime performance and gaining much in other areas. Then I discovered Lisp and realized, "wow, it's powerful \*and\* fast" 🙂 However, I think it would take too much work for me to get a Lisp environment to the point of being as productive for me as Ruby on Rails is currently (reminds me of Python web dev several years ago, maybe older), and I really do enjoy programming in Ruby. Python is a fair amount quicker than Ruby, but I like Ruby better, and the speed difference doesn't appear to be huge - I did get used to the white space, but I still don't like it in principle. It's interesting that I learned Python first, but at the time (2 to 3 yrs ago), I didn't feel the web frameworks were ready, so I jumped into Ruby via Rails (common story) and then discovered that I like the feel of the language better - it has some warts, but it's still a blast to program in. The Python frameworks seem to have progressed significantly since then. I just took a look at Smalltalk, but despite Seaside's success, I don't think it's quite ready (plus I just got away from an IDE heavy environment with Eclipse), and despite the super duper IDE capabilities, I don't feel it warrants the learning effort for me now - maybe later. So all in all, I'm pretty darn happy with Ruby/Rails at present. I have been on a language research blitz recently though (many, many hours) - I think it's motivated by Ruby's strengths and not its weaknesses. The logic goes something like this, "I was surprised about how much better Ruby is than Java, so I wonder if I could make that kind of jump again" - I guess I'm just greedy, not to mention susceptible to "the grass is always greener on the other side of the fence" I must say that everything I've read about Paul Graham's Arc indicates I would be very pleased with it. No idea when it will be completed, but I'd say it has a great shot at getting some traction. If there's an alternative I haven't mentioned that you feel is actually viable as a contender, feel free to pass it on. Brian > **···** > > > On Mar 6, 10:18 pm, Brian Adkins \ wrote: ---
**Author:** ![Jeremy\_Kemper](https://avatars.discourse-cdn.com/v4/letter/j/8e8cbc/32.png) [@Jeremy\_Kemper](https://rubytalk.org/u/Jeremy_Kemper)\ **Post date:** [8 March 2007 06:45 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/11 "2007-03-08T06:45:04Z")
Hi Brian, I wrapped this up in a simple script that anyone with MySQL or SQLite and the AR gem can run. It benchmarks AR create vs using the db connection directly. See attached. Excerpted results on a new MacBook Pro: user system total real raw quoted 0.460000 0.000000 0.460000 ( 0.480184) create 2.760000 0.080000 2.840000 ( 3.225227) (Nearly 7 times slower.) I haven’t tried profiling the methods yet. In my experience with typical Rails apps, you’ll hit a wall with ERB template rendering much sooner than with Active Record creation. This is an interesting pursuit nonetheless – I’m interested to see what you all come up with. Best regards, jeremy [ar\_bench.rb](https://rubytalk.org/uploads/short-url/hSaTh3oAWOh53hZ8RVZJRC61AUZ.rb) (2.84 KB) > **···** > > On 3/7/07, Brian Adkins [lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > > zdennis wrote: > > > > > On Mar 6, 10:30 pm, Brian Adkins [lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > > > > > > When running a test that primarily involves loading up a few MySQL > > > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > > > this workload to be heavier on MySQL than that. > > > > > > What is your script doing? Can you post it? > > > > I created a smaller test that I could post that exhibits the same > > characteristics: > > > > class PerfTestController \< ApplicationController > > def index > > t1 = Time.now > > 3000.times do > > member = Member.new > > member.first\_name = ‘Fred’ > > member.last\_name = ‘Flintstone’ > > member.address1 = ‘123 High St.’ > > member.city = ‘Reykjavik’ > > member.state = ‘Michigan’ > > member.email = ‘fred-Dsuv1UYZSSTz1n+OaKNE4w@public.gmane.org’ > > member.save! > > end > > t2 = Time.now > > puts “Time elapsed = #{t2-t1}” > > end > > end > > > > That took 35.7 seconds (84 inserts per second) on a dual core 2 GHz AMD > > Opteron. It pegged Mongrel and MySQL didn’t break a sweat. > > > > I just ran another test with a short ruby program inserting records > > directly using the mysql gem and it only took 1.6 seconds (1,875 inserts > > per second!), and the CPU utilization was as it should be - the MySQL > > CPU was ten times as much as Ruby. So it definitely appears that > > Rails/ActiveRecord is about 22 times as slow than a straight Ruby > > program - wow! > > > > This result makes me feel much better since the performance of Ruby > > seems fine. The fact that Rails/ActiveRecord is way slow isn’t hurting > > me yet, and there is hope it can be sped up since it doesn’t appear to > > be an inherent problem with Ruby. > > > > Here’s the schema for Member: > > > > create table members ( > > id int not null auto\_increment, > > created\_at datetime not null, > > updated\_at datetime not null, > > first\_name varchar(30) null, > > last\_name varchar(30) null, > > address1 varchar(50) null, > > address2 varchar(50) null, > > city varchar(30) null, > > state varchar(5) null, > > email varchar(100) null, > > home\_phone varchar(25) null, > > primary key(id) > > ) engine=InnoDB; > > –~–~---------~–~----~------------~-------~–~----~ > You received this message because you are subscribed to the Google Groups “Ruby on Rails: Talk” group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > For more options, visit this group at [http://groups.google.com/group/rubyonrails-talk?hl=en](http://groups.google.com/group/rubyonrails-talk?hl=en) > -~----------~----~----~----~------~----~------~–~— ---
**Author:** ![Brian\_Adkins1](https://avatars.discourse-cdn.com/v4/letter/b/258eb7/32.png) [@Brian\_Adkins1](https://rubytalk.org/u/Brian_Adkins1)\ **Post date:** [7 March 2007 16:45 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/12 "2007-03-07T16:45:07Z")
Ben Bleything wrote: > > Hey Brian, could you start this conversation on the Rails list? It's a > > pretty narrow topic. > > For what it's worth, I'm using AR extensively outside rails and would > have missed this had it started on the Rails list. I like it when AR > discussions happen here 😉 > > Ben I never saw Jeremy's message ☹ I don't know if the problem is with my ISP (Bellsouth DSL) or my mail reader (Thunderbird), but this isn't the first time that I've missed a relevant message. It makes me wonder how many other posts aren't getting through. Anyway, Jeremy, I actually did search for a rails newsgroup and Thunderbird didn't show any newsgroups with rails in the name except for (msn.onstage.motorsite.offroad.favtrails) 🙂 So I did try, then I searched this newsgroup and noticed a fair amount of Rails related posts, so I thought it would be ok to post it here. Also, in this case, even though Rails may be slow, I do think the speed of Ruby is an important factor in my results. If you can point me to the Rails newsgroup, I'll try subscribing even if it doesn't show in Thunderbird. If you're referring to a mailing list instead, that is less desirable to me, but I could make use of it for Rails posts that are less relevant to Ruby. Brian > **···** > > > On Wed, Mar 07, 2007, Jeremy Kemper wrote: ---
**Author:** ![Sam\_Smoot](https://avatars.discourse-cdn.com/v4/letter/s/f17d59/32.png) [@Sam\_Smoot](https://rubytalk.org/u/Sam_Smoot)\ **Post date:** [7 March 2007 17:06 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/13 "2007-03-07T17:06:32Z")
I'd definitely echo the perception that AR is unreasonably taxing on the CPU. Because it is. Performance just hasn't been a focus for it. Load up 1,000 AR records with 10 attributes and you've got 10,000 strings to represent the column names. That's just one example. I'd expect Og to be faster, but that's just a guess since I haven't actually benchmarked it. I wasn't trying to imply looking at other languages, just maybe other techniques. As I understand it Og+Rails has some issues right now because ActiveSupport and Facets step on each other (that's just what I gather from the nitro-general list). But I imagine contributing to resolve the issues between Og & Rails probably isn't as great an undertaking as making AR faster. Depending on Og's performance that may or may not be worthwhile I suppose. One other thing to consider though: MySQL is the fastest mainstream database I know of by a large margin, especially concerning INSERT and UPDATE performance. So if you were using Oracle or MSSQL before, that would skew the results a bit more. > **···** > > On Mar 7, 10:31 am, Brian Adkins \ wrote: > > > khai...@enigo.com wrote: > > \> On Wed, 7 Mar 2007, Brian Adkins wrote: > > > > \>\> When running a test that primarily involves loading up a few MySQL > > \>\> tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > \>\> utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > \>\> this workload to be heavier on MySQL than that. > > > > \>\> I would think inserts (particularly with updating several foreign key > > \>\> indices) would tax the database more than Ruby. > > > > \>\> Has this been other folks' experience? Is running in the test > > \>\> environment incredibly different than production with respect to CPU > > \>\> utilization? I suppose my next step is to run in production to see > > \>\> what kind of results I get. > > > > \> No, that isn't surprising. Any ORM trades CPU utilization outside of > > \> the database for convenience in the data representation. AR is fairly > > \> heavyweight in that regard, so it's doing a lot of work to give you the > > \> API that it does. Latency to a database can be a significant bottleneck > > \> to some applications, but relatively speaking, CPU utilization by the db > > \> will usually be a small fraction of the CPU utilization of the ORM using > > \> application that is talking with the db. > > > > I agree about ORMs trading CPU for convenience, but I've used other ORMs > > (such as Hibernate with Java), and I don't recall the CPU ratio to be > > quite so high. In fact, my prior experience has always been that a piece > > of code such as I'm using would be database bound, and not CPU bound. I > > mean, seriously, all the code is doing is a bunch of database inserts! > > > > Also, since I'm not using stored procedures with Rails, MySQL has to > > work harder to parse the statements, so if stored procedures were used, > > the Rails/MySQL CPU ratio would be even higher than 93/7. > > > > Frankly, it's not worth my time to whip up a Java/Hibernate example for > > comparison, but I'm pretty darn curious now - both at what the CPU ratio > > would be as well as the time for completion. > > > > I'm willing to live with it, but that doesn't mean I have to like it 🙂 > > > > \> Kirk Haines ---
**Author:** ![Rob\_Sanheim](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@Rob\_Sanheim](https://rubytalk.org/u/Rob_Sanheim)\ **Post date:** [8 March 2007 14:50 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/14 "2007-03-08T14:50:24Z")
Jeremy Are there, or could there be, performance tests like this added to the Rails test suite? It would be great to be able to track performance like this over Rails releases. - Rob > **···** > > On 3/8/07, Jeremy Kemper [jeremy-w7CzD/W5Ocjk1uMJSBkQmQ@public.gmane.org](mailto:jeremy-w7CzD/W5Ocjk1uMJSBkQmQ@public.gmane.org) wrote: > > > On 3/7/07, Brian Adkins [lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > > > > zdennis wrote: > > > > > > > On Mar 6, 10:30 pm, Brian Adkins [lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > > > > > > > > When running a test that primarily involves loading up a few MySQL > > > > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > > > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > > > > this workload to be heavier on MySQL than that. > > > > > > > > What is your script doing? Can you post it? > > > > > > I created a smaller test that I could post that exhibits the same > > > characteristics: > > > > > > class PerfTestController \< ApplicationController > > > def index > > > t1 = Time.now > > > 3000.times do > > > member = Member.new > > > member.first\_name = ‘Fred’ > > > member.last\_name = ‘Flintstone’ > > > member.address1 = ‘123 High St.’ > > > member.city = ‘Reykjavik’ > > > member.state = ‘Michigan’ > > > member.email = ‘fred-Dsuv1UYZSSTz1n+OaKNE4w@public.gmane.org’ > > > member.save! > > > end > > > t2 = Time.now > > > puts “Time elapsed = #{t2-t1}” > > > end > > > end > > > > > > That took 35.7 seconds (84 inserts per second) on a dual core 2 GHz AMD > > > Opteron. It pegged Mongrel and MySQL didn’t break a sweat. > > > > > > I just ran another test with a short ruby program inserting records > > > directly using the mysql gem and it only took 1.6 seconds (1,875 inserts > > > per second!), and the CPU utilization was as it should be - the MySQL > > > CPU was ten times as much as Ruby. So it definitely appears that > > > Rails/ActiveRecord is about 22 times as slow than a straight Ruby > > > program - wow! > > > > > > This result makes me feel much better since the performance of Ruby > > > seems fine. The fact that Rails/ActiveRecord is way slow isn’t hurting > > > me yet, and there is hope it can be sped up since it doesn’t appear to > > > be an inherent problem with Ruby. > > > > > > Here’s the schema for Member: > > > > > > create table members ( > > > id int not null auto\_increment, > > > created\_at datetime not null, > > > updated\_at datetime not null, > > > first\_name varchar(30) null, > > > last\_name varchar(30) null, > > > address1 varchar(50) null, > > > address2 varchar(50) null, > > > city varchar(30) null, > > > state varchar(5) null, > > > email varchar(100) null, > > > home\_phone varchar(25) null, > > > primary key(id) > > > ) engine=InnoDB; > > > > Hi Brian, > > > > I wrapped this up in a simple script that anyone with MySQL or SQLite > > and the AR gem can run. It benchmarks AR create vs using the db > > connection directly. See attached. > > > > Excerpted results on a new MacBook Pro: > > user system total real > > raw quoted 0.460000 0.000000 0.460000 ( 0.480184) > > create 2.760000 0.080000 2.840000 ( 3.225227) > > > > (Nearly 7 times slower.) I haven’t tried profiling the methods yet. > > > > In my experience with typical Rails apps, you’ll hit a wall with ERB > > template rendering much sooner than with Active Record creation. This > > is an interesting pursuit nonetheless – I’m interested to see what > > you all come up with. > > > > Best regards, > > jeremy > > –~–~---------~–~----~------------~-------~–~----~ > You received this message because you are subscribed to the Google Groups “Ruby on Rails: Talk” group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > For more options, visit this group at [http://groups.google.com/group/rubyonrails-talk?hl=en](http://groups.google.com/group/rubyonrails-talk?hl=en) > -~----------~----~----~----~------~----~------~–~— ---
**Author:** ![Devin\_Mullins](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@Devin\_Mullins](https://rubytalk.org/u/Devin_Mullins)\ **Post date:** [8 March 2007 05:05 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/15 "2007-03-08T05:05:39Z")
Brian Adkins wrote: > I created a smaller test that I could post that exhibits the same characteristics: > >     3000.times do >       member = Member.new >       member.first\_name = 'Fred' \> ... >       member.save! >     end Is this being run in development mode, still? Dev mode does database reflection pretty often -- you might be getting pinged by that. ---
**Author:** ![Mohit\_Sindhwani](https://avatars.discourse-cdn.com/v4/letter/m/ba8739/32.png) [@Mohit\_Sindhwani](https://rubytalk.org/u/Mohit_Sindhwani)\ **Post date:** [8 March 2007 09:46 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/16 "2007-03-08T09:46:31Z")
Jeremy Kemper wrote: > > zdennis wrote: > > > > > > When running a test that primarily involves loading up a few MySQL > > > > tables with ActiveRecord objects, I was surprised to see the Ruby CPU > > > > utilization at 93% and the MySQL CPU utilization at 7%. I would expect > > > > this workload to be heavier on MySQL than that. > > > > > > What is your script doing? Can you post it? > > > > I created a smaller test that I could post that exhibits the same > > characteristics: > > > > class PerfTestController \< ApplicationController > > def index > > t1 = Time.now > > 3000.times do > > member = Member.new > > member.first\_name = ‘Fred’ > > member.last\_name = ‘Flintstone’ > > member.address1 = ‘123 High St.’ > > member.city = ‘Reykjavik’ > > member.state = ‘Michigan’ > > member.email = ‘fred-Dsuv1UYZSSTz1n+OaKNE4w@public.gmane.org’ > > member.save! > > end > > t2 = Time.now > > puts “Time elapsed = #{t2-t1}” > > end > > end > > > > That took 35.7 seconds (84 inserts per second) on a dual core 2 GHz AMD > > Opteron. It pegged Mongrel and MySQL didn’t break a sweat. > > > > I just ran another test with a short ruby program inserting records > > directly using the mysql gem and it only took 1.6 seconds (1,875 inserts > > per second!), and the CPU utilization was as it should be - the MySQL > > CPU was ten times as much as Ruby. So it definitely appears that > > Rails/ActiveRecord is about 22 times as slow than a straight Ruby > > program - wow! > > > > This result makes me feel much better since the performance of Ruby > > seems fine. The fact that Rails/ActiveRecord is way slow isn’t hurting > > me yet, and there is hope it can be sped up since it doesn’t appear to > > be an inherent problem with Ruby. > > > > Here’s the schema for Member: > > > > create table members ( > > id int not null auto\_increment, > > created\_at datetime not null, > > updated\_at datetime not null, > > first\_name varchar(30) null, > > last\_name varchar(30) null, > > address1 varchar(50) null, > > address2 varchar(50) null, > > city varchar(30) null, > > state varchar(5) null, > > email varchar(100) null, > > home\_phone varchar(25) null, > > primary key(id) > > ) engine=InnoDB; > > Hi Brian, > > I wrapped this up in a simple script that anyone with MySQL or SQLite > and the AR gem can run. It benchmarks AR create vs using the db > connection directly. See attached. > > Excerpted results on a new MacBook Pro: > user system total real > raw quoted 0.460000 0.000000 0.460000 ( 0.480184) > create 2.760000 0.080000 2.840000 ( 3.225227) > > (Nearly 7 times slower.) I haven’t tried profiling the methods yet. > > In my experience with typical Rails apps, you’ll hit a wall with ERB > template rendering much sooner than with Active Record creation. This > is an interesting pursuit nonetheless – I’m interested to see what > you all come up with. > > Best regards, > jeremy Hi, are these results in production or development? Talking about template rendering, I’m just wondering if anyone has thought of pre-processing the template for a production environment. For example, I’d imagine it’s might convenient to do things like ‘\<% tylesheet\_tag %\>’ etc when linking in files. But, each time Rails hits one of those, it needs to render it. Would it make sense to have a smart pre-processor that goes through the templates to see what constants there are (like links to the same stylesheet, javascript files, etc.) and even things like “form\_start\_tag”/ “form\_end\_tag” etc. to pre-render it so that the number of times you need to create something is reduced. Has anyone benchmarked the time for rendering a page with many or few such items? It would be really interesting to have a tool that could pre-render such things for production… Cheers Mohit. > **···** > > > On 3/7/07, Brian Adkins [lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOSPAM-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > > > > > On Mar 6, 10:30 pm, Brian Adkins [lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org](mailto:lojicdotcomNOS...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org) wrote: > > –~–~---------~–~----~------------~-------~–~----~ > You received this message because you are subscribed to the Google Groups “Ruby on Rails: Talk” group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > For more options, visit this group at [http://groups.google.com/group/rubyonrails-talk?hl=en](http://groups.google.com/group/rubyonrails-talk?hl=en) > -~----------~----~----~----~------~----~------~–~— ---
**Author:** ![Jeremy\_Kemper1](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@Jeremy\_Kemper1](https://rubytalk.org/u/Jeremy_Kemper1)\ **Post date:** [8 March 2007 09:36 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/17 "2007-03-08T09:36:01Z")
An updated script that actually works with MySQL is attached 😉 Transactions make a huge difference with MySQL compared to the negligible impact with in-memory SQLite.                      user system total real raw quoted 0.480000 0.120000 0.600000 ( 2.530020) raw quoted txn 0.230000 0.060000 0.290000 ( 0.546332) create 3.170000 0.320000 3.490000 ( 5.685493) create txn 1.990000 0.090000 2.080000 ( 2.392676) (2.2x slower without a transaction; 4.4x slower with) jeremy [ar\_bench.rb](https://rubytalk.org/uploads/short-url/nVbkN8f5feh0ikMFYpMOl7ggts5.rb) (2.88 KB) > **···** > > On 3/7/07, Jeremy Kemper \ wrote: > > > I wrapped this up in a simple script that anyone with MySQL or SQLite > > and the AR gem can run. It benchmarks AR create vs using the db > > connection directly. See attached. > > > > Excerpted results on a new MacBook Pro: > >                           user system total real > > raw quoted 0.460000 0.000000 0.460000 ( 0.480184) > > create 2.760000 0.080000 2.840000 ( 3.225227) > > > > (Nearly 7 times slower.) I haven't tried profiling the methods yet. ---
**Author:** ![Jeremy\_Kemper1](https://avatars.discourse-cdn.com/v4/letter/j/c6cbf5/32.png) [@Jeremy\_Kemper1](https://rubytalk.org/u/Jeremy_Kemper1)\ **Post date:** [8 March 2007 16:01 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/18 "2007-03-08T16:01:31Z")
Sure; I think so. But I don't want to compile or maintain the suite 😉 jeremy > **···** > > On 3/8/07, Rob Sanheim \ wrote: > > > Are there, or could there be, performance tests like this added to the > > Rails test suite? It would be great to be able to track performance > > like this over Rails releases. ---
**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\ **Post date:** [7 March 2007 08:59 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/19 "2007-03-07T08:59:46Z")
> If there's an alternative I haven't mentioned that you feel is actually viable as a contender, feel free to pass it on. Not sure about a viable alternative; but from what you've said I thought you might find some of Kirk Haines' posts in the last month or two on the Eventmachine mailing list interesting: [http://rubyforge.org/pipermail/eventmachine-talk/](http://rubyforge.org/pipermail/eventmachine-talk/) [http://rubyforge.org/pipermail/eventmachine-talk/2007-February/000395.html](http://rubyforge.org/pipermail/eventmachine-talk/2007-February/000395.html) [http://rubyforge.org/pipermail/eventmachine-talk/2007-March/000494.html](http://rubyforge.org/pipermail/eventmachine-talk/2007-March/000494.html) [http://rubyforge.org/pipermail/eventmachine-talk/2007-March/000510.html](http://rubyforge.org/pipermail/eventmachine-talk/2007-March/000510.html) Regards, Bill > **···** > > From: "Brian Adkins" \ ---
**Author:** ![Avdi\_Grimm1](https://avatars.discourse-cdn.com/v4/letter/a/e47c2d/32.png) [@Avdi\_Grimm1](https://rubytalk.org/u/Avdi_Grimm1)\ **Post date:** [7 March 2007 17:14 UTC](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939/20 "2007-03-07T17:14:32Z")
The best place to look for ruby-related mailing lists and newsgroups is [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). You can find a link to the Rails mailing list from there. You may find it easier to use the ruby-forum frontend, though, because it's a very high-volume list. That said, I concur that ActiveRecord has wider applicability than just Rails, and I find this an interesting topic of discussion. > **···** > > On 3/7/07, Brian Adkins \ wrote: > > > If you can point me to the Rails newsgroup, I'll try subscribing even if > > it doesn't show in Thunderbird. If you're referring to a mailing list > > instead, that is less desirable to me, but I could make use of it for > > Rails posts that are less relevant to Ruby. > > -- > Avdi [Next page](https://rubytalk.org/t/high-activerecord-cpu-utilization/35939.md?page=2)