High ActiveRecord CPU Utilization

Devin Mullins wrote:

Brian Adkins wrote:

I created a smaller test that I could post that exhibits the same characteristics:

    3000.times do
      member = Member.new
      member.first_name = 'Fred'

> ...

      member.save!
    end

Is this being run in development mode, still? Dev mode does database reflection pretty often -- you might be getting pinged by that.

Nope, it's running in the production environment.

mongrel_rails start -e production

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

There is no difference between the two in this case, so the script
doesn't set RAILS_ENV at all.

Talking about template rendering, I'm just wondering if anyone has
thought of pre-processing the template for a production environment.
For example, I'd imagine it's might convenient to do things like '<%
tylesheet_tag %>' etc when linking in files. But, each time Rails hits
one of those, it needs to render it. Would it make sense to have a
smart pre-processor that goes through the templates to see what
constants there are (like links to the same stylesheet, javascript
files, etc.) and even things like "form_start_tag"/ "form_end_tag" etc.
to pre-render it so that the number of times you need to create
something is reduced.

Yes! See Stefan Kaes' template optimizer: http://railsexpress.de/plugins/trac

Has anyone benchmarked the time for rendering a page with many or few
such items?

Yes. See Stefan's blog; he regularly benchmarks his apps against each
new Rails release. I think he's covered the impact of the template
optimizer.

jeremy

···

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:

···

On 3/8/07, Rob Sanheim <rsanheim@gmail.com> wrote:

Are there, or could there be, performance tests like this added to the
Rails test suite? It would be great to be able to track performance
like this over Rails releases.

Sure; I think so. But I don't want to compile or maintain the suite :wink:

jeremy

I would think someone in the "core Rails team" would step up to the plate for Rails-specific benchmarks. But if the specific case under discussion here -- ActiveRecord -- turns out to have a bottleneck somewhere in the Ruby interpreter, rather than just an inefficient strategy in the ActiveRecord code itself, then I'd say this benchmark belongs in a Ruby benchmark suite rather than a Rails benchmark suite.

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

No, sorry: it doesn’t load a Rails environment at all, just vanilla
Active Record.

The Rails environment has no bearing on Active Record in isolation,
beyond choosing the default database connection:
$ grep -r RAILS_ENV lib/ |grep -v svn|wc -l
2

Your Rails app reloads application classes during development so some
AR caches, like the per-class table metadata, are wiped as a result.

jeremy

···

On 3/8/07, Tom Mornini tmornini-W/9V78bTXriB+jHODAdFcQ@public.gmane.org wrote:

Are you saying that config/production.rb and config/development.rb
are identical?

–~–~---------~–~----~------------~-------~–~----~
You received this message because you are subscribed to the Google Groups “Ruby on Rails: Talk” group.
To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~–~—

Hello Jeremy.

Are you saying that config/production.rb and config/development.rb
are identical?

···


– Tom Mornini, CTO
– Engine Yard, Ruby on Rails Hosting
– Reliability, Ease of Use, Scalability
– (866) 518-YARD (9273)

On Mar 8, 2007, at 4:55 AM, Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani mo_mail-RxrYI66vbj0AvxtiuMwx3w@public.gmane.org wrote:

Jeremy Kemper wrote:

Excerpted results on a new MacBook Pro:
user system total real
raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
create 2.760000 0.080000 2.840000 ( 3.225227)

Hi, are these results in production or development?

There is no difference between the two in this case, so the script
doesn’t set RAILS_ENV at all.

–~–~---------~–~----~------------~-------~–~----~
You received this message because you are subscribed to the Google Groups “Ruby on Rails: Talk” group.
To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~–~—

Avdi Grimm wrote:

If you can point me to the Rails newsgroup, I'll try subscribing even if
it doesn't show in Thunderbird. If you're referring to a mailing list
instead, that is less desirable to me, but I could make use of it for
Rails posts that are less relevant to Ruby.

The best place to look for ruby-related mailing lists and newsgroups
is http://www.ruby-forum.com/\. You can find a link to the Rails
mailing list from there. You may find it easier to use the ruby-forum
frontend, though, because it's a very high-volume list.

That said, I concur that ActiveRecord has wider applicability than
just Rails, and I find this an interesting topic of discussion.

Yes, I'd like to see this discussion continue on the Ruby list, because there are some things to be learned from experiences with ActiveRecord. So, some questions for Brian:

1. What's your platform, and what version of Ruby are you running?
2. Is it possible for you to abstract a subset of your application as a benchmark, suitable for profiling?
3. Is it "simple enough" that it will "probably work" with the recent jRuby implementation of Rails?

···

On 3/7/07, Brian Adkins <lojicdotcomNOSPAM@gmail.com> wrote:

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

Jeremy Kemper wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby? While reading Jeremy's posts I realized I didn't see either of your posts via my newsreader. When I noticed the first time, I thought my ISP might have just randomly dropped some posts, but when I went to:

I saw both of your posts, but neither showed up via my newsreader (Thunderbird). That seemed too coincidental for a Bellsouth screw up, so I thought I'd ask.

Thanks,
Brian

···

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

M. Edward (Ed) Borasky wrote:

Avdi Grimm wrote:

If you can point me to the Rails newsgroup, I'll try subscribing even if
it doesn't show in Thunderbird. If you're referring to a mailing list
instead, that is less desirable to me, but I could make use of it for
Rails posts that are less relevant to Ruby.

The best place to look for ruby-related mailing lists and newsgroups
is http://www.ruby-forum.com/\. You can find a link to the Rails
mailing list from there. You may find it easier to use the ruby-forum
frontend, though, because it's a very high-volume list.

That said, I concur that ActiveRecord has wider applicability than
just Rails, and I find this an interesting topic of discussion.

Yes, I'd like to see this discussion continue on the Ruby list, because there are some things to be learned from experiences with ActiveRecord. So, some questions for Brian:

1. What's your platform, and what version of Ruby are you running?

$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

$ uname -a
Linux airstream 2.6.17-11-generic #2 SMP Thu Feb 1 19:52:28 UTC 2007 i686 GNU/Linux

$ rails -v
Rails 1.2.1

I'm running Ubuntu 6.10

2. Is it possible for you to abstract a subset of your application as a benchmark, suitable for profiling?

Well, the "application" in this case is just a simple test for benchmarking :slight_smile: See my previous post for the Rails controller code (what little there is of it).

3. Is it "simple enough" that it will "probably work" with the recent jRuby implementation of Rails?

I would hope so - it doesn't get much simpler.

···

On 3/7/07, Brian Adkins <lojicdotcomNOSPAM@gmail.com> wrote:

I believe that was a cross-posted reply by Jeremy. I never saw the message either.

James Edward Gray II

···

On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:

Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby?

Brian Adkins wrote:

M. Edward (Ed) Borasky wrote:

Avdi Grimm wrote:

If you can point me to the Rails newsgroup, I'll try subscribing even if
it doesn't show in Thunderbird. If you're referring to a mailing list
instead, that is less desirable to me, but I could make use of it for
Rails posts that are less relevant to Ruby.

The best place to look for ruby-related mailing lists and newsgroups
is http://www.ruby-forum.com/\. You can find a link to the Rails
mailing list from there. You may find it easier to use the ruby-forum
frontend, though, because it's a very high-volume list.

That said, I concur that ActiveRecord has wider applicability than
just Rails, and I find this an interesting topic of discussion.

Yes, I'd like to see this discussion continue on the Ruby list, because there are some things to be learned from experiences with ActiveRecord. So, some questions for Brian:

1. What's your platform, and what version of Ruby are you running?

$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

A couple of suggestions here:

1. Download the latest Ruby source -- 1.8.6 pre something.
2. Compile it for your architecture -- set CFLAGS = "-O2 -march=xxx"

     where "xxx" is your architecture -- it's an AMD64 of some kind, right?

     If you're running a 64-bit system, make sure you have a recent 64-bit kernel and GCC 4.1 -- older compilers suck wet dog fur on the 64 bit machines.

That should get you somewhere in the 10 - 30 percent speed improvement over a 486-compiled Ruby 1.8.4. It might be more, but just doing the 1.8.6 and the -O2 / march= stuff is pretty much mandatory.

$ uname -a
Linux airstream 2.6.17-11-generic #2 SMP Thu Feb 1 19:52:28 UTC 2007 i686 GNU/Linux

That should be OK -- most likely it's user time anyhow, not kernel time.

$ rails -v
Rails 1.2.1

I'm running Ubuntu 6.10

That should be fine too. The compiler is probably more important.

2. Is it possible for you to abstract a subset of your application as a benchmark, suitable for profiling?

Well, the "application" in this case is just a simple test for benchmarking :slight_smile: See my previous post for the Rails controller code (what little there is of it).

Yeah, I saw that.

3. Is it "simple enough" that it will "probably work" with the recent jRuby implementation of Rails?

I would hope so - it doesn't get much simpler.

I know Charles Oliver Nutter reads this list -- he's looking for tests for the latest jRuby/Rails.

···

On 3/7/07, Brian Adkins <lojicdotcomNOSPAM@gmail.com> wrote:

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

James Edward Gray II wrote:

···

On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:

Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby?

I believe that was a cross-posted reply by Jeremy. I never saw the message either.

James Edward Gray II

Interesting. I guess I mistakenly assumed a one-to-one correspondence between comp.lang.ruby and the "ruby" forum at http://www.ruby-forum.com/ but now that I look more closely at the latter, it does say "Gateway to the ruby-talk mailing list."

Thanks for confusing me Jeremy :frowning:

M. Edward (Ed) Borasky wrote:

Brian Adkins wrote:

M. Edward (Ed) Borasky wrote:

1. What's your platform, and what version of Ruby are you running?

$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

A couple of suggestions here:

1. Download the latest Ruby source -- 1.8.6 pre something.
2. Compile it for your architecture -- set CFLAGS = "-O2 -march=xxx"

    where "xxx" is your architecture -- it's an AMD64 of some kind, right?

    If you're running a 64-bit system, make sure you have a recent 64-bit kernel and GCC 4.1 -- older compilers suck wet dog fur on the 64 bit machines.

That should get you somewhere in the 10 - 30 percent speed improvement over a 486-compiled Ruby 1.8.4. It might be more, but just doing the 1.8.6 and the -O2 / march= stuff is pretty much mandatory.

Interesting. Do you mind if I ask where you got the 10 to 30% figure? Stability is more important to me than raw speed, so I'd prefer to not use anything newer than Ruby 1.8.5-p12.

I'm running a 32 bit kernel because 64 bit was, let's say, problematic. My gcc is 4.1.2.

I don't believe ruby-forum is involved here. I think Jeremy cross-posted from the Rails list.

James Edward Gray II

···

On Mar 8, 2007, at 1:50 PM, Brian Adkins wrote:

James Edward Gray II wrote:

On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:

Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby?

I believe that was a cross-posted reply by Jeremy. I never saw the message either.
James Edward Gray II

Interesting. I guess I mistakenly assumed a one-to-one correspondence between comp.lang.ruby and the "ruby" forum at http://www.ruby-forum.com/ but now that I look more closely at the latter, it does say "Gateway to the ruby-talk mailing list."

Ha! Sorry. I just hit "Reply All" to Mohit's message. Blame GMail :wink:

jeremy

···

On 3/8/07, Brian Adkins <lojicdotcomNOSPAM@gmail.com> wrote:

James Edward Gray II wrote:
> On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:
>> Mohit, do you mind if I ask how you're posting messages to
>> comp.lang.ruby?
>
> I believe that was a cross-posted reply by Jeremy. I never saw the
> message either.

Interesting. I guess I mistakenly assumed a one-to-one correspondence
between comp.lang.ruby and the "ruby" forum at
http://www.ruby-forum.com/ but now that I look more closely at the
latter, it does say "Gateway to the ruby-talk mailing list."

Thanks for confusing me Jeremy :frowning:

Brian Adkins wrote:

Interesting. Do you mind if I ask where you got the 10 to 30% figure? Stability is more important to me than raw speed, so I'd prefer to not use anything newer than Ruby 1.8.5-p12.

I'm running a 32 bit kernel because 64 bit was, let's say, problematic. My gcc is 4.1.2.

http://www.jhaampe.org/software/ruby-gcc and http://rubyforge.org/viewvc/MatrixBenchmark/?root=cougar

The 1.8.6 preview 3 is due for release this weekend, IIRC. I'm not sure how one measures stability for a Ruby interpreter, but there are definitely a few performance tweaks -- my MatrixBenchmark picked up about 10 or 11 percent on a Pentium III from the switch. (with -O2 -march=pentium3, of course.) :slight_smile:

···

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

James Edward Gray II wrote:

···

On Mar 8, 2007, at 1:50 PM, Brian Adkins wrote:

James Edward Gray II wrote:

On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:

Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby?

I believe that was a cross-posted reply by Jeremy. I never saw the message either.
James Edward Gray II

Interesting. I guess I mistakenly assumed a one-to-one correspondence between comp.lang.ruby and the "ruby" forum at http://www.ruby-forum.com/ but now that I look more closely at the latter, it does say "Gateway to the ruby-talk mailing list."

I don't believe ruby-forum is involved here. I think Jeremy cross-posted from the Rails list.

James Edward Gray II

Well if you go to: High ActiveRecord CPU Utilization - Ruby - Ruby-Forum

You'll see this thread and you'll see Mohit's postings. As far as I can tell, the above forum appears to be a superset of this comp.lang.ruby thread. That's why I thought some postings were dropped, but if you didn't see them either, I'll stop worrying about it and continue on my way :slight_smile:

Preview 3 is out:
ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-preview3.tar.gz

I used it for the earlier benches and it has posed no problems for any
of my Rails apps in development.

Be sure to work with Rails' 1-2-stable branch (or edge) for a
smattering of 1.8.6 compatibility fixes due out in the upcoming Rails
1.2.3.

jeremy

···

On 3/8/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:

The 1.8.6 preview 3 is due for release this weekend, IIRC. I'm not sure
how one measures stability for a Ruby interpreter, but there are
definitely a few performance tweaks -- my MatrixBenchmark picked up
about 10 or 11 percent on a Pentium III from the switch. (with -O2
-march=pentium3, of course.) :slight_smile:

Ah yes, I see it now. This is some quirk of ruby-forum I'm not familiar with. Disregard my earlier posts. Sorry for spreading confusion.

James Edward Gray II

···

On Mar 8, 2007, at 3:10 PM, Brian Adkins wrote:

James Edward Gray II wrote:

On Mar 8, 2007, at 1:50 PM, Brian Adkins wrote:

James Edward Gray II wrote:

On Mar 8, 2007, at 1:30 PM, Brian Adkins wrote:

Jeremy Kemper wrote:

On 3/8/07, Mohit Sindhwani <mo_mail@onghu.com> wrote:

Jeremy Kemper wrote:
> Excerpted results on a new MacBook Pro:
> user system total real
> raw quoted 0.460000 0.000000 0.460000 ( 0.480184)
> create 2.760000 0.080000 2.840000 ( 3.225227)
>
Hi, are these results in production or development?

Mohit, do you mind if I ask how you're posting messages to comp.lang.ruby?

I believe that was a cross-posted reply by Jeremy. I never saw the message either.
James Edward Gray II

Interesting. I guess I mistakenly assumed a one-to-one correspondence between comp.lang.ruby and the "ruby" forum at http://www.ruby-forum.com/ but now that I look more closely at the latter, it does say "Gateway to the ruby-talk mailing list."

I don't believe ruby-forum is involved here. I think Jeremy cross-posted from the Rails list.
James Edward Gray II

Well if you go to: High ActiveRecord CPU Utilization - Ruby - Ruby-Forum

You'll see this thread and you'll see Mohit's postings. As far as I can tell, the above forum appears to be a superset of this comp.lang.ruby thread. That's why I thought some postings were dropped, but if you didn't see them either, I'll stop worrying about it and continue on my way :slight_smile:

And it's already in portage (I \heart Gentoo!)

martin

···

On 3/8/07, Jeremy Kemper <jeremy@bitsweat.net> wrote:

On 3/8/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> The 1.8.6 preview 3 is due for release this weekend, IIRC. I'm not sure
> how one measures stability for a Ruby interpreter, but there are
> definitely a few performance tweaks -- my MatrixBenchmark picked up
> about 10 or 11 percent on a Pentium III from the switch. (with -O2
> -march=pentium3, of course.) :slight_smile:

Preview 3 is out:
ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-preview3.tar.gz