Why is rake really slow?

ruby-talk
1

Is rake supposed to be really slow?

I'm using InstantRails and running rake in a project with one unit
test and two functional tests and it takes 33 seconds total. Is that
normal? It seems really slow to me.

When I turn on trace I can see that it spends most of its time in
"Execute environment" and loading the test files:

C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
user_test.rb"

Anyone else have this problem or is it normal? Thanks.

2

Is rake supposed to be really slow?

no, not that it is supposed to be really fast... but it isn't what gets in your way.

...
C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
user_test.rb"

My guess is that your test is slow. To start, take rake out of the equation:

  ruby -rtest/unit test/unit/user_test.rb

After that, you can do something simple like:

   def setup
     @start_time = Time.now
   end

   def teardown
     puts "** TIME: #{self.name} : #{Time.now - @start_time} seconds
   end

And see what's going so slow a little clearer.

Anyone else have this problem or is it normal? Thanks.

I run thousands of tests/assertions in just seconds many times a day:

ruby: Finished in 0.105172 seconds. 252 tests, 735 assertions, 0 failures, 0 errors
ruby: Finished in 1.275824 seconds. 750 tests, 2220 assertions, 0 failures, 0 errors
rails: Finished in 81.681246 seconds. 1924 tests, 12195 assertions, 0 failures, 0 errors

···

On Aug 13, 2007, at 22:04 , richpoirier@gmail.com wrote:

3

unknown wrote:

Is rake supposed to be really slow?

I'm using InstantRails and running rake in a project with one unit
test and two functional tests and it takes 33 seconds total. Is that
normal? It seems really slow to me.

When I turn on trace I can see that it spends most of its time in
"Execute environment" and loading the test files:

C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
user_test.rb"

Anyone else have this problem or is it normal? Thanks.

Rake in and of itself is quite fast. There are two types of things that
can slow you down.

(1) Executing the Rakefile. If you have lots of tasks or complex
dependencies, then whatever time it takes to resolve those can build up
and slow down every rake command.

(2) Slow tasks. Some tasks in rake are naturely slow. Slow takes make
anything that depends upon them slow as well.

I assume you are working in Rails? Add this to your top level Rakefile:

   task :noop

Then time the "rake noop" command. This will be the time it takes rake
to run essentially a do nothing task and will give you a sense of the
general Rakefile overhead.

Then time "rake environment". This task loads the rails environment and
can take some time. The environment task is a dependency of most rails
related rake tasks in a Rails project.

In the project I'm currently working on, "rake noop" takes under a
second. "rake environment" takes nearly 3 seconds. ("rake noop" in a
lightweight non-rails project takes under 0.2 seconds).

It sounds like the environment task on your system is painfully slow.
Is it doing anything unusual?

-- Jim Weirich

···

--
Posted via http://www.ruby-forum.com/\.

4

Thanks for the replies. Yes I'm working with rails. I think you are
both right. Running a single unit test without rake takes about 11
seconds (even though it says it finished in 0.118 seconds). The test
simply creates a new user in my User model. So maybe it's taking a
while to connect to my mysql db? I'll have to investigate that (not
sure how to though). rake environment takes about 13 seconds. Where is
the environment task defined? I'd like to take a look at it to see
what it's doing. Using rake to run that one unit test takes 24
seconds, so it's definitely a combination of the environment task
being painfully slow and the test itself being slow too.

Thanks again for the help Jim and Ryan.

···

On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:

unknown wrote:
> Israkesupposed to be reallyslow?

> I'm using InstantRails and runningrakein a project with one unit
> test and two functional tests and it takes 33 seconds total. Is that
> normal? It seems reallyslowto me.

> When I turn on trace I can see that it spends most of its time in
> "Execute environment" and loading the test files:

> C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> user_test.rb"

> Anyone else have this problem or is it normal? Thanks.

Rakein and of itself is quite fast. There are two types of things that
canslowyou down.

(1) Executing the Rakefile. If you have lots of tasks or complex
dependencies, then whatever time it takes to resolve those can build up
andslowdown everyrakecommand.

(2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
anything that depends upon themslowas well.

I assume you are working in Rails? Add this to your top level Rakefile:

   task :noop

Then time the "rakenoop" command. This will be the time it takesrake
to run essentially a do nothing task and will give you a sense of the
general Rakefile overhead.

Then time "rakeenvironment". This task loads the rails environment and
can take some time. The environment task is a dependency of most rails
relatedraketasks in a Rails project.

In the project I'm currently working on, "rakenoop" takes under a
second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
lightweight non-rails project takes under 0.2 seconds).

It sounds like the environment task on your system is painfullyslow.
Is it doing anything unusual?

-- Jim Weirich

--
Posted viahttp://www.ruby-forum.com/.

5

Ok I've narrowed down the environment part of the problem. What's
taking long is the require_frameworks method in rails' Initializer.
The active_record, action_controller and action_web_services
frameworks take a while (a few seconds each) to load. So now the
question is why? I'm not sure where to look to dig deeper into this,
i.e., see what the require method is actually doing. Any ideas?

I still have to look into why the test itself is slow too.

···

On Aug 15, 12:41 am, richpoir...@gmail.com wrote:

On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:

> unknown wrote:
> > Israkesupposed to be reallyslow?

> > I'm using InstantRails and runningrakein a project with one unit
> > test and two functional tests and it takes 33 seconds total. Is that
> > normal? It seems reallyslowto me.

> > When I turn on trace I can see that it spends most of its time in
> > "Execute environment" and loading the test files:

> > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > user_test.rb"

> > Anyone else have this problem or is it normal? Thanks.

> Rakein and of itself is quite fast. There are two types of things that
> canslowyou down.

> (1) Executing the Rakefile. If you have lots of tasks or complex
> dependencies, then whatever time it takes to resolve those can build up
> andslowdown everyrakecommand.

> (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> anything that depends upon themslowas well.

> I assume you are working in Rails? Add this to your top level Rakefile:

> task :noop

> Then time the "rakenoop" command. This will be the time it takesrake
> to run essentially a do nothing task and will give you a sense of the
> general Rakefile overhead.

> Then time "rakeenvironment". This task loads the rails environment and
> can take some time. The environment task is a dependency of most rails
> relatedraketasks in a Rails project.

> In the project I'm currently working on, "rakenoop" takes under a
> second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> lightweight non-rails project takes under 0.2 seconds).

> It sounds like the environment task on your system is painfullyslow.
> Is it doing anything unusual?

> -- Jim Weirich

> --
> Posted viahttp://www.ruby-forum.com/.

Thanks for the replies. Yes I'm working with rails. I think you are
both right. Running a single unit test withoutraketakes about 11
seconds (even though it says it finished in 0.118 seconds). The test
simply creates a new user in my User model. So maybe it's taking a
while to connect to my mysql db? I'll have to investigate that (not
sure how to though).rakeenvironment takes about 13 seconds. Where is
the environment task defined? I'd like to take a look at it to see
what it's doing. Usingraketo run that one unit test takes 24
seconds, so it's definitely a combination of the environment task
being painfullyslowand the test itself beingslowtoo.

Thanks again for the help Jim and Ryan.

6

Its normally a good idea to take out actual db connections when
running tests. Did you look into mocha or something?

···

On 8/16/07, Rich <richpoirier@gmail.com> wrote:

On Aug 15, 12:41 am, richpoir...@gmail.com wrote:
> On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:
>
>
>
> > unknown wrote:
> > > Israkesupposed to be reallyslow?
>
> > > I'm using InstantRails and runningrakein a project with one unit
> > > test and two functional tests and it takes 33 seconds total. Is that
> > > normal? It seems reallyslowto me.
>
> > > When I turn on trace I can see that it spends most of its time in
> > > "Execute environment" and loading the test files:
>
> > > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > > user_test.rb"
>
> > > Anyone else have this problem or is it normal? Thanks.
>
> > Rakein and of itself is quite fast. There are two types of things that
> > canslowyou down.
>
> > (1) Executing the Rakefile. If you have lots of tasks or complex
> > dependencies, then whatever time it takes to resolve those can build up
> > andslowdown everyrakecommand.
>
> > (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> > anything that depends upon themslowas well.
>
> > I assume you are working in Rails? Add this to your top level Rakefile:
>
> > task :noop
>
> > Then time the "rakenoop" command. This will be the time it takesrake
> > to run essentially a do nothing task and will give you a sense of the
> > general Rakefile overhead.
>
> > Then time "rakeenvironment". This task loads the rails environment and
> > can take some time. The environment task is a dependency of most rails
> > relatedraketasks in a Rails project.
>
> > In the project I'm currently working on, "rakenoop" takes under a
> > second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> > lightweight non-rails project takes under 0.2 seconds).
>
> > It sounds like the environment task on your system is painfullyslow.
> > Is it doing anything unusual?
>
> > -- Jim Weirich
>
> > --
> > Posted viahttp://www.ruby-forum.com/.
>
> Thanks for the replies. Yes I'm working with rails. I think you are
> both right. Running a single unit test withoutraketakes about 11
> seconds (even though it says it finished in 0.118 seconds). The test
> simply creates a new user in my User model. So maybe it's taking a
> while to connect to my mysql db? I'll have to investigate that (not
> sure how to though).rakeenvironment takes about 13 seconds. Where is
> the environment task defined? I'd like to take a look at it to see
> what it's doing. Usingraketo run that one unit test takes 24
> seconds, so it's definitely a combination of the environment task
> being painfullyslowand the test itself beingslowtoo.
>
> Thanks again for the help Jim and Ryan.

Ok I've narrowed down the environment part of the problem. What's
taking long is the require_frameworks method in rails' Initializer.
The active_record, action_controller and action_web_services
frameworks take a while (a few seconds each) to load. So now the
question is why? I'm not sure where to look to dig deeper into this,
i.e., see what the require method is actually doing. Any ideas?

I still have to look into why the test itself is slow too.

7

Probably useless data point:

On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
with RAID-0 10,000-RPM drives, initializing the rails environment takes
7-10 seconds.

On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
initializing the rails environment takes a second or two.

You can imagine my workaround.

Jay Levitt

···

On Thu, 16 Aug 2007 02:57:30 -0000, Rich wrote:

Ok I've narrowed down the environment part of the problem. What's
taking long is the require_frameworks method in rails' Initializer.
The active_record, action_controller and action_web_services
frameworks take a while (a few seconds each) to load. So now the
question is why? I'm not sure where to look to dig deeper into this,
i.e., see what the require method is actually doing. Any ideas?

8

Hemant Kumar wrote:

···

On 8/16/07, Rich <richpoirier@gmail.com> wrote:

> > > normal? It seems reallyslowto me.
> > Rakein and of itself is quite fast. There are two types of things that
>
> > In the project I'm currently working on, "rakenoop" takes under a
>
>

Its normally a good idea to take out actual db connections when
running tests. Did you look into mocha or something?

Could this be your problem too.
http://forums.mysql.com/read.php?35,64514,64514

I had same problem on a Linux box.

by
TheR
--
Posted via http://www.ruby-forum.com/\.

9

I'm running rails tests though. Don't most rails tests require a db
connection?

···

On Aug 15, 11:50 pm, hemant <gethem...@gmail.com> wrote:

On 8/16/07, Rich <richpoir...@gmail.com> wrote:

> On Aug 15, 12:41 am, richpoir...@gmail.com wrote:
> > On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:

> > > unknown wrote:
> > > > Israkesupposed to be reallyslow?

> > > > I'm using InstantRails and runningrakein a project with one unit
> > > > test and two functional tests and it takes 33 seconds total. Is that
> > > > normal? It seems reallyslowto me.

> > > > When I turn on trace I can see that it spends most of its time in
> > > > "Execute environment" and loading the test files:

> > > > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > > > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > > > user_test.rb"

> > > > Anyone else have this problem or is it normal? Thanks.

> > > Rakein and of itself is quite fast. There are two types of things that
> > > canslowyou down.

> > > (1) Executing the Rakefile. If you have lots of tasks or complex
> > > dependencies, then whatever time it takes to resolve those can build up
> > > andslowdown everyrakecommand.

> > > (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> > > anything that depends upon themslowas well.

> > > I assume you are working in Rails? Add this to your top level Rakefile:

> > > task :noop

> > > Then time the "rakenoop" command. This will be the time it takesrake
> > > to run essentially a do nothing task and will give you a sense of the
> > > general Rakefile overhead.

> > > Then time "rakeenvironment". This task loads the rails environment and
> > > can take some time. The environment task is a dependency of most rails
> > > relatedraketasks in a Rails project.

> > > In the project I'm currently working on, "rakenoop" takes under a
> > > second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> > > lightweight non-rails project takes under 0.2 seconds).

> > > It sounds like the environment task on your system is painfullyslow.
> > > Is it doing anything unusual?

> > > -- Jim Weirich

> > > --
> > > Posted viahttp://www.ruby-forum.com/.

> > Thanks for the replies. Yes I'm working with rails. I think you are
> > both right. Running a single unit test withoutraketakes about 11
> > seconds (even though it says it finished in 0.118 seconds). The test
> > simply creates a new user in my User model. So maybe it's taking a
> > while to connect to my mysql db? I'll have to investigate that (not
> > sure how to though).rakeenvironment takes about 13 seconds. Where is
> > the environment task defined? I'd like to take a look at it to see
> > what it's doing. Usingraketo run that one unit test takes 24
> > seconds, so it's definitely a combination of the environment task
> > being painfullyslowand the test itself beingslowtoo.

> > Thanks again for the help Jim and Ryan.

> Ok I've narrowed down the environment part of the problem. What's
> taking long is the require_frameworks method in rails' Initializer.
> The active_record, action_controller and action_web_services
> frameworks take a while (a few seconds each) to load. So now the
> question is why? I'm not sure where to look to dig deeper into this,
> i.e., see what the require method is actually doing. Any ideas?

> I still have to look into why the test itself isslowtoo.

Its normally a good idea to take out actual db connections when
running tests. Did you look into mocha or something?

10

Err... is it to run One-Click Installer Ruby instead of Cygwin?

···

On 8/16/07, Jay Levitt <jay+news@jay.fm> wrote:

On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
with RAID-0 10,000-RPM drives, initializing the rails environment takes
7-10 seconds.

On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
initializing the rails environment takes a second or two.

You can imagine my workaround.

--
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]

11

That's really interesting Jay. I'm running RoR on my HP dual core AMD
Turion 1.6GHz laptop. So maybe the 10+ seconds my environment takes to
initialize is normal for my system then? Should I just get a MacBook
or is there any way I can speed the initialization up?

···

On Aug 16, 9:14 am, Jay Levitt <jay+n...@jay.fm> wrote:

On Thu, 16 Aug 2007 02:57:30 -0000, Rich wrote:
> Ok I've narrowed down the environment part of the problem. What's
> taking long is the require_frameworks method in rails' Initializer.
> The active_record, action_controller and action_web_services
> frameworks take a while (a few seconds each) to load. So now the
> question is why? I'm not sure where to look to dig deeper into this,
> i.e., see what the require method is actually doing. Any ideas?

Probably useless data point:

On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
with RAID-0 10,000-RPM drives, initializing the rails environment takes
7-10 seconds.

On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
initializing the rails environment takes a second or two.

You can imagine my workaround.

Jay Levitt

12

Also, switching to RSpec might help here. In addition to mocks to
isolate the DB from the tests, you can run rspec_server which loads
the Rails environment once and then when you run your tests, they
connect to the server, and respond much more quickly.

Ed

···

On 8/15/07, hemant <gethemant@gmail.com> wrote:

On 8/16/07, Rich <richpoirier@gmail.com> wrote:

Its normally a good idea to take out actual db connections when
running tests. Did you look into mocha or something?

--
Ed Howland

"The information transmitted is intended only for the person or entity
to which it is addressed and may contain proprietary, confidential
and/or legally privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance
upon, this information by persons or entities other than the intended
recipient is prohibited. If you received this in error, please contact
the sender and delete the material from all computers."

13

I'm just connecting to my local db, that post has to do with
connecting to a remote one. Thanks though.

···

On Aug 16, 2:24 am, Damjan Rems <d_r...@yahoo.com> wrote:

Hemant Kumar wrote:
> On 8/16/07, Rich <richpoir...@gmail.com> wrote:
>> > > > normal? It seems reallyslowto me.
>> > > Rakein and of itself is quite fast. There are two types of things that

>> > > In the project I'm currently working on, "rakenoop" takes under a

> Its normally a good idea to take out actual db connections when
> running tests. Did you look into mocha or something?

Could this be your problem too.http://forums.mysql.com/read.php?35,64514,64514

I had same problem on a Linux box.

by
TheR
--
Posted viahttp://www.ruby-forum.com/.

14

Ok here's an easy way to compare performance. I decided to see if it
would be any quicker at work so I installed InstantRails and just ran
rake in the rails_apps/typo-2.6.0 directory and it took 44 seconds
total. With trace on, I can see that the environment task takes about
8 seconds, and the test_unit and test_functional tasks take about 12
seconds each -- that's 12 seconds of loading before any actual tests
are run; the tests themselves run quite quickly (a fraction of a
second each). This was a completely clean installation of InstantRails
on a machine that had never touched ruby or rails before. Could
someone else try the same and see how long rake takes in the
typo-2.6.0 app? Thanks.

···

On Aug 16, 1:18 pm, Rich <richpoir...@gmail.com> wrote:

On Aug 16, 9:14 am, Jay Levitt <jay+n...@jay.fm> wrote:

> On Thu, 16 Aug 2007 02:57:30 -0000, Rich wrote:
> > Ok I've narrowed down the environment part of the problem. What's
> > taking long is the require_frameworks method in rails' Initializer.
> > The active_record, action_controller and action_web_services
> > frameworks take a while (a few seconds each) to load. So now the
> > question is why? I'm not sure where to look to dig deeper into this,
> > i.e., see what the require method is actually doing. Any ideas?

> Probably useless data point:

> On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
> with RAID-0 10,000-RPM drives, initializing the rails environment takes
> 7-10 seconds.

> On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
> initializing the rails environment takes a second or two.

> You can imagine my workaround.

> Jay Levitt

That's really interesting Jay. I'm running RoR on my HP dual core AMD
Turion 1.6GHz laptop. So maybe the 10+ seconds my environment takes to
initialize is normal for my system then? Should I just get a MacBook
or is there any way I can speed the initialization up?

15

Ok Ed, I've taken your advice and tried out rspec. I actually noticed
that the environment is loaded once before the tests are run and once
before each test task starts, so it's unnecessarily run three times if
you're testing units and functionals. So spec_server sounded like a
great solution and BDD is also very appealing.

However, I can't seem to get spec_server working properly. I can get
it running but executing a spec after that tells me the server is not
running, when I know it definitely is. I've tried with and without
rake. I'm using Windows Vista, so maybe that's the problem? Do you
know anyone who's gotten it working in Vista? Thanks.

···

On Aug 16, 5:05 pm, "Ed Howland" <ed.howl...@gmail.com> wrote:

On 8/15/07, hemant <gethem...@gmail.com> wrote:

> On 8/16/07, Rich <richpoir...@gmail.com> wrote:

> Its normally a good idea to take out actual db connections when
> running tests. Did you look into mocha or something?

Also, switching to RSpec might help here. In addition to mocks to
isolate the DB from the tests, you can run rspec_server which loads
the Rails environment once and then when you run your tests, they
connect to the server, and respond much more quickly.

Ed
--
Ed Howlandhttp://greenprogrammer.blogspot.com
"The information transmitted is intended only for the person or entity
to which it is addressed and may contain proprietary, confidential
and/or legally privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance
upon, this information by persons or entities other than the intended
recipient is prohibited. If you received this in error, please contact
the sender and delete the material from all computers."

16

Ok I got it working. Turns out I didn't actually have any specs to
run. But it was still telling me no server was running which is
strange. I created a spec and ran it and man did it run fast compared
to Test::Unit.

So I think I'm all set now. rspec is awesome! Thanks for everyone's
help.

···

On Aug 17, 1:34 am, Rich <richpoir...@gmail.com> wrote:

On Aug 16, 5:05 pm, "Ed Howland" <ed.howl...@gmail.com> wrote:

> On 8/15/07, hemant <gethem...@gmail.com> wrote:

> > On 8/16/07, Rich <richpoir...@gmail.com> wrote:

> > Its normally a good idea to take out actual db connections when
> > running tests. Did you look into mocha or something?

> Also, switching to RSpec might help here. In addition to mocks to
> isolate the DB from the tests, you can run rspec_server which loads
> the Rails environment once and then when you run your tests, they
> connect to the server, and respond much more quickly.

> Ed
> --
> Ed Howlandhttp://greenprogrammer.blogspot.com
> "The information transmitted is intended only for the person or entity
> to which it is addressed and may contain proprietary, confidential
> and/or legally privileged material. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance
> upon, this information by persons or entities other than the intended
> recipient is prohibited. If you received this in error, please contact
> the sender and delete the material from all computers."

Ok Ed, I've taken your advice and tried out rspec. I actually noticed
that the environment is loaded once before the tests are run and once
before each test task starts, so it's unnecessarily run three times if
you're testing units and functionals. So spec_server sounded like a
great solution and BDD is also very appealing.

However, I can't seem to get spec_server working properly. I can get
it running but executing a spec after that tells me the server is not
running, when I know it definitely is. I've tried with and withoutrake. I'm using Windows Vista, so maybe that's the problem? Do you
know anyone who's gotten it working in Vista? Thanks.