Okay, this is sort of embarrassing, but for some time now I've been using the (deprecated) RubyUnit instead of Test::Unit. Since RubyConf I've been using Test::Unit for newer projects, and I'm definitely happy with all the various improvements.
But now I'm trying to switch over a pre-existing private project to Test::Unit, and it seems to run a lot slower than RubyUnit. I've got about 350 unit tests, which used to run in about a minute in RubyUnit. But now when I try to run them in Test::Unit, they won't even complete. Individual tests run fine, but everything seems like it's a lot slower.
Has anybody else encountered this problem? Any particular quirks or workarounds I should be aware of? Thanks in advance.
F.
In my testing of quiz submissions for Regex.build(), I ran into some nasty slowdowns with Test::Unit. I was running a LOT of tests, but it was still quite a bit slower than it needed to be.
James Edward Gray II
···
On Nov 9, 2004, at 8:06 AM, Francis Hwang wrote:
Has anybody else encountered this problem? Any particular quirks or workarounds I should be aware of? Thanks in advance.
Let me ask more specifically: Is anybody out there using Test::Unit to run hundreds of tests? If you are, and have any tips as to how I can manage the speed issue, your tips would be much much appreciated.
F.
In article <368A22C4-3259-11D9-90EE-000A95DBF456@fhwang.net>, Francis Hwang
wrote:
Okay, this is sort of embarrassing, but for some time now I've been
using the (deprecated) RubyUnit instead of Test::Unit. Since RubyConf
I've been using Test::Unit for newer projects, and I'm definitely happy
with all the various improvements.
But now I'm trying to switch over a pre-existing private project to
Test::Unit, and it seems to run a lot slower than RubyUnit. I've got
about 350 unit tests, which used to run in about a minute in RubyUnit.
But now when I try to run them in Test::Unit, they won't even complete.
Individual tests run fine, but everything seems like it's a lot slower.
Has anybody else encountered this problem? Any particular quirks or
workarounds I should be aware of? Thanks in advance.
I never used RubyUnit, but the setup and teardown methods are called once
per test-case. Perhaps you're expecting that they're only called once per
TestCase class.
Francis,
I've certainly had test suites with hundreds of tests, and never seemed to have problems. Have you been able to run a profiler on it? I'd be more than willing to apply patches, especially if they don't affect the API.
Nathaniel
Terralien, Inc.
<:((><
···
On Nov 9, 2004, at 20:28, Francis Hwang wrote:
Let me ask more specifically: Is anybody out there using Test::Unit to run hundreds of tests? If you are, and have any tips as to how I can manage the speed issue, your tips would be much much appreciated.
On a lowly Pentium 233, I run 303 tests from Ruby2C in 12 seconds. The time to run the test suite starting from 10s of tests seems to have grown linearly. On a decent box (PowerBook) the tests can all run in 1-2 seconds.
$ wc -l test_* | grep total
3793 total
$ time ./test_all.rb
Loaded suite ./test_all
Started
...............................................................................................................................................................................................................................................................................................................
Finished in 12.027582 seconds.
303 tests, 481 assertions, 0 failures, 0 errors
real 0m13.810s
user 0m12.445s
sys 0m0.168s
···
On Nov 9, 2004, at 5:28 PM, Francis Hwang wrote:
Let me ask more specifically: Is anybody out there using Test::Unit to run hundreds of tests? If you are, and have any tips as to how I can manage the speed issue, your tips would be much much appreciated.
Hi Nathaniel,
I'll start playing around with a few of the tests, and profiling them. I just wanted to check first if others were trying to test on this scale before I bothered. It's quite possible I'm just doing something braindead. I'll poke around and let you know what I find.
F.
···
On Nov 9, 2004, at 8:38 PM, Nathaniel Talbott wrote:
On Nov 9, 2004, at 20:28, Francis Hwang wrote:
Let me ask more specifically: Is anybody out there using Test::Unit to run hundreds of tests? If you are, and have any tips as to how I can manage the speed issue, your tips would be much much appreciated.
Francis,
I've certainly had test suites with hundreds of tests, and never seemed to have problems. Have you been able to run a profiler on it? I'd be more than willing to apply patches, especially if they don't affect the API.
Nathaniel
Terralien, Inc.
<:((><
Just wanted to follow up on this, a month later: The speed issues I thought were coming from Test::Unit was actually coming from a lot of different issues relating to switching to Ruby 1.8.2 and using certain libs inefficiently. Test::Unit doesn't seem to be causing any problems, and it's definitely got a lot of nice features that make organizing my test cases easier. Thanks, Nathaniel!
One question I haven't been able to resolve: Is there a way to use multiple file args or fileglobs to have more than one test-case file loaded in at the command line? When I do something like
ruby test/article/ArticleIndexElement.rb test/article/ArticleSummary.rb
or just
ruby test/article/*.rb
then Test::Unit seems to just load in the first TestCase and drop everything else.
···
On Nov 9, 2004, at 8:38 PM, Nathaniel Talbott wrote:
On Nov 9, 2004, at 20:28, Francis Hwang wrote:
Let me ask more specifically: Is anybody out there using Test::Unit to run hundreds of tests? If you are, and have any tips as to how I can manage the speed issue, your tips would be much much appreciated.
Francis,
I've certainly had test suites with hundreds of tests, and never seemed to have problems. Have you been able to run a profiler on it? I'd be more than willing to apply patches, especially if they don't affect the API.
Nathaniel
Terralien, Inc.
<:((><
Francis Hwang