# Unit Testing Style

**URL:** <https://rubytalk.org/t/unit-testing-style/6214>\
**Category:** ruby-talk\
**Created:** [2 June 2003 20:08 UTC](https://rubytalk.org/t/unit-testing-style/6214 "2003-06-02T20:08:10Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Schwartz6](https://avatars.discourse-cdn.com/v4/letter/e/59ef9b/32.png) [@Eric\_Schwartz6](https://rubytalk.org/u/Eric_Schwartz6)\
**Post date:** [2 June 2003 20:08 UTC](https://rubytalk.org/t/unit-testing-style/6214/1 "2003-06-02T20:08:10Z")

</div>

I know I should have written my tests first, but I didn’t, so now I’m  
trying to do them after the fact. I’m running into a few style  
questions, though, not being used to the Test::Unit style of testing:

1. I have several test cases that do multiple assert\_\* calls. Is that  
a Good Thing, a Bad Thing, or not something that’s even worth  
worrying about? Usually it’s things like “Was the file created?  
Does it contain data of the right format? Is the data correct?”

2. How do people generally feel about interactivity in test cases? I  
have a method that should send email to a random address. Since  
I’d like everybody on my team to be able to run my tests, I can’t  
use a specific address, as that address might not (probably won’t)  
exist on their machine, and depending on which OS they’re running,  
the mail spool dirs will be in a different location.

Other than that, I’m enjoying Test::Unit quite a lot.

-=Eric

> **···**
>
> –  
> Come to think of it, there are already a million monkeys on a million  
> typewriters, and Usenet is NOTHING like Shakespeare.  
> – Blair Houghton.

---

<div class="post-metadata">

**Author:** ![Michael\_Campbell1](https://avatars.discourse-cdn.com/v4/letter/m/e274bd/32.png) [@Michael\_Campbell1](https://rubytalk.org/u/Michael_Campbell1)\
**Post date:** [2 June 2003 20:11 UTC](https://rubytalk.org/t/unit-testing-style/6214/2 "2003-06-02T20:11:13Z")

</div>

> 1. I have several test cases that do multiple assert\_\* calls. Is  
> that  
> a Good Thing, a Bad Thing, or not something that’s even worth  
> worrying about? Usually it’s things like “Was the file created?  
> Does it contain data of the right format? Is the data correct?”

In the general sense, I have a number of tests (in java, mind you)  
that have multiple asserts.

> **···**
>
> * * *
> 
> Do you Yahoo!?  
> Yahoo! Calendar - Free online calendar with sync to Outlook™.  
> [http://calendar.yahoo.com](http://calendar.yahoo.com)

---

<div class="post-metadata">

**Author:** ![Walter1](https://avatars.discourse-cdn.com/v4/letter/w/c77e96/32.png) [@Walter1](https://rubytalk.org/u/Walter1)\
**Post date:** [2 June 2003 20:26 UTC](https://rubytalk.org/t/unit-testing-style/6214/3 "2003-06-02T20:26:58Z")

</div>

see inline…

> I know I should have written my tests first, but I didn’t, so now I’m  
> trying to do them after the fact. I’m running into a few style  
> questions, though, not being used to the Test::Unit style of testing:
> 
> 1. I have several test cases that do multiple assert\_\* calls. Is that  
> a Good Thing, a Bad Thing, or not something that’s even worth  
> worrying about? Usually it’s things like “Was the file created?  
> Does it contain data of the right format? Is the data correct?”

It seems to me like you are trying to do too much in a single test.  
I would have tests for the item that you are testing remotely. Since  
those test would actually be locally, you can then break them down  
into small simple tests, and then when that is tested you only have  
to simulate your remote procedure call in a test.

> 1. How do people generally feel about interactivity in test cases? I  
> have a method that should send email to a random address. Since  
> I’d like everybody on my team to be able to run my tests, I can’t  
> use a specific address, as that address might not (probably won’t)  
> exist on their machine, and depending on which OS they’re running,  
> the mail spool dirs will be in a different location.

This too could be broken down. I have test that tests  
sending/reading of an email class. That test does in fact use a mail  
server. Once you have done that and you know that your email class  
works, it looks like you need a test that tests the random email  
address to send to. If you actually need to test sending an email,  
you can then use the class interface to simulate the sending of an  
email (Google mock object).

> My idea is to have the test ask for the address by stdin, and also  
> ask the user to verify the contents manually. The user’s response  
> will determine if the test passes or fails.

In general I think it is a bad idea to require input. One of the key  
things to automated unit tests, is that they are, well… automated.  
They just run and tell you success or failure. For me that’s why I  
like them.

> Other than that, I’m enjoying Test::Unit quite a lot.

cool.

Walt

> **···**
>
> * * *
> 
> Walter Szewelanczyk  
> IS Director  
> M.W. Sewall & CO. email : [walter@mwsewall.com](mailto:walter@mwsewall.com)  
> 259 Front St. Phone : (207) 442-7994 x 128  
> Bath, ME 04530 Fax : (207) 443-6284
> 
> * * *

---

<div class="post-metadata">

**Author:** ![Jim\_Weirich2](https://avatars.discourse-cdn.com/v4/letter/j/ecc23a/32.png) [@Jim\_Weirich2](https://rubytalk.org/u/Jim_Weirich2)\
**Post date:** [3 June 2003 01:30 UTC](https://rubytalk.org/t/unit-testing-style/6214/4 "2003-06-03T01:30:00Z")

</div>

> I know I should have written my tests first, but I didn’t, so now I’m  
> trying to do them after the fact. I’m running into a few style  
> questions, though, not being used to the Test::Unit style of testing:
> 
> 1. I have several test cases that do multiple assert\_\* calls. Is that  
> a Good Thing, a Bad Thing, or not something that’s even worth  
> worrying about? Usually it’s things like “Was the file created?  
> Does it contain data of the right format? Is the data correct?”

Just remember that the test method will terminate with the first failing  
assert\_\*. So if you have a lot of them in a single method, you could  
have multiple failures hiding behind a single failure.

> In one case, however, the test case is “run a command remotely that  
> logs to yet another remote database, and verify the command was  
> run, output was generated, logged to the right database with the  
> right timestamp, from the right machine, etc., etc., etc.”

I try to keep the external environment as simple and self contained as  
possible. I want to run the unit tests and KNOW everything is ok. If  
my tests depend on a lot of external resources, then failure may mean a  
problem with the code or a problem with the environment. I would rather  
spend time working on the code than tracing down environment problems.

Sometimes you DO need to talk to external resources (e.g. databases,  
servers, etc.). I usually isolate these tests in their own area  
(usually a separate Rake/Make target). Then I can run the unit tests to  
check the code as often as I want and get a lots of “feel good” karma.  
And then I can run the environment dependent tests separately to make  
sure I still can talk to the outside world.

> 1. How do people generally feel about interactivity in test cases?

Evil! Don’t do this! Never (at least in unit tests).

If you REALLY need some interactivity, I would move it outside the unit  
tests.

> Other than that, I’m enjoying Test::Unit quite a lot.

Great!

> **···**
>
> On Mon, 2003-06-02 at 16:08, Eric Schwartz wrote:
> 
> ## – – Jim Weirich [jweirich@one.net](mailto:jweirich@one.net)[http://jimweirich.umlcoop.net](http://jimweirich.umlcoop.net)
> 
> “Beware of bugs in the above code; I have only proved it correct,  
> not tried it.” – Donald Knuth (in a memo to Peter van Emde Boas)

---

<div class="post-metadata">

**Author:** ![Phlip5](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Phlip5](https://rubytalk.org/u/Phlip5)\
**Post date:** [3 June 2003 03:50 UTC](https://rubytalk.org/t/unit-testing-style/6214/5 "2003-06-03T03:50:09Z")

</div>

Eric Schwartz wrote:

> I know I should have written my tests first, but I didn’t, so now I’m  
> trying to do them after the fact.

The best way to do this (in my exalted opinion) is to point the tests at an  
empty project, use the testless project as a reference, and write tests that  
force you to copy tiny bits of code from the testless project into the empty  
project.

> I’m running into a few style  
> questions, though, not being used to the Test::Unit style of testing:
> 
> 1. I have several test cases that do multiple assert\_\* calls. Is that  
> a Good Thing, a Bad Thing, or not something that’s even worth  
> worrying about? Usually it’s things like “Was the file created?  
> Does it contain data of the right format? Is the data correct?”

Dave Astels (who I suspect wrote a book about TDD) and I pretended to have a  
vicious fight over this subject.

He claimed to only do one (1) assert per test case. I made him admit this is  
a goal, not a rule.

If you try to approach that goal, you will maximize re-use and encapsulation  
of your tests’ setup code. It grows into resplendent fixtures and resources  
in the test case.

I test GUIs, which optimize for displaying a window that I never give a  
chance to display. So the toolkit wastes a lot of cycles getting ready to  
display, and I throw all these cycles away. Hence, after every setup and  
test call, I pack in as many assertions as I can think of.

I defer to Astels’ style as (potentially) peer-reviewed, but I still think  
this topic awaits a more subtle metric or rule-of-thumb. I suggested 3  
assertions, to provide triangulation within R3 space.

> In one case, however, the test case is “run a command remotely that  
> logs to yet another remote database, and verify the command was  
> run, output was generated, logged to the right database with the  
> right timestamp, from the right machine, etc., etc., etc.”

Yup. Don’t run that one over and over again at test time.

If your tests get slower than the traditional time a UI event should take,  
split the folder they run in, and keep going. Put another way, the time  
tests take to run are a good metric for the number of source files you  
should keep in a folder.

> 1. How do people generally feel about interactivity in test cases? I  
> have a method that should send email to a random address. Since  
> I’d like everybody on my team to be able to run my tests, I can’t  
> use a specific address, as that address might not (probably won’t)  
> exist on their machine, and depending on which OS they’re running,  
> the mail spool dirs will be in a different location.

I hope nobody’s as tired of the following anecdote as I am of telling it:

The first project I ran full-bore test-first, it sent faxes. So I got the  
numbers of all the fax machines in our offices, and put them in my test  
resources. So, as I developed, every 30-90 seconds I’d hit the test button,  
and 20 to 40 bogus faxes would come out of a machine somewhere in the  
offices.

I came in one day and found they’d installed a new fax machine, in my cube.

I then signed up for an online free fax service, and sent it faxes. They  
came back as e-mails, each with an ad attached, of course. Then one night I  
left my tests running over a huge number of records, to test a new  
high-volume phone card they put on the server. I flooded the fax service  
with ~45,000 faxes in a couple hours, and they cancelled my account.

Uh… what was your question again?

> My idea is to have the test ask for the address by stdin, and also  
> ask the user to verify the contents manually. The user’s response  
> will determine if the test passes or fails.

Nix. Send to a common office account, or a Yahoo account, or something. Or  
register each tester, and let them put their favorite addy in a  
configuration file. Tests \>must\< run unattended. Everyone needs the minimum  
possible excuses not to run all the tests, all the time.

> Other than that, I’m enjoying Test::Unit quite a lot.

It’s a trip!

> **···**
>
> –  
> Phlip  
> [http://www.c2.com/cgi/wiki?TestFirstUserInterfaces](http://www.c2.com/cgi/wiki?TestFirstUserInterfaces)

---

<div class="post-metadata">

**Author:** ![Jim\_Freeze2](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@Jim\_Freeze2](https://rubytalk.org/u/Jim_Freeze2)\
**Post date:** [3 June 2003 02:15 UTC](https://rubytalk.org/t/unit-testing-style/6214/6 "2003-06-03T02:15:59Z")

</div>

So Jim, do you have a short example of using rake to  
manage unit tests?  
I am thinking about moving from make to rake so my tests  
will be environment independent (at least it will be easier  
to install rake than make).

> **···**
>
> On Tuesday, 3 June 2003 at 10:30:00 +0900, Jim Weirich wrote:
> 
> > Sometimes you DO need to talk to external resources (e.g. databases,  
> > servers, etc.). I usually isolate these tests in their own area  
> > (usually a separate Rake/Make target). Then I can run the unit tests to  
> > check the code as often as I want and get a lots of “feel good” karma.  
> > And then I can run the environment dependent tests separately to make  
> > sure I still can talk to the outside world.
> 
> ## – Jim Freeze
> 
> New systems generate new problems.

---

<div class="post-metadata">

**Author:** ![Eric\_Schwartz6](https://avatars.discourse-cdn.com/v4/letter/e/59ef9b/32.png) [@Eric\_Schwartz6](https://rubytalk.org/u/Eric_Schwartz6)\
**Post date:** [3 June 2003 16:52 UTC](https://rubytalk.org/t/unit-testing-style/6214/7 "2003-06-03T16:52:02Z")

</div>

“Phlip” [phlipcpp@yahoo.com](mailto:phlipcpp@yahoo.com) writes:

> Eric Schwartz wrote:
> 
> > I know I should have written my tests first, but I didn’t, so now I’m  
> > trying to do them after the fact.
> 
> The best way to do this (in my exalted opinion) is to point the tests at an  
> empty project, use the testless project as a reference, and write tests that  
> force you to copy tiny bits of code from the testless project into the empty  
> project.

Unfortunately, that works out to effectively rewriting the code from  
scratch, which is (alas) not terribly feasable at the moment.

> Dave Astels (who I suspect wrote a book about TDD) and I pretended to have a  
> vicious fight over this subject.
> 
> He claimed to only do one (1) assert per test case. I made him admit this is  
> a goal, not a rule.

That certainly was the goal I went for. But it seems to me that if  
I’ve got a method which is supposed to create a file with a specified  
format and data, isn’t it reasonable to first assert() the file  
exists, then to assert() its format is correct, and finally to  
assert() the data in it is what you expect? I’m not sure how one  
would make these separate tests.

Or rather, I can guess how you might do that, but it seems a bit  
absurd for the sake of correctness to run the exact same code three  
times just so you can say you didn’t run more than one assert() per  
test case.

> If you try to approach that goal, you will maximize re-use and encapsulation  
> of your tests’ setup code. It grows into resplendent fixtures and resources  
> in the test case.

Well, this was exactly my goal; it’s just that given the above  
situation, I didn’t see any (to my mind) reasonable way around it.

> I hope nobody’s as tired of the following anecdote as I am of telling it:

> Uh… what was your question again?

Heh. I can certainly create a mock object to pretend to be a close of  
TMail, which will solve 80% of my problem, but this still doesn’t  
answer two important questions:

1. Has the interface to tmail changed? How can I tell?
2. Is my mailserver config screwed up?

I think I care less about 2), because if the mailserver on my test  
host is screwed up, I’m going to have a number of problems that should  
shortly become obvious. I’m not sure how to test 1) without actually  
sending an email.

> Nix. Send to a common office account, or a Yahoo account, or  
> something.

But then, how can the test verify that the email got there? All it  
can know is if it thinks it was sent properly. Again though, given my  
previous paragraph, I think I most likely don’t care.

> Or register each tester, and let them put their favorite addy in a  
> configuration file.

And if the configuration file isn’t set up? I assume the test  
fails… and then you get into much harder problems, as someone could  
easily set up an email address in their configuration file that I  
could have no way of knowing how to check.

> Tests \>must\< run unattended. Everyone needs the minimum  
> possible excuses not to run all the tests, all the time.

I know, that’s why I _really_ didn’t like my ‘solution’, but couldn’t  
think of any way around it. But using a mock object for the email  
will solve most of my problems, as my concern when testing my TestItem  
class should not be whether or not the mailserver works, but whether  
TestItem is doing what it thinks it should.

> > Other than that, I’m enjoying Test::Unit quite a lot.
> 
> It’s a trip!

Yeah, it reminds me a bit of when I first learned OO-- it took a month  
or two to truly understand the difference between an object and a  
struct with function pointers. (Which is to say, on the one hand:  
none. On the other: everything.) If I force myself to write  
test-first in the future, it’ll be a lot easier in the long run.

-=Eric

> **···**
>
> –  
> Come to think of it, there are already a million monkeys on a million  
> typewriters, and Usenet is NOTHING like Shakespeare.  
> – Blair Houghton.

---

<div class="post-metadata">

**Author:** ![Jim\_Weirich2](https://avatars.discourse-cdn.com/v4/letter/j/ecc23a/32.png) [@Jim\_Weirich2](https://rubytalk.org/u/Jim_Weirich2)\
**Post date:** [3 June 2003 03:16 UTC](https://rubytalk.org/t/unit-testing-style/6214/8 "2003-06-03T03:16:50Z")

</div>

I was wondering if anyone would catch the Rake reference.

A dirt simple example follows. Assume all of your unit tests are in a  
directory named “test” and that your source code is in a directory named  
“lib”.

task :test do # Run the Unit Tests  
Sys.ruby %{-Ilib -Itest } +  
%{-e ‘Dir[“test/test\*.rb”].each { |fn| require fn }’}  
end

Truthfully, I’m still learning to use Rake effectively. I’ve been  
limited by Make for so long, it is taking me a while to see some of the  
possibilities.

For example, I’m playing around with a Builder object that is  
knowledgable about how a, for example, Ruby project is built. All you  
do is specify a few special directories or actions and the Builder will  
construct the make targets for you programmatically. I’m watching the  
discussion on “standard” project directories closely, because if we can  
standardize the directory structure, then more knowledge can be moved  
into the Builder.

> **···**
>
> On Mon, 2003-06-02 at 22:15, Jim Freeze wrote:
> 
> > On Tuesday, 3 June 2003 at 10:30:00 +0900, Jim Weirich wrote:
> > 
> > > […] (usually a separate Rake/Make target). […]
> > 
> > So Jim, do you have a short example of using rake to  
> > manage unit tests?  
> > I am thinking about moving from make to rake so my tests  
> > will be environment independent (at least it will be easier  
> > to install rake than make).
> 
> ## – – Jim Weirich [jweirich@one.net](mailto:jweirich@one.net)[http://onestepback.org](http://onestepback.org)
> 
> “Beware of bugs in the above code; I have only proved it correct,  
> not tried it.” – Donald Knuth (in a memo to Peter van Emde Boas)

---

<div class="post-metadata">

**Author:** ![Jim\_Freeze2](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@Jim\_Freeze2](https://rubytalk.org/u/Jim_Freeze2)\
**Post date:** [3 June 2003 03:39 UTC](https://rubytalk.org/t/unit-testing-style/6214/9 "2003-06-03T03:39:12Z")

</div>

> > > […] (usually a separate Rake/Make target). […]
> > 
> > So Jim, do you have a short example of using rake to  
> > manage unit tests?  
> > I am thinking about moving from make to rake so my tests  
> > will be environment independent (at least it will be easier  
> > to install rake than make).
> 
> I was wondering if anyone would catch the Rake reference.
> 
> A dirt simple example follows. Assume all of your unit tests are in a  
> directory named “test” and that your source code is in a directory named  
> “lib”.
> 
> task :test do # Run the Unit Tests  
> Sys.ruby %{-Ilib -Itest } +  
> %{-e ‘Dir[“test/test\*.rb”].each { |fn| require fn }’}  
> end

Thanks for the example.

> Truthfully, I’m still learning to use Rake effectively. I’ve been  
> limited by Make for so long, it is taking me a while to see some of the  
> possibilities.
> 
> For example, I’m playing around with a Builder object that is  
> knowledgable about how a, for example, Ruby project is built. All you  
> do is specify a few special directories or actions and the Builder will  
> construct the make targets for you programmatically. I’m watching the  
> discussion on “standard” project directories closely, because if we can  
> standardize the directory structure, then more knowledge can be moved  
> into the Builder.

I think the Standard Project Directories discussion has died  
down. Either we reached a consensus, or wore everyone out. 😉  
Below is the final state it reached:

> **···**
>
> On Tuesday, 3 June 2003 at 12:16:50 +0900, Jim Weirich wrote:
> 
> > On Mon, 2003-06-02 at 22:15, Jim Freeze wrote:
> > 
> > > On Tuesday, 3 June 2003 at 10:30:00 +0900, Jim Weirich wrote:
> 
> * * *
> 
> Below is a suggested directory hierarchy.  
> All of the directories, except possibly test/ ;),  
> are optional. Most of the filenames listed  
> are optional but are listed for consistent  
> spelling and capitalization.
> 
> CHANGELOG  
> INSTALL  
> LICENSE  
> Manifest  
> README # if multi lang, README.en, README.ja, etc  
> TODO # optional  
> install.rb # place holder  
> setup.rb # place holder  
> bin/  
> # ruby executables  
> # other executables  
> contrib/  
> # third party contributions  
> doc/ # If multi lang, then use doc.en, doc.ja, etc  
> man/ # for unix man  
> api/ # rdoc output  
> rd/ # if rd based docs  
> tutorial/  
> example/  
> ext/  
> ext\_name/  
> extconf.rb  
> … other source  
> lib/  
> # .rb files that go into RUBY\_LIB  
> # may want to use parallel directory structure below  
> # site\_ruby//  
> src/  
> \<other\_lang\_name\>/  
> test/  
> # test suites. Best if dirs mirror lib/.
> 
> ## – Jim Freeze
> 
> Common sense is instinct, and enough of it is genius.  
> – Josh Billings