As I've watched the uses of test/unit grow and change over time, I've also considered adding more ability to run tests in specific orders... however, this is because I see test/unit being used in various places at an acceptance (or customer) testing level, where those needs are much more legitimate. It's a challenge to introduce something like that, though, and still encourage good unit testing practices. Perhaps it's time for test/accept?
Forgive me if this has already been asked and answered, but what exactly *are* good unit testing practices, and how should they be implemented? I've been trying to use test/unit more and more, and to be honest I keep running into situations where I would like tests to be run in a specific order. If I have understood the comments in this thread, it sounds like several of you think that this is a bad thing... I'm having trouble seeing how else to do it, I guess.
Competition doing its job of making software better. =)
Sean O'Dell
···
On Sunday 13 June 2004 07:34, Nathaniel Talbott wrote:
Anyhow, I'm glad that Sean has released this new library... it will
only make test/unit better (I've already gleaned a few ideas from it).
And when I get a chance to overhaul the test/unit documentation, part
of the overhaul will be an attempt to communicate not just what
test/unit does, but also what it doesn't do, and why.
There is no rhyme, reason or principal determining the order; it was decided by the author at his prerogative.
I think David's point was that the order was decided after much community discussion and end-user participation, not on some developer's whim. Perhaps you disagree, or are unfamiliar, with the reasons for selecting this order, but that's not to say none exist.
If, after you've looked into the history of the code, you still find the order arbitrary, then there's probably not much more anyone can say, except perhaps that most users don't find the order arbitrary at all.
But, of course, maybe they are all wrong and just haven't been enlightened.
I suspect that there is some misunderstanding here. That you would
call the order "arbitrary" conveys some negative connotation that
you certainly want to avoid not to appear rude or condescending.
In France a typical "arbitrary" decision was King's one to put you
on jail (lettre de cachet). As a result "arbitrary" tends to
mean "lack of respect".
I also suspect that the decision to "order" tests is not at all
arbitrary compared to the decision not to sort tests. I tend to
think that it is a good thing to order things and alphabetic order
is a reasonable choice that is easy to understand by everybody.
Once things are ordered, you can take advantage of that and for
example name your test AAA until they first work. That way you
get early results. Once your test passes, you then rename it in a
more understandable way. That's just a trick, but it helps, and
would not be possible if there was no documented order.
For sure, it is always a good thing to give rationals and if
the "ordering" was not documented properly then that could be
improved. Something that seems arbitrary at first look may
turn out to be solid once understood. This requires both some
help from the author and some good will of the reader.
Yours,
JeanHuguesRobert
···
At 08:12 13/06/2004 +0900, you wrote:
On Saturday 12 June 2004 13:34, David A. Black wrote:
> arbitrary:
> Determined by chance, whim, or impulse, and not by necessity, reason, or
> principle.
>
> There is no fundamental reason tests should be ordered alphabetically.
Maybe, maybe not -- but in either case, I was addressing your original
claim, which was not that choosing alphabetical order over other designs
was arbitrary, but rather that in Nathaniel's design, the tests were
run in arbitrary (i.e., determined by chance, whim, etc.) order. They
aren't, so I wanted to let you and other readers know.
This had (and has) nothing to do with the processing of choosing
alphabetical order (which you introduced into the discussion later),
but rather with the fact that alphabetical order is not, itself,
arbitrary -- and, concommitantly, with the fact that Nathaniel didn't
deserve to have his design misrepresented as involving something it
doesn't.
When you're given no choice, as far as you're concerned as a library user,
they're run in an arbitrary order. The fact that it turns out they are run
alphabetically doesn't change the fact that the system of ordering is
arbitrary. There is no rhyme, reason or principal determining the order; it
was decided by the author at his prerogative. If the tests were run
according to the length of the method names, that would still be an arbitrary
order. As far as I'm concerned, the order that tests are run is a black box.
I don't think people think its a bad thing, people just don't like when people
come along and criticize their beloved development framework, and I've done
that. They're just being defensive. They know running tests in a controlled
order is the right thing to do, they just don't like it when people point out
a library they love doesn't do this or that. I find a lot of things I ask
for in Ruby gets responses like that. Something doesn't do this or that, so
people start arguing how perhaps it shouldn't be doing that in the first
place. A lot of people love Ruby so much, they're trying to fantasize that
it's is as perfect as can be, and they don't like people pointing out cracks.
It's pretty normal, I think. Frustrating, but normal.
Sean O'Dell
···
On Sunday 13 June 2004 07:52, Jamis Buck wrote:
Nathaniel Talbott wrote:
> As I've watched the uses of test/unit grow and change over time, I've
> also considered adding more ability to run tests in specific orders...
> however, this is because I see test/unit being used in various places at
> an acceptance (or customer) testing level, where those needs are much
> more legitimate. It's a challenge to introduce something like that,
> though, and still encourage good unit testing practices. Perhaps it's
> time for test/accept?
Forgive me if this has already been asked and answered, but what exactly
*are* good unit testing practices, and how should they be implemented?
I've been trying to use test/unit more and more, and to be honest I keep
running into situations where I would like tests to be run in a specific
order. If I have understood the comments in this thread, it sounds like
several of you think that this is a bad thing... I'm having trouble
seeing how else to do it, I guess.
Ordering tests alphabetically, to me, is arbitrary. I can't say I really know
how many people got involved in the decision to order tests alphabetically,
but I can say that whether it's one or a thousand, the method makes zero
sense to me and it's arbitrary as far as I'm concerned.
I am about to put the word "arbitrary" back into every single
celsoft.com/Battery document I've written if people refuse to respect my
opinion in the matter and continue to berate me over it. I took it out
initially trying to avoid hurting anyone's feelings, but since it seems
apparent that this discussion is going to go on and on, I feel almost
compelled to hold up the word on a billboard. I used the word properly, and
I still meant what I said. I'm starting to feel sorry for taking the word
out of the docs.
Sean O'Dell
···
On Sunday 13 June 2004 00:13, Jean-Hugues ROBERT wrote:
At 08:12 13/06/2004 +0900, you wrote:
>
>When you're given no choice, as far as you're concerned as a library user,
>they're run in an arbitrary order. The fact that it turns out they are
> run alphabetically doesn't change the fact that the system of ordering is
> arbitrary. There is no rhyme, reason or principal determining the order;
> it was decided by the author at his prerogative. If the tests were run
> according to the length of the method names, that would still be an
> arbitrary order. As far as I'm concerned, the order that tests are run
> is a black box.
I suspect that there is some misunderstanding here. That you would
call the order "arbitrary" conveys some negative connotation that
you certainly want to avoid not to appear rude or condescending.
In France a typical "arbitrary" decision was King's one to put you
on jail (lettre de cachet). As a result "arbitrary" tends to
mean "lack of respect".
I also suspect that the decision to "order" tests is not at all
arbitrary compared to the decision not to sort tests. I tend to
think that it is a good thing to order things and alphabetic order
is a reasonable choice that is easy to understand by everybody.
It is the worst case of over self-confidence and stubbornness I have ever seen. Simply amusing. I just want to point out that it is not only in Ruby that unit testing frameworks strongly discourage dependency on test order. Kent Beck in his original work took considerable amount of time talking about importance of individual tests being independent. Sometimes I even get into trouble because of alphabetical ordering we have now in Test::Unit. So I would be happy if Nathaniel finally decides to implement random test invocation on every run.
Sincerely,
Gennady Bystritsky
···
On Jun 14, 2004, at 9:22, Sean O'Dell wrote:
On Sunday 13 June 2004 07:52, Jamis Buck wrote:
Nathaniel Talbott wrote:
As I've watched the uses of test/unit grow and change over time, I've
also considered adding more ability to run tests in specific orders...
however, this is because I see test/unit being used in various places at
an acceptance (or customer) testing level, where those needs are much
more legitimate. It's a challenge to introduce something like that,
though, and still encourage good unit testing practices. Perhaps it's
time for test/accept?
Forgive me if this has already been asked and answered, but what exactly
*are* good unit testing practices, and how should they be implemented?
I've been trying to use test/unit more and more, and to be honest I keep
running into situations where I would like tests to be run in a specific
order. If I have understood the comments in this thread, it sounds like
several of you think that this is a bad thing... I'm having trouble
seeing how else to do it, I guess.
I don't think people think its a bad thing, people just don't like when people
come along and criticize their beloved development framework, and I've done
that. They're just being defensive. They know running tests in a controlled
order is the right thing to do, they just don't like it when people point out
a library they love doesn't do this or that. I find a lot of things I ask
for in Ruby gets responses like that. Something doesn't do this or that, so
people start arguing how perhaps it shouldn't be doing that in the first
place. A lot of people love Ruby so much, they're trying to fantasize that
it's is as perfect as can be, and they don't like people pointing out cracks.
It's pretty normal, I think. Frustrating, but normal.
I am about to put the word "arbitrary" back into every single celsoft.com/Battery document I've written if people refuse to respect my opinion in the matter and continue to berate me over it. I took it out initially trying to avoid hurting anyone's feelings, but since it seems apparent that this discussion is going to go on and on, I feel almost compelled to hold up the word on a billboard. I used the word properly, and I still meant what I said. I'm starting to feel sorry for taking the word out of the docs.
Picking a different word to avoid hurt feelings is probably not a good goal; picking better words to improve accuracy might be.
Seems some people don't understand your insistence on using that particular word, which apparently makes its use arbitrary; of course, you are free to, um, arbitrarily insert that word anyplace you like, though I doubt it will help anyone's comprehension.
Anyway, I think this is the point where somebody quotes Humpty Dumpty, then someone else calls Mr. Dumpty a Nazi, and then the thread is arbitrarily ended.
although i would concede that some __other__ hacky method could have been
chosen and, in that sense, it is arbitrary. however, strictly speaking there
was a rational behind the choice so it cannot accurately be called arbitrary.
i suppose the point you are making is that - so long as some ordering method
exists it's implementation is not important, which i would agree with.
cheers.
-a
···
On Tue, 15 Jun 2004, Sean O'Dell wrote:
Ordering tests alphabetically, to me, is arbitrary. I can't say I really know
how many people got involved in the decision to order tests alphabetically,
but I can say that whether it's one or a thousand, the method makes zero
sense to me and it's arbitrary as far as I'm concerned.
I am about to put the word "arbitrary" back into every single
celsoft.com/Battery document I've written if people refuse to respect my
opinion in the matter and continue to berate me over it. I took it out
initially trying to avoid hurting anyone's feelings, but since it seems
apparent that this discussion is going to go on and on, I feel almost
compelled to hold up the word on a billboard. I used the word properly, and
I still meant what I said. I'm starting to feel sorry for taking the word
out of the docs.
Sean O'Dell
--
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it; and a weed grows, even though we do
not love it. --Dogen
I am about to put the word "arbitrary" back into every single celsoft.com/Battery document I've written if people refuse to respect my opinion in the matter and continue to berate me over it. I took it out initially trying to avoid hurting anyone's feelings, but since it seems apparent that this discussion is going to go on and on, I feel almost compelled to hold up the word on a billboard. I used the word properly, and I still meant what I said. I'm starting to feel sorry for taking the word out of the docs.
Relax, this isn't a war.
I can see how 1) unit tests "ought" not to rely on an order but 2) we
might sometimes want to break that rule and control the ordering.
But half of my interest in this thread is seeing how various people
think and use words.
Would you say that the ordering of the words in a dictionary is
arbitrary?
Sometimes I even get into trouble because of alphabetical ordering we have now in Test::Unit. So I would be happy if Nathaniel finally decides to implement random test invocation on every run.
Meaning what, that Nathaniel didn't post saying he found some good ideas in my
library and would be adding some of them to his own? Where is the
over-confidence.
I think what we have is more than one person who is annoyed that my ideas are
not something they can just swat down, so long as I can code them myself.
That's honestly what I think, that people are EXTRA annoyed that I went and
did it for myself. It took the power away from you, didn't it? You can't
tell someone "no, we won't do that" when they can just go do it themselves.
Petty. I can't believe how petty some of you are.
Sean O'Dell
···
On Monday 14 June 2004 11:07, Gennady wrote:
On Jun 14, 2004, at 9:22, Sean O'Dell wrote:
> On Sunday 13 June 2004 07:52, Jamis Buck wrote:
>> Nathaniel Talbott wrote:
>>> As I've watched the uses of test/unit grow and change over time, I've
>>> also considered adding more ability to run tests in specific
>>> orders...
>>> however, this is because I see test/unit being used in various
>>> places at
>>> an acceptance (or customer) testing level, where those needs are much
>>> more legitimate. It's a challenge to introduce something like that,
>>> though, and still encourage good unit testing practices. Perhaps it's
>>> time for test/accept?
>>
>> Forgive me if this has already been asked and answered, but what
>> exactly
>> *are* good unit testing practices, and how should they be implemented?
>> I've been trying to use test/unit more and more, and to be honest I
>> keep
>> running into situations where I would like tests to be run in a
>> specific
>> order. If I have understood the comments in this thread, it sounds
>> like
>> several of you think that this is a bad thing... I'm having trouble
>> seeing how else to do it, I guess.
>
> I don't think people think its a bad thing, people just don't like
> when people
> come along and criticize their beloved development framework, and I've
> done
> that. They're just being defensive. They know running tests in a
> controlled
> order is the right thing to do, they just don't like it when people
> point out
> a library they love doesn't do this or that. I find a lot of things I
> ask
> for in Ruby gets responses like that. Something doesn't do this or
> that, so
> people start arguing how perhaps it shouldn't be doing that in the
> first
> place. A lot of people love Ruby so much, they're trying to fantasize
> that
> it's is as perfect as can be, and they don't like people pointing out
> cracks.
> It's pretty normal, I think. Frustrating, but normal.
>
> Sean O'Dell
It is the worst case of over self-confidence and stubbornness I have
ever seen. Simply amusing. I just want to point out that it is not only
in Ruby that unit testing frameworks strongly discourage dependency on
test order. Kent Beck in his original work took considerable amount of
time talking about importance of individual tests being independent.
Sometimes I even get into trouble because of alphabetical ordering we
have now in Test::Unit. So I would be happy if Nathaniel finally
decides to implement random test invocation on every run.
Even Nathaniel himself admitted the decision to order tests alphabetically was
an arbitrary one. Accuracy is 100%. Annoyance at having this discussion is
somewhere above that.
Sean O'Dell
···
On Monday 14 June 2004 09:27, James Britt wrote:
Sean O'Dell wrote:
> I am about to put the word "arbitrary" back into every single
> celsoft.com/Battery document I've written if people refuse to respect my
> opinion in the matter and continue to berate me over it. I took it out
> initially trying to avoid hurting anyone's feelings, but since it seems
> apparent that this discussion is going to go on and on, I feel almost
> compelled to hold up the word on a billboard. I used the word properly,
> and I still meant what I said. I'm starting to feel sorry for taking the
> word out of the docs.
Picking a different word to avoid hurt feelings is probably not a good
goal; picking better words to improve accuracy might be.
No, because there is a solid reason for the order of the words in the
dictionary, and that decision makes sense to me. It could appear arbitrary
to SOME people, but I think only those in the fringes of society (aka, crazy,
fanatical, etc.). Since I easily implemented test ordering, and can think of
many reasons to not be forced into running them alphabetically, yet Test/Unit
didn't give me the option, that's clearly arbitrary. It was imposed on me,
there was no good reason, etc. Lots of qualities make "arbitrary" a great
way to describe the ordering.
Sean O'Dell
···
On Monday 14 June 2004 09:57, Hal Fulton wrote:
Sean O'Dell wrote:
> I am about to put the word "arbitrary" back into every single
> celsoft.com/Battery document I've written if people refuse to respect my
> opinion in the matter and continue to berate me over it. I took it out
> initially trying to avoid hurting anyone's feelings, but since it seems
> apparent that this discussion is going to go on and on, I feel almost
> compelled to hold up the word on a billboard. I used the word properly,
> and I still meant what I said. I'm starting to feel sorry for taking the
> word out of the docs.
Relax, this isn't a war.
I can see how 1) unit tests "ought" not to rely on an order but 2) we
might sometimes want to break that rule and control the ordering.
But half of my interest in this thread is seeing how various people
think and use words.
Would you say that the ordering of the words in a dictionary is
arbitrary?
James, you arbitrarily picked the * character for emphasis. I shall
provide a patch that uses the _far_ superior _ character.
Yours,
Tom
···
On Mon, 2004-06-14 at 14:23, James Britt wrote:
Gennady wrote:
>>
>
...
> Sometimes I even get into trouble because of alphabetical ordering we
> have now in Test::Unit. So I would be happy if Nathaniel finally decides
> to implement random test invocation on every run.
Arbitrary isn't an ordering method per se. Alphabetic can be arbitrary, as
can random. You can say alphabetic and arbitrary and be talking about the
same thing. Any method used that is arrived at imperiously and without solid
reason or principal is arbitrary.
Sean O'Dell
···
On Monday 14 June 2004 11:23, James Britt wrote:
Gennady wrote:
...
> Sometimes I even get into trouble because of alphabetical ordering we
> have now in Test::Unit. So I would be happy if Nathaniel finally decides
> to implement random test invocation on every run.
I personally am EXTRA glad that you took the initiative to go and code it yourself. I am getting EXTRA annoyed at your attitude and tone, though. I am worried that many will not look at or use your contributions simply because of your demeanor on this list, and I think that would be a loss.
Nathaniel
Terralien, Inc.
<:((><
···
On Jun 14, 2004, at 13:25, Sean O'Dell wrote:
I think what we have is more than one person who is annoyed that my ideas are
not something they can just swat down, so long as I can code them myself.
That's honestly what I think, that people are EXTRA annoyed that I went and
did it for myself. It took the power away from you, didn't it? You can't
tell someone "no, we won't do that" when they can just go do it themselves.
I am about to put the word "arbitrary" back into every single
celsoft.com/Battery document I've written if people refuse to respect my
opinion in the matter and continue to berate me over it. I took it out
initially trying to avoid hurting anyone's feelings, but since it seems
apparent that this discussion is going to go on and on, I feel almost
compelled to hold up the word on a billboard. I used the word properly,
and I still meant what I said. I'm starting to feel sorry for taking the
word out of the docs.
Picking a different word to avoid hurt feelings is probably not a good
goal; picking better words to improve accuracy might be.
Even Nathaniel himself admitted the decision to order tests alphabetically was an arbitrary one. Accuracy is 100%. Annoyance at having this discussion is somewhere above that.
I think I'm seeing the root of the disagreement (taking off my hacker
hat and putting on my Amateur Logician/Linguist Hat).
I think the decision to use alphabetical order may be an arbitrary
decision. But the ordering itself is anything but arbitrary. A is
followed by B is followed by C, and so on.
As I implied earlier, I don't think most of us would say that the words
in a dictionary are in "arbitrary" order.
Getting back to your dictionary definition, there is a definite reason
or principal in the operation of the ordering itself; but not
necessarily in the decision to use that particular ordering.
But me, I'm just a natural nitpicker. I think it's genetic.