<ANN> TeSLa, a Domain Specific Language for Unit Testing

ruby-talk
1

How does the TeSLa version know what :item4 is for, or where
it goes, or what 'size' refers to? The example has no
reference to Catalog; how does the code know to test that class?

I know nothing about TeSLA, but I can make a guess.

test_method :add_item => [:item4] do
  requires {@items = [:item1, :item2, :item3]}
  assert {size == 4}
end

This creates a testcase that will call the "add_item" method on an
instance of the class (I'm guessing that you define test_method within
the context of a class). Before calling the method, it will set the
instance variable @items to the value [:item1, :item2, :item3]. It will
then pass that arguments [:item4] to the method, and afterwards call the
size method and ensure that it equals 4. It does the equivalent of:

c = Catalogue.new
c.instance_eval { @items = [:item1, :item2, :item3] }
c.instance_eval { send(:add_item, :item4) }
raise unless c.instance_eval { size == 4 }

Or, that's my impression....

···

-----Original Message-----
From: James Britt [mailto:james_b@neurogami.com]
Sent: Monday, 31 October 2005 4:41 PM
To: ruby-talk ML
Subject: Re: <ANN> TeSLa, a Domain Specific Language for Unit Testing

javierg1975@gmail.com wrote:
> Hi
> I just posted version 0.1.0 of TeSLa, a Domain Specific
Language (DSL)
> for Unit Testing.
> You can download TeSLa along with a small example script from
> http://theniceweb.com/projects/tesla/tesla.zip (zip) or here
> http://theniceweb.com/projects/tesla/tesla.tar.gz (tar.gz) I also
> posted a small article/tutorial explaining the rationale and use of
> TeSLa here
>
http://www.theniceweb.com/JaviersBlog/2005/10/tesla-test-specific-lang
> uage-for-ruby.html

This looks quite interesting. Two questions:

1. How does this compare to behavior-driven development?

http://daveastels.com/index.php?p=5

2. The TeSLa docs give this example, comparing Test::Unit and TesLa:

class TestCatalog < Test::Unit::TestCase
  def test_add_item
   catalog = Catalog.new()
   catalog.items = [:item1, :item2, :item3]
   catalog.add_item :item4
   assert_equal(catalog.size, 4, "length should be 4")
  end
end

would look like this in TeSLa

test_method :add_item => [:item4] do
  requires {@items = [:item1, :item2, :item3]}
  assert {size == 4}
end

How does the TeSLa version know what :item4 is for, or where
it goes, or what 'size' refers to? The example has no
reference to Catalog; how does the code know to test that class?

Thanks,

James
--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for
Ruby & XML http://www.rubystuff.com - The Ruby Store for Ruby
Stuff http://www.jamesbritt.com - Playing with Better Toys

#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################

2

Well, it doesn't quite works that way (the implementation, that is... I
tried to avoid using 'eval' as long as I could) but yes, all references
to properties or methods are to an instance of the class where the
tests are specified. If you take a look at the example.rb script
included in the package, you'll notice

test_method :add_item => [:item4] do
   requires {@items = [:item1, :item2, :item3]}
   assert {size == 4}
end

is indeed declared just bellow the 'add_item' method in class Catalog
and that 'items' is a property of that class. 'test_method' will create
a clean instance of class Catalog and execute the 'add_item' method
with :item4 as a parameter and 'assert' will check that 'size' (of the
Catalog instance) is indeed 4 as long of the 'precondition' stated by
requires was met.
As for behavior-driven development, I never knew it existed, thanks a
lot for the link, I'm going to have a look at it right away.
By the way, I never intended for TeSLa to be a "pure" unit testing
framework, maybe even include concepts from Eiffel's Design By Contract
(so far that's just an idea I'm toying with) so let me know of any
interesting concept you might have about testing in general and I'll
take a look at it.
Again, let me know of ways you think the syntax might be more
readable/intuitive (even though I know that's a subject where people
will never totally agree)
Thanks a lot for the feedback

3

obscured by code wrote:

Well, it doesn't quite works that way (the implementation, that is... I
tried to avoid using 'eval' as long as I could) but yes, all references
to properties or methods are to an instance of the class where the
tests are specified. If you take a look at the example.rb script
included in the package, you'll notice

test_method :add_item => [:item4] do
   requires {@items = [:item1, :item2, :item3]}
   assert {size == 4}
end

is indeed declared just bellow the 'add_item' method in class Catalog
and that 'items' is a property of that class. 'test_method' will create
a clean instance of class Catalog and execute the 'add_item' method
with :item4 as a parameter and 'assert' will check that 'size' (of the
Catalog instance) is indeed 4 as long of the 'precondition' stated by
requires was met.

Ah. Thank you for the clarification. I often put my unit tests in the same source file as the class code, and being able to have tests right next to methods is quite slick.

As for behavior-driven development, I never knew it existed, thanks a
lot for the link, I'm going to have a look at it right away.
By the way, I never intended for TeSLa to be a "pure" unit testing
framework, maybe even include concepts from Eiffel's Design By Contract
(so far that's just an idea I'm toying with) so let me know of any
interesting concept you might have about testing in general and I'll
take a look at it.

I'm interested in the idea of having executable comments, so that the rdoc for a method both renders as examples for the reader, and can be run as unit tests. I have no idea how your work might fit into that, though. One out-of-the-blue idea: an rdoc directive call that grabs the TeSLa code and translates it into an example using the normal calling syntax.

So,

   test_method :add_item => [:item4] do
     requires {@items = [:item1, :item2, :item3]}
     assert {size == 4}
   end

would become, in the rendered method docs,

c = Catalog.new
c.items = [:item1, :item2, :item3]
c.add_item( :item4 )
Expected results: c.size == 4

Or something.

Thanks,

James Britt

···

--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for Ruby & XML
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys

4

I like this direction.

obscured by code wrote:

test_method :add_item => [:item4] do
   requires {@items = [:item1, :item2, :item3]}
   assert {size == 4}
end

is indeed declared just bellow the 'add_item' method in class Catalog
and that 'items' is a property of that class. 'test_method' will create
a clean instance of class Catalog and execute the 'add_item' method
with :item4 as a parameter and 'assert' will check that 'size' (of the
Catalog instance) is indeed 4 as long of the 'precondition' stated by
requires was met.

So what part of the code makes {@items = [:...]} true?

You might also find it useful to look at ruby-contract:
http://ruby-contract.rubyforge.org/wiki/wiki.pl?HomePage

5

No part of the code makes that line true, that line is a precondition
to the success of the assert once the method is run. That is,
  requires {@items = [:item1, :item2, :item3]}
is executed *before*
  add_item => [:item4]
Look at it as initialization code if you like. In other words, it is
*required* for the 'items' property to hold items 1 to 3 so the
addition of a fourth one would make the
  assert {size == 4}
line true.
Below is the entire Catalog class from example.rb. Perhaps that would
help make things clearer

class ShoppingCart

  def add_item item
    @items << item
   end

  def size
    @items.length
  end

  attr_accessor :items

  test_method :add_item => [:item5] do

    # requires makes sure certain conditions are met before the
                # assert is run

    requires {@items = [:item1, :item2, :item3]}
    assert {size == 4}
  end
  
end

6

"obscured by code" <javierg1975@gmail.com> wrote in message

No part of the code makes that line true, that line is a precondition
to the success of the assert once the method is run.

A precondition is a check with no side effects.

requires {@items = [:item1, :item2, :item3]}
is executed *before*
add_item => [:item4]
Look at it as initialization code if you like.

Initialization code will set the @items variable, right?

So your "requires ..." is an executed statement that has the effect of
making the precondition true, not just a boolean test, correct?

If so, it makes sense. If not, I am still a bit lost.

7

Well, I think your confusion stems from my use of the word
"precondition". Thinking back, I realize it might have been better if
I'd have used a different wording.
In Eiffel a precondition is indeed (and I quote from Meyer's OOSC) "The
boolean expression which defines the domain", but in this thread I've
been using the word in the sense of:
"A condition that must exist or be established before something can
occur or be considered; a prerequisite."
The word 'requires' in the code is therefore not used in the sense of
the Eiffel keyword but rather to imply that what follows it is a
*requirement* for 'assert' to succeed. In plain English, the following
segment of code

test_method :add_item => [:item4] do
  requires {@items = [:item1, :item2, :item3]}
  assert {size == 4}
end

should be read as, "it is required for the 'items' property of any
Catalog instance to hold three items if the condition of 'size' being 4
is to be met after the method 'add_item' is called with an 'item' as
parameter".
All this leads me to think maybe a better syntax is, well... required
(no pun intended). Any suggestions regarding this will be very much
welcomed.
As an aside note, I've been reading Dave Astels' proposal of a Behavior
Driven approach (thanks James) and tend to agree with much of what he
proposes. Perhaps I could use some ideas (and syntax) from BDD to make
TeSLa more readable/intuitive. I'd still use Test::Unit underneath,
though, as maintaining tool integration is still very important to me.
Again, I'm very open to suggestions.

8

test_method :add_item => [:item4] do
        requires {@items = [:item1, :item2, :item3]}
        assert {size == 4}
end

...

Again, I'm very open to suggestions.

What about:

test_method :add_item => [:item4] do
        before {@items = [:item1, :item2, :item3]}
        after {size == 4}
end

Or variations thereof.

Ryan

···

On 11/1/05, obscured by code <javierg1975@gmail.com> wrote:

9

Ryan Leavengood wrote:

What about:

test_method :add_item => [:item4] do
        before {@items = [:item1, :item2, :item3]}
        after {size == 4}
end

Or variations thereof.

Such as

test_method :add_item => [:item4] do
    given {@items = [:item1, :item2, :item3]}
    expected {size == 4}
end

James

···

--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for Ruby & XML
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys

10

You could probably get off with the "assert" used already, ala:

test_method :add_item => [:item4] do
    given {@items = [:item1, :item2, :item3]}
    assert {size == 4}
end

Or:

test :adding_an_item do
    given { @items = [:item1, :item2, :item3] }
    and_on { add_item :item4 }
    assert { size == 4 }
end

Where and_on is much the same as given.

Douglas

···

2005/11/2, James Britt <james_b@neurogami.com>:

Such as

test_method :add_item => [:item4] do
    given {@items = [:item1, :item2, :item3]}
    expected {size == 4}
end

11

James Britt <james_b@neurogami.com> writes:

Ryan Leavengood wrote:

What about:
test_method :add_item => [:item4] do
        before {@items = [:item1, :item2, :item3]}
        after {size == 4}
end
Or variations thereof.

Such as

test_method :add_item => [:item4] do
    given {@items = [:item1, :item2, :item3]}
    expected {size == 4}
end

Continuing the brainstorm (I used "assuming" before "given", but that
reads better):

given { @items = [:item1, :item2, :item3] }.expect {
  add_item_should_increase_size {
    given {
      @previous_size = size
    }
    doing { add_item :item4 }
    results_in { size > @previous_size }
  }

  add_item_should_increase_size_by_one {
    doing { add_item :item4 }
    results_in { size == 4 }
  }
}

I'm still not sure whether instance_eval'ing tests is a good idea.

···

James

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

12

That's my concern too. I'm not sure I trust the validity of of the tests
since they are applied in a way that they are not used. This type of testing
also ties the tests to a particular implementation. If you change the
implementation (not the functionality), you'll likely have to change the
testing since you are directly setting and accessing instance variables.

On the other hand, with some types of objects, you may not want to expose
very much internal state at the API and this type of testing may make it
much easier. I can draw an analogy from this to the hardware world - DFT
(design for test) and ATPG (automatic test pattern generation) - where it is
common practice to add control and observe points for testing (most state
points at least). That is testing for defects, but even for functional tests
it is common to at least have observability at some internal points in a
chip. BTW, this is where I draw many of my software testing ideas from. I
think the software world can learn some things from the hardware world in
terms of testing.

···

On 11/3/05, Christian Neukirchen <chneukirchen@gmail.com> wrote:

I'm still not sure whether instance_eval'ing tests is a good idea.

13

Eric Mahurin <eric.mahurin@gmail.com> writes:

I'm still not sure whether instance_eval'ing tests is a good idea.

That's my concern too. I'm not sure I trust the validity of of the tests
since they are applied in a way that they are not used. This type of testing
also ties the tests to a particular implementation. If you change the
implementation (not the functionality), you'll likely have to change the
testing since you are directly setting and accessing instance variables.

Also, how do you develop test-first with non-existing instance variables?

At least, one should make two groups of test, one that test
"blackbox", and one that test the actual implementation. I'd write
the blackbox test first, and the whitebox tests in parallel to the
implementation. That way, you can oversee the programming and still
test for implementation details. The latter tests will need to be
thrown away on a reimplementation, of course.

On the other hand, with some types of objects, you may not want to expose
very much internal state at the API and this type of testing may make it
much easier. I can draw an analogy from this to the hardware world - DFT
(design for test) and ATPG (automatic test pattern generation) - where it is
common practice to add control and observe points for testing (most state
points at least). That is testing for defects, but even for functional tests
it is common to at least have observability at some internal points in a
chip. BTW, this is where I draw many of my software testing ideas from. I
think the software world can learn some things from the hardware world in
terms of testing.

I recently searched a bit about "behavior driven design" and found a
paper on "spec driven design"---which turned out to be about hardware.

···

On 11/3/05, Christian Neukirchen <chneukirchen@gmail.com> wrote:

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

14

Here are a couple:
http://blog.daveastels.com/files/sdbp2005/BDD%20Intro%20slides.pdf
http://blog.daveastels.com/files/sdbp2005/BDD%20Intro.pdf

···

On Thursday 03 November 2005 10:41 am, Christian Neukirchen wrote:

I recently searched a bit about "behavior driven design" and found a
paper on "spec driven design"---which turned out to be about hardware.

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

15

Thank you all for your responses, I though on posting here some of my
opinions but it turned out to be too big so I decided I'd better blog
about it. You can find the article here
http://www.theniceweb.com/JaviersBlog/2005/11/ruby-20-and-tesla-upcoming-syntax_04.html
Oh, would you guys please elaborate on "I'm not sure I trust the
validity of the tests since they are applied in a way that they are not
used." I don't really understand what you mean by that.

16

Selon obscured by code <javierg1975@gmail.com>:

Thank you all for your responses, I though on posting here some of my
opinions but it turned out to be too big so I decided I'd better blog
about it. You can find the article here

http://www.theniceweb.com/JaviersBlog/2005/11/ruby-20-and-tesla-upcoming-syntax_04.html

I know it's not exactly what you asked for when you've written this post, but
your comments on your blog about possible mandatory parentheses in Ruby2
concern me. Can people read Javier's blog and comment on that part? (if
possible by saying that it ain't gonna happen so!) Thanks in advance.

···

--
Christophe Grandsire.

http://rainbow.conlang.free.fr

It takes a straight mind to create a twisted conlang.

17

Sometimes, you can get different results by calling the method without a
receiver (what you do in this testing infrastructure) vs. with a receiver
(what you do when you use this class). I think the main problem revolves
around the use of #method_missing. If you declare a private method #foo and
also #method_missing, calling #foo without a receiver will give you the
private method #foo and calling #foo with a receiver will put you in
#method_missing. In general you are not testing with the protections that
you have from outside the class. There may be other suttle issues to be
concerned about. I don't think this invalidates your approach, but it should
be a concern.

···

On 11/4/05, obscured by code <javierg1975@gmail.com> wrote:

Thank you all for your responses, I though on posting here some of my
opinions but it turned out to be too big so I decided I'd better blog
about it. You can find the article here

http://www.theniceweb.com/JaviersBlog/2005/11/ruby-20-and-tesla-upcoming-syntax_04.html
Oh, would you guys please elaborate on "I'm not sure I trust the
validity of the tests since they are applied in a way that they are not
used." I don't really understand what you mean by that.

18

Jim Weirich <jim@weirichhouse.org> writes:

I recently searched a bit about "behavior driven design" and found a
paper on "spec driven design"---which turned out to be about hardware.

Here are a couple:
http://blog.daveastels.com/files/sdbp2005/BDD%20Intro%20slides.pdf
http://blog.daveastels.com/files/sdbp2005/BDD%20Intro.pdf

Yeah, I've seen those already. It's all rehashing the same content.
:wink: I'd like to see code...

···

On Thursday 03 November 2005 10:41 am, Christian Neukirchen wrote:

-- Jim Weirich jim@weirichhouse.org http://onestepback.org

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

19

Christophe Grandsire <christophe.grandsire@free.fr> writes:

Selon obscured by code <javierg1975@gmail.com>:

Thank you all for your responses, I though on posting here some of my
opinions but it turned out to be too big so I decided I'd better blog
about it. You can find the article here

http://www.theniceweb.com/JaviersBlog/2005/11/ruby-20-and-tesla-upcoming-syntax_04.html

I know it's not exactly what you asked for when you've written this post, but
your comments on your blog about possible mandatory parentheses in Ruby2
concern me. Can people read Javier's blog and comment on that part? (if
possible by saying that it ain't gonna happen so!) Thanks in advance.

I don't think the behavior will be changed compared to 1.8.2:
    assert raises(ZeroDivisionError)
and
    attr_accessor :items
will stay valid. (IANM, but I'd be very sad if parens are enforced.)

···

Christophe Grandsire.

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

20

gem install rspec

···

On Friday 04 November 2005 02:49 pm, Christian Neukirchen wrote:

> Here are a couple:
> http://blog.daveastels.com/files/sdbp2005/BDD%20Intro%20slides.pdf
> http://blog.daveastels.com/files/sdbp2005/BDD%20Intro.pdf

Yeah, I've seen those already. It's all rehashing the same content.
:wink: I'd like to see code...

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)