Forcing some code to run at the end of tests

ruby-talk
21

yn Tue, 16 May 2006, Pistos Christou wrote:

Sy Ali wrote:

You can setup your test runner manually:

<example snipped>

Thank you. This was just the solution I needed. It was easy to get
working too. =)

I have a similar situation: I have to setup and teardown various bits in
a database before and after (respectively) [the combined whole of] ALL
tests. Furthermore, the teardown must occur regardless of whether any
tests pass or fail. I've posted to ruby-talk about this a while ago.
My final solution was to just use a wrapper script to call rake, and
then call the teardown code.

Would the above TestRunner solution work for my case, too?

Pistos

what's wrong with BEGIN and END blocks?

-a

···

On 5/13/06, Daniel Harple <dharple@generalconsumption.org> wrote:

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

22

I think so, yes. TestRunner was simple for me to implement, and it
would guarantee that certain code could run before and after your
tests.

···

On 5/15/06, Pistos Christou <jesusrubsyou.5.pistos@geoshell.com> wrote:

I have a similar situation: I have to setup and teardown various bits in
a database before and after (respectively) [the combined whole of] ALL
tests. Furthermore, the teardown must occur regardless of whether any
tests pass or fail. I've posted to ruby-talk about this a while ago.
My final solution was to just use a wrapper script to call rake, and
then call the teardown code.

Would the above TestRunner solution work for my case, too?

23

i'ts simple : they are run in alphabetical order. so, if you name them

   test_000
   test_001
   test_002
   ...

they are run in 'order'.

-a

···

On Mon, 15 May 2006, Sy Ali wrote:

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Sun, 14 May 2006, Sy Ali wrote:
> Is there an easy way for me to abort a test? (i.e. to exit the test
> method cleanly?)

     harp:~ > cat a.rb

<snip>

regards.

Awesome! That's just what I was looking for (for that particular
problem). It's strange that the feature isn't there already.

Now I'm going to play with setting the order of my tests to get a
block of code to execute last.

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

24

unknown wrote:

tests. Furthermore, the teardown must occur regardless of whether any
tests pass or fail. I've posted to ruby-talk about this a while ago.
My final solution was to just use a wrapper script to call rake, and
then call the teardown code.

Would the above TestRunner solution work for my case, too?

what's wrong with BEGIN and END blocks?

Where exactly do they go? In the Rakefile?

Pistos

···

--
Posted via http://www.ruby-forum.com/\.

25

Sy Ali wrote:

Would the above TestRunner solution work for my case, too?

I think so, yes. TestRunner was simple for me to implement, and it
would guarantee that certain code could run before and after your
tests.

I think I'll stick with what I've got, because to do TestRunner, I'd
have to write up some test suite wrappers. At the moment, I just have
wads of .rb files for individual test sets, and my Rakefile builds up a
heirarchical set of tasks that lets me run a whole subtree of tests, or
just a specific test (.rb file). My ./run-tests script passes args to
rake, so I can run a specific test or set of tests, but STILL run my
global setup and teardown properly.

Thanks for the TestRunner tip though.

Pistos

···

--
Posted via http://www.ruby-forum.com/\.

26

I wouldn't want to put my non-test code in a test.. it seems wierd.

As I understand, tests are not guaranteed to run in any order and
while this feature may work right now.. it may be removed in the
future.

···

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

i'ts simple : they are run in alphabetical order. so, if you name them

   test_000
   test_001
   test_002
   ...

they are run in 'order'.

27

i'ts simple : they are run in alphabetical order. so, if you name them

   test_000
   test_001
   test_002
   ...

they are run in 'order'.

I wouldn't want to put my non-test code in a test.. it seems wierd.

well, if you have non-test code you can do whatever you like to order it

   class FooBarTest < Test::Unit::TestCase
     def self.call_it
     end

     p 'do_this'
     p 'do_that'

     call_it
   end

etc.

As I understand, tests are not guaranteed to run in any order and while this
feature may work right now.. it may be removed in the future.

http://groups.google.com/group/comp.lang.ruby/tree/browse_frm/thread/b1c0410a7a230f62/85da1271088c609d?rnum=11&q=test+unit+order+noaa&_done=%2Fgroup%2Fcomp.lang.ruby%2Fbrowse_frm%2Fthread%2Fb1c0410a7a230f62%2F643c7de96374c3ed%3Fq%3Dtest+unit+order+noaa%26rnum%3D1%26#doc_ff84696d7878f61b

where did you hear it might be removed in the future?

regards.

-a

···

On Mon, 15 May 2006, Sy Ali wrote:

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

28

As I understand, tests are not guaranteed to run in any order and
while this feature may work right now.. it may be removed in the
future.

That was probably me who said that. I was wrong. They are alphabetical.

Check the wtr-general list, however, for a patch i released that makes
them run in the order you define them in.

Bret

29

well, if you have non-test code you can do whatever you like to order it

Not really. All the testing code will be run after other code,
nomatter what you do.. even using begin/end blocks wouldn't work for
me. All tests would run afterwards. It was quite wierd. Only the
testrunner stuff ended up working.

···

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Mon, 15 May 2006, Sy Ali wrote:
> As I understand, tests are not guaranteed to run in any order and while this
> feature may work right now.. it may be removed in the future.

where did you hear it might be removed in the future?

I wish I had a source for this, but I don't. I'm just told it by
people who know better than me. =)

30

well, if you have non-test code you can do whatever you like to order it

Not really. All the testing code will be run after other code,
nomatter what you do.. even using begin/end blocks wouldn't work for
me. All tests would run afterwards. It was quite wierd. Only the
testrunner stuff ended up working.

can you show us this? it works for me:

   harp:~ > cat a.rb
   BEGIN{ puts '======== BEGIN ========' }

   END{ puts '======== END ========' }

   require 'test/unit'

   class ATest < Test::Unit::TestCase
     def test_a() assert true end
   end

   harp:~ > ruby a.rb
   ======== BEGIN ========
   Loaded suite a
   Started
   .
   Finished in 0.000684 seconds.

   1 tests, 1 assertions, 0 failures, 0 errors
   ======== END ========

···

On Tue, 16 May 2006, Sy Ali wrote:

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Mon, 15 May 2006, Sy Ali wrote:
> As I understand, tests are not guaranteed to run in any order and while this
> feature may work right now.. it may be removed in the future.

where did you hear it might be removed in the future?

I wish I had a source for this, but I don't. I'm just told it by
people who know better than me. =)

hmm. well the link i sent, although a bit old, was from the test/unit
maintainer, so i think that's probably right for now.

cheers.

-a
--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

31

It looks like the ordering is important. If you do:

require 'test/unit'
BEGIN{ puts '======== BEGIN ========' }
END{ puts '======== END ========' }

class ATest < Test::Unit::TestCase
   def test_a() assert true end
end

then the result will be:

======== BEGIN ========
======== END ========
Loaded suite a
Started
.
Finished in 0.001859 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

The problem is that include/import on top is idiomatic (even sometimes necessary) in for instance C/C++ and Java, so it seems natural to carry that over to Ruby. In Ruby, require is an executable statement, that may have side effects, and thus ordering becomes significant:

bash$ cat foo.rb
puts "bar"

bash$ irb
irb(main):001:0> require "foo.rb"
bar
=> true
irb(main):002:0>

/Vidar

···

Den 16. mai. 2006 kl. 02:07 skrev ara.t.howard@noaa.gov:

On Tue, 16 May 2006, Sy Ali wrote:

On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

well, if you have non-test code you can do whatever you like to order it

Not really. All the testing code will be run after other code,
nomatter what you do.. even using begin/end blocks wouldn't work for
me. All tests would run afterwards. It was quite wierd. Only the
testrunner stuff ended up working.

can you show us this? it works for me:

  harp:~ > cat a.rb
  BEGIN{ puts '======== BEGIN ========' }

  END{ puts '======== END ========' }

  require 'test/unit'

  class ATest < Test::Unit::TestCase
    def test_a() assert true end
  end

  harp:~ > ruby a.rb
  ======== BEGIN ========
  Loaded suite a
  Started
  .
  Finished in 0.000684 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors
  ======== END ========

32

Ok, so instead of basing my preference on a rumour I'll just base my
preference on avoiding the requirement to run tests in any particular
order. =)

Tests should all be independant of oneanother, otherwise you can't run
just one arbitrary test when you want.

···

>> On Mon, 15 May 2006, Sy Ali wrote:
>> > As I understand, tests are not guaranteed to run in any order and while
>> this
>> > feature may work right now.. it may be removed in the future.

> On 5/14/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
>> where did you hear it might be removed in the future?

On Tue, 16 May 2006, Sy Ali wrote:
> I wish I had a source for this, but I don't. I'm just told it by
> people who know better than me. =)

On 5/15/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

hmm. well the link i sent, although a bit old, was from the test/unit
maintainer, so i think that's probably right for now.

33

Yes, this is exactly what I was doing. Curious.

···

On 5/16/06, Vidar Larsen <vi_larsen@yahoo.no> wrote:

It looks like the ordering is important. If you do:

require 'test/unit'
BEGIN{ puts '======== BEGIN ========' }
END{ puts '======== END ========' }

class ATest < Test::Unit::TestCase
   def test_a() assert true end
end

then the result will be:

======== BEGIN ========
======== END ========
Loaded suite a
Started
.
Finished in 0.001859 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

34

it's also idiomatic to have BEGIN/END blocks before require though, because
BEGIN actually happens before any requires take place and END needs to occur
(typically) even if require throws an exception.

     harp:~ > cat a.rb
     BEGIN{ p Queue rescue warn $! }

     END{ p Queue }

     require 'thread'
     raise

     harp:~ > ruby a.rb
     uninitialized constant Queue
     a.rb:4: unhandled exception
     Queue

if you move the BEGIN/END blocks to the end of this script you will only get

     uninitialized constant Queue
     a.rb:4: unhandled exception

which is very counter-intuitive. in any case it's good pratice to put
BEGIN/END blocks up front imho.

cheers.

-a

···

On Tue, 16 May 2006, Vidar Larsen wrote:

It looks like the ordering is important. If you do:

require 'test/unit'
BEGIN{ puts '======== BEGIN ========' }
END{ puts '======== END ========' }

class ATest < Test::Unit::TestCase
def test_a() assert true end
end

then the result will be:

======== BEGIN ========
======== END ========
Loaded suite a
Started
.
Finished in 0.001859 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

The problem is that include/import on top is idiomatic (even sometimes
necessary) in for instance C/C++ and Java, so it seems natural to carry that
over to Ruby.

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

35

it's because test/unit registers it's own blocks too.

-a

···

On Tue, 16 May 2006, Sy Ali wrote:

On 5/16/06, Vidar Larsen <vi_larsen@yahoo.no> wrote:

It looks like the ordering is important. If you do:

require 'test/unit'
BEGIN{ puts '======== BEGIN ========' }
END{ puts '======== END ========' }

class ATest < Test::Unit::TestCase
   def test_a() assert true end
end

then the result will be:

======== BEGIN ========
======== END ========
Loaded suite a
Started
.
Finished in 0.001859 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Yes, this is exactly what I was doing. Curious.

--
be kind whenever possible... it is always possible.
- h.h. the 14th dali lama

36

There is a good reason for this.

test-unit itself puts code in an END block that creates and executes a
test runner if one hasn't already been created.

And END blocks are executed (at the end) in the REVERSE order than how
they were defined.

Therefore, you have to define your END block before 'test/unit' is
loaded. Or create the test runner yourself.

37

Oh I get it. So it's not so arcane an issue after all.

For a while I thought I was going mental.. things weren't happening in
the order I was specifying.. but it's just that I didn't understand
was was going on when I was specifying things the way I was.

Neat.

···

On 5/17/06, bpettichord@gmail.com <bpettichord@gmail.com> wrote:

test-unit itself puts code in an END block that creates and executes a
test runner if one hasn't already been created.

And END blocks are executed (at the end) in the REVERSE order than how
they were defined.

Therefore, you have to define your END block before 'test/unit' is
loaded. Or create the test runner yourself.

38

I don't mean to beat a dead horse, but i think i've come up with a
viable solution for this that doesn't involve manualy launching every
test through TestRunner, or stratigicaly placing BEGIN & END blocks.

All you really need to to is extend Test::Unit::TestCase and
programaticly add functionality for a startup() method (to run before
all testcases) and a close() method to run after everything.

In the class below i've overloaded #run to check if a startup methods
been run, and if not launch it before running anything. I've also made
use of the default_test (which curiously always runs after? - this might
be changed later on so i won't count in it working forever), to launch
the close method.

require 'test/unit/assertions'
require 'test/unit/failure'
require 'test/unit/error'
require 'test/unit/assertionfailederror'
require 'test/unit/testcase'

module Test
  module Unit
    class BetterTestCase < TestCase

      # startup flag
      @@startup_flag = true

      # Overloaded from Test::Unit::TestCase
      def run(result)
        if(@@startup_flag)
          begin
            startup
          rescue
            #ignore
          ensure
            @@startup_flag = false
          end
        end

        yield(STARTED, name)
        @_result = result
        begin
          setup
          __send__(@method_name)
        rescue AssertionFailedError => e
          add_failure(e.message, e.backtrace)
        rescue StandardError, ScriptError
          add_error($!)
        ensure
          begin
            teardown
          rescue AssertionFailedError => e
            add_failure(e.message, e.backtrace)
          rescue StandardError, ScriptError
            add_error($!)
          end
        end
        result.add_run
        yield(FINISHED, name)
      end

      # Called once before any test method runs. Can be used to
      # setup script-wide variables and conditions.
      def startup
      end

      # Called once after all test methods have ran. Can be used to
      # cleaup the environment after completion of the test methods.
      def close
      end

      #overload default_test, actualy runs last?
      def default_test
        begin
          close
        rescue
          #noop
        end
      end
    end
  end
end

After all this you can build your Test::Unit::TestCases by extending
that instead, and using startup and close the same way you would use
setup & teardown. handy eh?

example test case with startup and close methods:

require 'test/unit'
require 'better_test_case'

class ExampleTest < BetterTestCase

def startup
    #do somthing once before any tests are run
end

def close
    #do somthing after all tests are run
end

def test_0010_somthing
    #test somthing...
end

end

···

--
Posted via http://www.ruby-forum.com/.

39

Hey thanks for the suggestion. My mind isn't in the right context to
understand it.. but when I next revisit the wonderful world of testing
I'll sit down and learn this.

···

On 6/15/06, Brian Cowdery <bcowdery@gmail.com> wrote:

I don't mean to beat a dead horse, but i think i've come up with a
viable solution for this that doesn't involve manualy launching every
test through TestRunner, or stratigicaly placing BEGIN & END blocks.

40

Brian Cowdery wrote:

<snip>

class ExampleTest < BetterTestCase #<< error here >>

def startup
    #do somthing once before any tests are run
end

def close
    #do somthing after all tests are run
end

def test_0010_somthing
    #test somthing...
end

end

When I try to run this script, I'm getting a "uninitialized constant
BetterTestCase (NameError)" error on line indicated above. Ideas anyone?

···

--
Posted via http://www.ruby-forum.com/\.