Does anybody else do random testing on their ruby code besides
me? I picked it up because of my background (IC design) where
random testing is one of the verification techniques of
hardware.
So far, I've used 3 random testing strategies for my ruby code.
Here are the different ways that the expected response is
determined using those strategies:
- compare multiple implementations of the same functionality
(i.e. StringIO vs. IO, ordered hash vs. Hash).
- AOP. Make a wrapper class or module that does pre and post
checks of the methods. Use that wrapper class/module when
testing. When the pre-check fails, the current test would be
aborted/skipped (random generation of inputs yielded something
invalid). When the post-check fails, the test would fail.
- determine the expected response (from some random input)
within the test suite.
For all of these strategies, I've had to hack up Test::Unit in
the same way to add various features. Here are the things I've
added:
+ instead of running the test_* methods in a fixed order once,
run them in a random order for N passes.
+ abort testing when the first failure is reached.
+ easy way to skip the rest of the current test (when the
random input is invalid). Don't include this in the number of
tests or mark it as skipped.
+ way to pass number of passes of the test_* methods on the
command line.
+ display random seed and way of passing it in on the command
line.
+ test suite should have access to the --verbose level or
another switch to control the debugging verbosity.
+ way to pass command-line options down to the test suite to
control various things - what methods to test, what
classes to test, various other flags.
I would like to see most of these (optional) features added to
test/unit. Anybody else have opinions on this subject?
···
______________________________________________________
Yahoo! for Good
Donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/