[ann] coverage 0.1

ruby-talk
1

Ruby - Coverage

see example output here:
http://neoneye.dk/coverage/

download it from here: tgz, zip, gem.
http://rubyforge.org/project/showfiles.php?group_id=18&release_id=360

Purpose

···

=======

This tool helps identify those lines of code in your
program which didn’t got executed while running.

NaHi is the author of the original ‘coverage.rb’,
47 lines of Ruby code. Today it outputs nice HTML.

License

Ruby’s license

Install

Follow these 3 steps in order to install

ruby install.rb config

ruby install.rb setup

ruby install.rb install

Usage

Insert ‘-rcoverage’ when you wish to make coverage output.
For instance:

ruby -rcoverage test_all.rb

It should generage an ‘index.html’ file, containing coverage info.

Credits

NAKAMURA Hiroshi (NaHi) for the original coverage code.

Mauricio Julio Fernández Pradier, lots of improvements.

Bugs reports (rubyforge account required)

If you find any misbehavier, then please submit it here:
http://rubyforge.org/tracker/?atid=149&group_id=18&func=browse

Feature requests (rubyforge account required)

If there is something you wish, then tell it so we can figure out
how to make ‘coverage’ capable of it.
http://rubyforge.org/tracker/?atid=152&group_id=18&func=browse

end of announcement

IMHO Can be handy in conjunction with unittesting.

–
Simon Strandgaard

2

Simon Strandgaard wrote:

Ruby - Coverage

Just wanted to note that you have some users of this…

The students in my “Requirements Engineering and Verification and
Validation” course have done statement coverage analysis while testing.
They used a script based on a previous version of yours.

BTW, there is a rather delicate issue you might want to address: which
rb files not to include in the analysis. My students used this when
running their tests to see how well the tests covered the code. They
found that all required files are included though, even std rb files
like the ones in the lib needed for unit testing etc. Default might be
to exclude these files.

Also they mentioned that the html files should be saved to a dir local
to where the script is run and not in the absolute path of the
corresponding rb file.

Maybe you have already fixed these issues…

BTW, I hacked up something similar to nounit.sf.net for them to use in
their project but never had the time to finish it. Maybe we should merge
our efforts into some kind of test evaluation package?

/Robert

3

Simon Strandgaard wrote:

Ruby - Coverage

Just wanted to note that you have some users of this…

The students in my “Requirements Engineering and Verification and
Validation” course have done statement coverage analysis while testing.
They used a script based on a previous version of yours.

awesome, I have only played with it as an experiment.
I have never thought it was being used.
I feel bad making packages without doing test-first and supplying a
testsuite to ensure the quality and robustness.
Batsman reminded me that I had this experimental package… I had
completely forgotten about it (my brain contains only regexp+book).

Sorry. I better make it more usable :wink:

BTW: Any ideas how to test the outputted webpages ?

BTW, there is a rather delicate issue you might want to address: which
rb files not to include in the analysis. My students used this when
running their tests to see how well the tests covered the code. They
found that all required files are included though, even std rb files
like the ones in the lib needed for unit testing etc. Default might be
to exclude these files.

Batsman has been kind to fix it. The problem was that #add_file, added
too many files :wink:

def add_file(file, line_marked)
percent = calc_coverage(line_marked)
path = File.expand_path(file)
return nil if path =~ /\A#{Regexp.escape(CONFIG[“rubylibdir”])}/
return nil if path =~ /\A#{Regexp.escape(CONFIG[“sitelibdir”])}/

Also they mentioned that the html files should be saved to a dir local
to where the script is run and not in the absolute path of the
corresponding rb file.

A local dir… I will add that to the todo list right away.
Will fix this tonight if im not getting too tired.

Maybe you have already fixed these issues…

yes/no. Thanks for the suggestions.

BTW, I hacked up something similar to nounit.sf.net for them to use in
their project but never had the time to finish it. Maybe we should merge
our efforts into some kind of test evaluation package?

I will check that project out.

Merge… Sounds good.

BTW: does there exists any static analysis tools for Ruby ?

···

On Mon, 22 Mar 2004 07:45:26 +0900, Robert Feldt wrote:

–
Simon Strandgaard

4

Simon Strandgaard wrote:

awesome, I have only played with it as an experiment.
I have never thought it was being used.
I feel bad making packages without doing test-first and supplying a
testsuite to ensure the quality and robustness.
Batsman reminded me that I had this experimental package… I had
completely forgotten about it (my brain contains only regexp+book).

Sorry. I better make it more usable :wink:

BTW: Any ideas how to test the outputted webpages ?

I would probably test the statement coverage calc independently of the
html generation and save the latter for later… :wink:
or test the latter with some simple, “faked” stmt coverage data.

BTW, some student reported it might sometimes miss the end statements so
coverage is not 100% even though all “executable” stmts have actually
been executed. I don’t have the details though.

BTW, I hacked up something similar to nounit.sf.net for them to use in
their project but never had the time to finish it. Maybe we should merge
our efforts into some kind of test evaluation package?

I will check that project out.

Merge… Sounds good.

BTW: does there exists any static analysis tools for Ruby ?

Not that I’ve heard of but I have a thesis student starting next week
who will work on a (simple) one.

/Robert

5

[snip]

Also they mentioned that the html files should be saved to a dir local
to where the script is run and not in the absolute path of the
corresponding rb file.

A local dir… I will add that to the todo list right away.
Will fix this tonight if im not getting too tired.

Its now fixed, I have released a version 0.2, which you may want to try
out. It fixes many things.

download as: tgz, zip, gem
http://rubyforge.org/project/showfiles.php?group_id=18&release_id=361

example of generated output (version 0.2)
http://neoneye.dk/coverage/

···

On Sun, 21 Mar 2004 23:58:55 +0100, Simon Strandgaard wrote:

On Mon, 22 Mar 2004 07:45:26 +0900, Robert Feldt wrote:

–
Simon Strandgaard

6

[snip]

Also they mentioned that the html files should be saved to a dir local
to where the script is run and not in the absolute path of the
corresponding rb file.

A local dir… I will add that to the todo list right away.
Will fix this tonight if im not getting too tired.

Now in version 0.2, a ‘coverage’ dir is being created in the current dir.
All output goes to that dir. The filenames is no longer the absolute path,
its simply just the basename (a suffix number is added in case there is a
nameclash).

Furthermore it is now XHTML1.0 strict.

I hope this is better. Please continue suggesting improvements.

···

On Sun, 21 Mar 2004 23:58:55 +0100, Simon Strandgaard wrote:

On Mon, 22 Mar 2004 07:45:26 +0900, Robert Feldt wrote:

–
Simon Strandgaard

7

Simon Strandgaard wrote:

BTW: Any ideas how to test the outputted webpages ?

I would probably test the statement coverage calc independently of the
html generation and save the latter for later… :wink:
or test the latter with some simple, “faked” stmt coverage data.

Good suggestion, I will keep that in mind.

BTW, some student reported it might sometimes miss the end statements so
coverage is not 100% even though all “executable” stmts have actually
been executed. I don’t have the details though.

Yes end is a problem, also when one initializes an array spanning over
multiple lines. The original coverage code didn’t deal with this, and I
have no clue how to extend it so its capable of doing so.
Not that I didn’t tried, but it seems as #trace_var doesn’t deal with it.

If anyone has an idea how to extend the original coverage to do this, then
please submit a few hints on how to do it.

···

On Mon, 22 Mar 2004 08:16:53 +0900, Robert Feldt wrote:

–
Simon Strandgaard

8

Simon Strandgaard wrote:

[snip]

Also they mentioned that the html files should be saved to a dir local
to where the script is run and not in the absolute path of the
corresponding rb file.

A local dir… I will add that to the todo list right away.
Will fix this tonight if im not getting too tired.

Now in version 0.2, a ‘coverage’ dir is being created in the current dir.
All output goes to that dir. The filenames is no longer the absolute path,
its simply just the basename (a suffix number is added in case there is a
nameclash).

Furthermore it is now XHTML1.0 strict.

Sounds nice, thx.

I hope this is better. Please continue suggesting improvements.

My course ended now so no more improvements to expect from the students
I guess…

I will take a look though.

/R

···

On Sun, 21 Mar 2004 23:58:55 +0100, Simon Strandgaard wrote:

On Mon, 22 Mar 2004 07:45:26 +0900, Robert Feldt wrote: