[ANN] rcov 0.6.0: "differential code coverage", full (faster) cross-references, vim integration

Source code, additional information, screenshots... available at
    http://eigenclass.org/hiki.rb?rcov
Release information:
    http://eigenclass.org/hiki.rb?rcov+0.6.0

Sample (fully) cross-referenced report at
    http://eigenclass.org/static/rcov-sample-report-full-crossref/

This release includes two RubyGems packages: a binary one for Win32 and a
platform-independent one for all those with a compiler, (or a lot of patience,
if willing to run rcov in pure-Ruby mode), so
  gem install rcov
should work.

Overview

···

========
rcov is a code coverage tool for Ruby. It is commonly used for viewing overall
test coverage of target code. It features:
* fast execution: 20-300 times faster than previous tools
* multiple analysis modes: standard, bogo-profile, "intentional testing",
  dependency analysis...
* detection of uncovered code introduced since the last run ("differential
  code coverage")
* fairly accurate coverage information through code linkage inference using
  simple heuristics
* cross-referenced XHTML and several kinds of text reports
* support for easy automation with Rake and Rant
* colorblind-friendliness

What's new in 0.6.0

See http://eigenclass.org/hiki.rb?rcov+0.6.0 for the detailed change summary.

0.6.0 features a new differential coverage mode (--text-coverage-diff/-D)
which will tell you when you've added new code that was not covered by
the tests:

!!!!! Uncovered code introduced in app/models/article.rb

    ### app/models/article.rb:44
   
         # Find all articles on a certain date
         def self.find_all_by_date(year, month = nil, day = nil)
    !! from, to = self.time_delta(year, month, day)
    !! Article.find(:all, :conditions => ["articles.created_at BETWEEN ? AND ? AND articles.published != 0", from, to], :order => 'articles.created_at DESC', :include => [:categories, :trackbacks, :comments])
    !! end
         
         # Find one article on a certain date

rcov 0.6.0 ships with a compiler plugin for integration with vim
(contributions for other editors/IDEs welcome).

Cross-referenced reports, which were recently introduced, have been expanded
to indicate where methods are called from and which methods were called for
each line (--xrefs). In addition to that, cross-referenced report generation
is now over 4 times faster for applications with deep call stacks (such as
Rails apps)

Downloading

The last version is available at
  http://eigenclass.org/hiki.rb?rcov

How do I use it?

In the common scenario, your tests are under test/ and the target code
(whose coverage you want) is in lib/. In that case, all you have to do is
use rcov to run the tests (instead of testrb), and a number of XHTML files
with the code coverage information will be generated, e.g.

    rcov -Ilib test/*.rb

will execute all the .rb files under test/ and generate the code coverage
report for the target code (i.e. for the files in lib/) under coverage/. The
target code needs not be under lib/; rcov will detect is as long as it is
require()d by the tests. rcov is smart enough to ignore "uninteresting"
files: the tests themselves, files installed in Ruby's standard locations,
etc. See rcov --help for the list of regexps rcov matches filenames
against.

rcov can also be used from Rake; see README.rake or the RDoc documentation
for more information.

rcov can output information in several formats, and perform different kinds
of analyses in addition to plain code coverage. See rcov --help for a
description of the available options.

Sample output

See http://eigenclass.org/hiki.rb?rcov (once again) for screenshots.

Take a look at a sample code coverage report generated by rcov at
  http://eigenclass.org/static/rcov-sample-report-full-crossref/

The text report (also used by default in RcovTasks) resembles

+-----------------------------------------------------+-------+-------+--------+

                 File | Lines | LOC | COV |

+-----------------------------------------------------+-------+-------+--------+

lib/rcov.rb | 572 | 358 | 91.3% |

+-----------------------------------------------------+-------+-------+--------+

Total | 572 | 358 | 91.3% |

+-----------------------------------------------------+-------+-------+--------+
91.3% 1 file(s) 572 Lines 358 LOC

The (undecorated) textual output with execution count information looks like this:

$ rcov --no-html --text-counts b.rb

./b.rb

                                                                       > 2
a, b, c = (1..3).to_a | 2
10.times do | 1
   a += 1 | 10
   20.times do |i| | 10
     b += i | 200
     b.times do | 200
       c += (j = (b-a).abs) > 0 ? j : 0 | 738800
     end | 0
   end | 0
end | 0

rcov can detect when you've added code that was not covered by your unit
tests:

    $ rcov --text-coverage-diff --no-color test/*.rb
    Started
    .......................................
    Finished in 1.163085 seconds.
    
    39 tests, 415 assertions, 0 failures, 0 errors
    
    ================================================================================
    !!!!! Uncovered code introduced in lib/rcov.rb
    
    ### lib/rcov.rb:207
       
   def precompute_coverage(comments_run_by_default = true)
     changed = false
     lastidx = lines.size - 1
     if (!is_code?(lastidx) || /^__END__$/ =~ @lines[-1]) && !@coverage[lastidx]
    !! # mark the last block of comments
    !! @coverage[lastidx] ||= :inferred
    !! (lastidx-1).downto(0) do |i|
    !! break if is_code?(i)
    !! @coverage[i] ||= :inferred
    !! end
    !! end
     (0...lines.size).each do |i|
       next if @coverage[i]
       line = @lines[i]

Thanks

Alex Wayne:
* reported problem with heredocs: they were not being marked as a whole if
  the "header" wasn't reported by Ruby.
* reported problem with the last line of literal data structs not being
  covered if there was stuff after the end delimiter

Coda Hale:
* reported problem with blocks were the first line is not being marked
  and ditto for the last line when end/} is followed by more stuff

Tim Shadel:
* reported that the last comment block was not being marked even when
  it was the last thing in the file

License
-------
rcov is released under the terms of Ruby's license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright (c) 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this
     list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice,
     this list of conditions and the following disclaimer in the documentation
     and/or other materials provided with the distribution.
  3. Neither the name of ePark Labs nor the names of its contributors may be
     used to endorse or promote products derived from this software without
     specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

[snip]

nice cross-referencing, when clicking on a method you can see who
calls it, and which methods it calls. Well done :slight_smile:

···

On 6/12/06, Mauricio Fernandez <mfp@acm.org> wrote:

Source code, additional information, screenshots... available at
    http://eigenclass.org/hiki.rb?rcov
Release information:
    http://eigenclass.org/hiki.rb?rcov+0.6.0

Sample (fully) cross-referenced report at
    http://eigenclass.org/static/rcov-sample-report-full-crossref/

--
Simon Strandgaard

It just keeps getting better and better!

I just ran my first xref in rcov, and I'd like to request that the
file reference box that appears should (soft) wrap the filenames. I
have some long paths, and the filename goes way off the browser
window.

Please feel free to contact me off list for a screen shot.

Otherwise, I'm loving it! Thanks!

Sean

···

On 6/12/06, Mauricio Fernandez <mfp@acm.org> wrote:

Source code, additional information, screenshots... available at
    http://eigenclass.org/hiki.rb?rcov
Release information:
    http://eigenclass.org/hiki.rb?rcov+0.6.0

Sample (fully) cross-referenced report at
    http://eigenclass.org/static/rcov-sample-report-full-crossref/

This release includes two RubyGems packages: a binary one for Win32 and a
platform-independent one for all those with a compiler, (or a lot of patience,
if willing to run rcov in pure-Ruby mode), so
  gem install rcov
should work.

Overview

rcov is a code coverage tool for Ruby. It is commonly used for viewing overall
test coverage of target code. It features:
* fast execution: 20-300 times faster than previous tools
* multiple analysis modes: standard, bogo-profile, "intentional testing",
  dependency analysis...
* detection of uncovered code introduced since the last run ("differential
  code coverage")
* fairly accurate coverage information through code linkage inference using
  simple heuristics
* cross-referenced XHTML and several kinds of text reports
* support for easy automation with Rake and Rant
* colorblind-friendliness

What's new in 0.6.0

See http://eigenclass.org/hiki.rb?rcov+0.6.0 for the detailed change summary.

0.6.0 features a new differential coverage mode (--text-coverage-diff/-D)
which will tell you when you've added new code that was not covered by
the tests:

!!!!! Uncovered code introduced in app/models/article.rb

    ### app/models/article.rb:44

         # Find all articles on a certain date
         def self.find_all_by_date(year, month = nil, day = nil)
    !! from, to = self.time_delta(year, month, day)
    !! Article.find(:all, :conditions => ["articles.created_at BETWEEN ? AND ? AND articles.published != 0", from, to], :order => 'articles.created_at DESC', :include => [:categories, :trackbacks, :comments])
    !! end

         # Find one article on a certain date

rcov 0.6.0 ships with a compiler plugin for integration with vim
(contributions for other editors/IDEs welcome).

Cross-referenced reports, which were recently introduced, have been expanded
to indicate where methods are called from and which methods were called for
each line (--xrefs). In addition to that, cross-referenced report generation
is now over 4 times faster for applications with deep call stacks (such as
Rails apps)

Downloading

The last version is available at
  http://eigenclass.org/hiki.rb?rcov

How do I use it?

In the common scenario, your tests are under test/ and the target code
(whose coverage you want) is in lib/. In that case, all you have to do is
use rcov to run the tests (instead of testrb), and a number of XHTML files
with the code coverage information will be generated, e.g.

    rcov -Ilib test/*.rb

will execute all the .rb files under test/ and generate the code coverage
report for the target code (i.e. for the files in lib/) under coverage/. The
target code needs not be under lib/; rcov will detect is as long as it is
require()d by the tests. rcov is smart enough to ignore "uninteresting"
files: the tests themselves, files installed in Ruby's standard locations,
etc. See rcov --help for the list of regexps rcov matches filenames
against.

rcov can also be used from Rake; see README.rake or the RDoc documentation
for more information.

rcov can output information in several formats, and perform different kinds
of analyses in addition to plain code coverage. See rcov --help for a
description of the available options.

Sample output

See http://eigenclass.org/hiki.rb?rcov (once again) for screenshots.

Take a look at a sample code coverage report generated by rcov at
  http://eigenclass.org/static/rcov-sample-report-full-crossref/

The text report (also used by default in RcovTasks) resembles

+-----------------------------------------------------+-------+-------+--------+
> File | Lines | LOC | COV |
+-----------------------------------------------------+-------+-------+--------+
>lib/rcov.rb | 572 | 358 | 91.3% |
+-----------------------------------------------------+-------+-------+--------+
>Total | 572 | 358 | 91.3% |
+-----------------------------------------------------+-------+-------+--------+
91.3% 1 file(s) 572 Lines 358 LOC

The (undecorated) textual output with execution count information looks like this:

$ rcov --no-html --text-counts b.rb

./b.rb

                                                                       > 2
a, b, c = (1..3).to_a | 2
10.times do | 1
   a += 1 | 10
   20.times do |i| | 10
     b += i | 200
     b.times do | 200
       c += (j = (b-a).abs) > 0 ? j : 0 | 738800
     end | 0
   end | 0
end | 0

rcov can detect when you've added code that was not covered by your unit
tests:

    $ rcov --text-coverage-diff --no-color test/*.rb
    Started
    .......................................
    Finished in 1.163085 seconds.

    39 tests, 415 assertions, 0 failures, 0 errors

    ================================================================================
    !!!!! Uncovered code introduced in lib/rcov.rb

    ### lib/rcov.rb:207

         def precompute_coverage(comments_run_by_default = true)
           changed = false
           lastidx = lines.size - 1
           if (!is_code?(lastidx) || /^__END__$/ =~ @lines[-1]) && !@coverage[lastidx]
    !! # mark the last block of comments
    !! @coverage[lastidx] ||= :inferred
    !! (lastidx-1).downto(0) do |i|
    !! break if is_code?(i)
    !! @coverage[i] ||= :inferred
    !! end
    !! end
           (0...lines.size).each do |i|
             next if @coverage[i]
             line = @lines[i]

Thanks

Alex Wayne:
* reported problem with heredocs: they were not being marked as a whole if
  the "header" wasn't reported by Ruby.
* reported problem with the last line of literal data structs not being
  covered if there was stuff after the end delimiter

Coda Hale:
* reported problem with blocks were the first line is not being marked
  and ditto for the last line when end/} is followed by more stuff

Tim Shadel:
* reported that the last comment block was not being marked even when
  it was the last thing in the file

License
-------
rcov is released under the terms of Ruby's license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright (c) 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this
     list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice,
     this list of conditions and the following disclaimer in the documentation
     and/or other materials provided with the distribution.
  3. Neither the name of ePark Labs nor the names of its contributors may be
     used to endorse or promote products derived from this software without
     specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Just trying to improve the code for XHTML generation you wrote originally :wink:

···

On Tue, Jun 13, 2006 at 05:56:37AM +0900, Simon Strandgaard wrote:

On 6/12/06, Mauricio Fernandez <mfp@acm.org> wrote:
>Source code, additional information, screenshots... available at
> http://eigenclass.org/hiki.rb?rcov
>Release information:
> http://eigenclass.org/hiki.rb?rcov+0.6.0
>
>Sample (fully) cross-referenced report at
> http://eigenclass.org/static/rcov-sample-report-full-crossref/
[snip]

nice cross-referencing, when clicking on a method you can see who
calls it, and which methods it calls. Well done :slight_smile:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Do you have a patch to the CSS handy? (another option would be truncating
filenames and maybe adding an onmouseover popup)

···

On Tue, Jun 13, 2006 at 06:23:01AM +0900, Sean Hussey wrote:

It just keeps getting better and better!

I just ran my first xref in rcov, and I'd like to request that the
file reference box that appears should (soft) wrap the filenames. I
have some long paths, and the filename goes way off the browser
window.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby