Using rcov in already running program

Hello,

How can I use rcov from an already running Ruby program?

I'm running a Ruby interpreter that is embedded within a C program.
This situation is the inverse of how rcov is presently used: rcov is
used to run *.rb files instead of ruby.

So, I want to be able to write something like "require
'rcov/runner'" and have rcov do it do its thing on the currently
running Ruby program. This is like running unit tests by simply
doing "require 'test/unit'" (which has a Kernel#at_exit statement
that runs all the tests before the program terminates).

Thanks for your consideration.

I've never used rcov with an embedded Ruby interpreter, but something like
this should work:

rcov/runner.rb:

require 'rcov'
require 'rbconfig'

# try to load xx, but don't give up if it's not available
begin
  require 'xx'
rescue LoadError
end
begin
  require 'rcov/report'
rescue NameError
  # hack to make textual reports work in the absence of xx-0.1.0
  # Rcov::HTMLCoverage cannot be used without it though
end

rcov_analyzer = Rcov::CodeCoverageAnalyzer.new
at_exit do
  rcov_analyzer.remove_hook
  rcov_analyzer.dump_coverage_info([Rcov::TextReport.new])
  # use Rcov::HTMLCoverage above to generate HTML reports; the formatters admit
  # a number of options, listed in rcov's RDoc documentation.
end
rcov_analyzer.install_hook

If you save that as rcov/runner.rb in your $LOAD_PATH, you will be able to

$ cat a.rb

srand(0)
c = 0
d = 1
10.times do |i|
  if rand % (i + 10)
    c += i
    d += 1
  end
end
puts "blergh"
if c > 4*d
  # stuff
  puts "yep"
else
  puts "nope"
  # more stuff
end

puts "Done: #{c+d}"
$ ruby -I ../head/lib -rrcov/runner a.rb
blergh
yep
Done: 56

···

On Mon, Sep 04, 2006 at 05:15:24PM +0900, Suraj N. Kurapati wrote:

How can I use rcov from an already running Ruby program?

I'm running a Ruby interpreter that is embedded within a C program.
This situation is the inverse of how rcov is presently used: rcov is
used to run *.rb files instead of ruby.

So, I want to be able to write something like "require
'rcov/runner'" and have rcov do it do its thing on the currently
running Ruby program. This is like running unit tests by simply
doing "require 'test/unit'" (which has a Kernel#at_exit statement
that runs all the tests before the program terminates).

+----------------------------------------------------+-------+-------+--------+

                 File | Lines | LOC | COV |

+----------------------------------------------------+-------+-------+--------+

a.rb | 20 | 16 | 81.2% |

+----------------------------------------------------+-------+-------+--------+

Total | 20 | 16 | 81.2% |

+----------------------------------------------------+-------+-------+--------+
81.2% 1 file(s) 20 Lines 16 LOC

In your case, you can require 'rcov/runner' in the embedded interpreter, and
the code loaded/parsed from that moment on will be monitored.
If you cannot load rcov/runner before the code you want to analyze is parsed,
it might still be possible to make it work by doing
  SCRIPT_LINES__ = {} unless defined? SCRIPT_LINES__
early on. Of course, only the code executed after rcov/runner is loaded would
be analyzed in that case.

rcov/runner.rb seems a good idea and I'd want to add it to the next release;
do you have any suggestions regarding the interface or the default options?
Dumping a textual report like the above one seems to make sense as it is akin
to Test::Unit's output.

Thanks,

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Mauricio Fernandez wrote:

How can I use rcov from an already running Ruby program?

I'm running a Ruby interpreter that is embedded within a C program.
This situation is the inverse of how rcov is presently used: rcov is
used to run *.rb files instead of ruby.

So, I want to be able to write something like "require
'rcov/runner'" and have rcov do it do its thing on the currently
running Ruby program. This is like running unit tests by simply
doing "require 'test/unit'" (which has a Kernel#at_exit statement
that runs all the tests before the program terminates).

I've never used rcov with an embedded Ruby interpreter, but something like
this should work:

[code snipped]

This works wonderfully. Thank you!

In your case, you can require 'rcov/runner' in the embedded interpreter, and
the code loaded/parsed from that moment on will be monitored.

This is fine for my purpose.

rcov/runner.rb seems a good idea and I'd want to add it to the next release;
do you have any suggestions regarding the interface or the default options?

Yes, I would like a mechanism to pass the regular rcov command-line
options to the embedded rcov runner. I'm thinking that this can be
done either by setting the global ARGV array or maybe a RCOV_ARGV
array before loading the runner:

RCOV_ARGV = ['--html', '--no-color', '--save', 'coverage.html']
require 'rcov/runner'

Dumping a textual report like the above one seems to make sense as it is akin
to Test::Unit's output.

This is sufficient, but I would like to see the more detailed HTML
report as well, so I can know where coverage needs improvement.

Thanks for your consideration.

···

On Mon, Sep 04, 2006 at 05:15:24PM +0900, Suraj N. Kurapati wrote:

Mauricio Fernandez wrote:

require 'rcov'
require 'rbconfig'

# try to load xx, but don't give up if it's not available
begin
  require 'xx'
rescue LoadError
end
begin
  require 'rcov/report'
rescue NameError
  # hack to make textual reports work in the absence of xx-0.1.0
  # Rcov::HTMLCoverage cannot be used without it though
end

rcov_analyzer = Rcov::CodeCoverageAnalyzer.new
at_exit do
  rcov_analyzer.remove_hook
  rcov_analyzer.dump_coverage_info([Rcov::TextReport.new])
  # use Rcov::HTMLCoverage above to generate HTML reports; the formatters admit
  # a number of options, listed in rcov's RDoc documentation.
end
rcov_analyzer.install_hook

I can't find Rcov::CodeCoverageAnalyzer#dump_coverage_info,
Rcov::HTMLCoverage, or Rcov::TextReport in the RDoc shipped with
rcov 0.7.0.1

Aren't these internal structures undocumented because they shouldn't
be used by outsiders?

Thanks for your attention.

Mauricio Fernandez wrote:
> I've never used rcov with an embedded Ruby interpreter, but something like
> this should work:
[code snipped]

[...]

> rcov/runner.rb seems a good idea and I'd want to add it to the next release;
> do you have any suggestions regarding the interface or the default options?

Yes, I would like a mechanism to pass the regular rcov command-line
options to the embedded rcov runner. I'm thinking that this can be
done either by setting the global ARGV array or maybe a RCOV_ARGV
array before loading the runner:

RCOV_ARGV = ['--html', '--no-color', '--save', 'coverage.html']
require 'rcov/runner'

What about something like this?

require 'rcov/runner'
# if nothing else is done, it generates a plain text report, no HTML

# Rcov::Runner.generate formatter [, options] is used to specify which
# reports are to be generated. The possible options will be explained in the
# RDoc documentation.
Rcov::Runner.generate Rcov::HTMLCoverage, :color => false
Rcov::Runner.generate Rcov::TextCoverageDiff, :coverage_diff_mode => :record,
                      :coverage_diff_file => "coverage.data"

Passing options with an ARGV feels clunky to me (I never liked that part of
RDoc's API).

> Dumping a textual report like the above one seems to make sense as it is akin
> to Test::Unit's output.

This is sufficient, but I would like to see the more detailed HTML
report as well, so I can know where coverage needs improvement.

I think I'll either move xx (currently embedded in bin/rcov) to a separate
file or rewrite the XHTML generation code using erb (since everybody's got
it). XHTML reports could then be created as shown above.

···

On Tue, Sep 05, 2006 at 03:00:07AM +0900, Suraj N. Kurapati wrote:

> On Mon, Sep 04, 2006 at 05:15:24PM +0900, Suraj N. Kurapati wrote:

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Yes, I originally didn't want them to be used externally because I didn't want
to commit to keeping API compatibility, etc. But you've presented a good use
case that warrants exposing some of them, so I'll think about the API to make
sure it is reasonable and will not require changes in short, and document it.

···

On Sat, Sep 09, 2006 at 01:54:19AM +0900, Suraj N. Kurapati wrote:

Mauricio Fernandez wrote:
> rcov_analyzer = Rcov::CodeCoverageAnalyzer.new
> at_exit do
> rcov_analyzer.remove_hook
> rcov_analyzer.dump_coverage_info([Rcov::TextReport.new])
> # use Rcov::HTMLCoverage above to generate HTML reports; the formatters admit
> # a number of options, listed in rcov's RDoc documentation.
> end
> rcov_analyzer.install_hook

I can't find Rcov::CodeCoverageAnalyzer#dump_coverage_info,
Rcov::HTMLCoverage, or Rcov::TextReport in the RDoc shipped with
rcov 0.7.0.1

Aren't these internal structures undocumented because they shouldn't
be used by outsiders?

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Mauricio Fernandez wrote:

RCOV_ARGV = ['--html', '--no-color', '--save', 'coverage.html']
require 'rcov/runner'

What about something like this?

require 'rcov/runner'
# if nothing else is done, it generates a plain text report, no HTML

# Rcov::Runner.generate formatter [, options] is used to specify which
# reports are to be generated. The possible options will be explained in the
# RDoc documentation.
Rcov::Runner.generate Rcov::HTMLCoverage, :color => false
Rcov::Runner.generate Rcov::TextCoverageDiff, :coverage_diff_mode => :record,
                      :coverage_diff_file => "coverage.data"

This is good. Please proceed with this approach.

Passing options with an ARGV feels clunky to me (I never liked that part of
RDoc's API).

Good point. I thought that maybe it would reduce the amount of work
for you by reusing the existing ARGV handling code. Nevertheless, as
you've shown, it is cleaner to use the Rcov API directly.

Dumping a textual report like the above one seems to make sense as it is akin
to Test::Unit's output.

This is sufficient, but I would like to see the more detailed HTML
report as well, so I can know where coverage needs improvement.

I think I'll either move xx (currently embedded in bin/rcov) to a separate
file or rewrite the XHTML generation code using erb (since everybody's got
it). XHTML reports could then be created as shown above.

As long as the interaction with xx is contained within 'rcov/runner'
there shouldn't be any problem from the user's perspective.

I'm looking forward to the next release. :slight_smile:

Thanks for your support.

···

On Tue, Sep 05, 2006 at 03:00:07AM +0900, Suraj N. Kurapati wrote:

Alright then, I'll document the available report generators ("formatters")
and their options in the RDoc.

···

On Tue, Sep 05, 2006 at 09:39:10AM +0900, Suraj N. Kurapati wrote:

Mauricio Fernandez wrote:
> On Tue, Sep 05, 2006 at 03:00:07AM +0900, Suraj N. Kurapati wrote:
>> RCOV_ARGV = ['--html', '--no-color', '--save', 'coverage.html']
>> require 'rcov/runner'
>
> What about something like this?
>
> require 'rcov/runner'
> # if nothing else is done, it generates a plain text report, no HTML
>
> # Rcov::Runner.generate formatter [, options] is used to specify which
> # reports are to be generated. The possible options will be explained in the
> # RDoc documentation.
> Rcov::Runner.generate Rcov::HTMLCoverage, :color => false
> Rcov::Runner.generate Rcov::TextCoverageDiff, :coverage_diff_mode => :record,
> :coverage_diff_file => "coverage.data"

This is good. Please proceed with this approach.

##
hmmm, which one is better?

lib/cov/runner.rb:

require 'rcov/runner'

  ruby -rcov/runner myscript.rb

or

lib/cov.rb -> ruby -rcov myscript.rb

Naming your lib r.* can be handy at times...

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby

Mauricio Fernandez wrote:

hmmm, which one is better?

require 'rcov/runner'

  ruby -rcov/runner myscript.rb

This would agree more with RubyGems convention.

or

lib/cov.rb -> ruby -rcov myscript.rb

Naming your lib r.* can be handy at times...

That is pretty neat, but it might be misleading when written in
source code: require 'cov' ... "cov? is that rcov or something else?"

You could make both versions available by having lib/cov.rb load
lib/rcov/runner.rb.

Nevertheless, the choice is yours. :slight_smile: