ANN: minitest-reporters-json_reporter gem version 0.1.1 released

minitest-reporters-json_reporter version 0.1.1 has been released.

Home, Code: https://github.com/edhowland/minitest-reporters-json_reporter
Bugs: https://github.com/edhowland/minitest-reporters-json_reporter/issues

minitest-reporters-json_reporter is an extension to the Ruby
minitest-reporters gem that provides JSON output from your MiniTest or
MiniSpec test run.
JSON has become the lingua franca of interprogram communications. There are
several very good output formats
for MiniTest, but no JSON format until now. If you want to hook up an IDE
or some other automatic
framework, it is natural to send output in some parsable format like XML or
JSON. This extension summarizes the result of
a test run in format like JSON.a test run in an easible consumable format.

Usage is very simple. Just require 'minitest/reporters/json_reporter' in
your test_helper.rb or spec_helper.rb.
This gem also depends on 'minitest' and 'minitest/reporters'. Then, you
should also invoke the use! method of MiniTest::Reporters class with a call
to MiniTest::Reporters::JsonReporter.new.
Here is an example test_helper.rb:

require 'minitest/reporters/json_reporter'
# ... Other requires
require 'minitest/autorun'

MiniTest::Reporters.use! [MiniTest::Reporters::JsonReporter.new]

Here is the URLs for the dependent gems:

gem list minitest

*** LOCAL GEMS ***

minitest (5.7.0, 5.4.3) : https://rubygems.org/gems/minitest
minitest-reporters (1.1.8) : https://rubygems.org/gems/minitest-reporters
minitest-reporters-json_reporter (0.1.1) :
https://rubygems.org/gems/minitest-reporters-json_reporter

The JSON format is sent to stdout when you run a test or all tests.
There are 5 objects contained within this output:

1. metadata - General information about this test run. Includes generator
name, version and the test run completion time in UTC formatted in ISO8601
format.
2. status - Status of the overall test run. Can be 'Success', 'Passed with
skipped tests' or 'Failed'. There also a color attribute: green, yellow or
red - respectfully.
3. statistics - Contains counts of the status of test cases. Includes:
Total, Skipped, Failed Errored and Passed.
4. fails - Array of failed or errored tests. Each object herein contains
the information of each test, its name, class, error message and location
of the test.
5. skips - Array of each skipped. Each object contains the information as a
failed test, including the skip message, if any.

Here is a sample test run:

vagrant ruby report_spec.rb
{"status":{"code":"Failed","color":"red"},"metadata":{"generated_by":"Minitest::Reporters::JsonReporter","version":"0.1.1","time":"2016-04-07T23:31:04Z"},"statistics":{"total":5,"failed":2,"errored":1,"skipped":1,"passed":1},"fails":[{"type":"error","class":"Error","name":"test_0001_anonymous","message":"RuntimeError:
should fail\n report_spec.rb:6:in `block (2 levels) in
<main>'","location":"report_spec.rb:6"},{"type":"failure","class":"second
failure","name":"test_0001_anonymous","message":"Expected: 9\n Actual:
3","location":"report_spec.rb:20"},{"type":"failure","class":"failure","name":"test_0001_anonymous","message":"Expected:
2\n Actual:
1","location":"report_spec.rb:14"}],"skips":[{"type":"skipped","class":"skipped
test","name":"test_0001_anonymous","message":"what a
layabout","location":"report_spec.rb:32"}]}vagrant

You can use JSON parsing tools to get elements or subsets of this format.
One such tool is JQ: https://stedolan.github.io/jq/

Here is how you might view only the color of the test:

$ ruby blog_test.rb | jq -r .status.color
green

Cheers,
Ed

Ed Howland


http://twitter.com/ed_howland

Looks good. It’d be nice to have a version of this that doesn’t depend on
minitest-reporters and just uses native minitest reporting (which is better
in almost every way).

minitest-reporters breaks a lot of normal minitest functionality because it
replaces the runner in subtly incompatible ways. What it does provide is a
way to add or set reporters *outside* of the nominal plug-in/setup system.

···

On Thu, Apr 7, 2016 at 8:24 PM, Ed Howland <ed.howland@gmail.com> wrote:

minitest-reporters-json_reporter version 0.1.1 has been released.

Home, Code: GitHub - edhowland/minitest-reporters-json_reporter: Source of minitest-reporters-json_reporter Gem. An extension to the minitest-reporters Gem that provides JSON output for test runs.
Bugs: Issues · edhowland/minitest-reporters-json_reporter · GitHub

minitest-reporters-json_reporter is an extension to the Ruby
minitest-reporters gem that provides JSON output from your MiniTest or
MiniSpec test run.
JSON has become the lingua franca of interprogram communications. There
are several very good output formats
for MiniTest, but no JSON format until now. If you want to hook up an IDE
or some other automatic
framework, it is natural to send output in some parsable format like XML
or JSON. This extension summarizes the result of
a test run in format like JSON.a test run in an easible consumable format.

Usage is very simple. Just require 'minitest/reporters/json_reporter' in
your test_helper.rb or spec_helper.rb.
This gem also depends on 'minitest' and 'minitest/reporters'. Then, you
should also invoke the use! method of MiniTest::Reporters class with a call
to MiniTest::Reporters::JsonReporter.new.
Here is an example test_helper.rb:

require 'minitest/reporters/json_reporter'
# ... Other requires
require 'minitest/autorun'

MiniTest::Reporters.use! [MiniTest::Reporters::JsonReporter.new]

Here is the URLs for the dependent gems:

gem list minitest

*** LOCAL GEMS ***

minitest (5.7.0, 5.4.3) : minitest | RubyGems.org | your community gem host
minitest-reporters (1.1.8) : minitest-reporters | RubyGems.org | your community gem host
minitest-reporters-json_reporter (0.1.1) :
minitest-reporters-json_reporter | RubyGems.org | your community gem host

The JSON format is sent to stdout when you run a test or all tests.
There are 5 objects contained within this output:

1. metadata - General information about this test run. Includes generator
name, version and the test run completion time in UTC formatted in ISO8601
format.
2. status - Status of the overall test run. Can be 'Success', 'Passed
with skipped tests' or 'Failed'. There also a color attribute: green,
yellow or red - respectfully.
3. statistics - Contains counts of the status of test cases. Includes:
Total, Skipped, Failed Errored and Passed.
4. fails - Array of failed or errored tests. Each object herein contains
the information of each test, its name, class, error message and location
of the test.
5. skips - Array of each skipped. Each object contains the information as
a failed test, including the skip message, if any.

Here is a sample test run:

vagrant ruby report_spec.rb
{"status":{"code":"Failed","color":"red"},"metadata":{"generated_by":"Minitest::Reporters::JsonReporter","version":"0.1.1","time":"2016-04-07T23:31:04Z"},"statistics":{"total":5,"failed":2,"errored":1,"skipped":1,"passed":1},"fails":[{"type":"error","class":"Error","name":"test_0001_anonymous","message":"RuntimeError:
should fail\n report_spec.rb:6:in `block (2 levels) in
<main>'","location":"report_spec.rb:6"},{"type":"failure","class":"second
failure","name":"test_0001_anonymous","message":"Expected: 9\n Actual:
3","location":"report_spec.rb:20"},{"type":"failure","class":"failure","name":"test_0001_anonymous","message":"Expected:
2\n Actual:
1","location":"report_spec.rb:14"}],"skips":[{"type":"skipped","class":"skipped
test","name":"test_0001_anonymous","message":"what a
layabout","location":"report_spec.rb:32"}]}vagrant

You can use JSON parsing tools to get elements or subsets of this format.
One such tool is JQ: Redirecting to jqlang.github.io

Here is how you might view only the color of the test:

$ ruby blog_test.rb | jq -r .status.color
green

Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

--
Austin Ziegler • halostatue@gmail.com • austin@halostatue.ca
http://www.halostatue.ca/http://twitter.com/halostatue

Thanks Austin for the feedback.
I have been asked to fold this separate class into the MiniTest::Reporters
gem via a PR. I am also looking into the normal MiniTest plugin system.
After I do the PR, I will start on creating a general plugin for JSON
reporting.

I would like to know more about packaging MT plugins. Via RubyGems? or just
via GitHub cloning. Also, I am curious about how to handle the actual
output stream. A file? A stdout?

Thanks,

Regards, Ed

Cheers,
Ed

Ed Howland

http://twitter.com/ed_howland

···

On Tue, Apr 12, 2016 at 2:40 PM, Austin Ziegler <halostatue@gmail.com> wrote:

Looks good. It’d be nice to have a version of this that doesn’t depend on
minitest-reporters and just uses native minitest reporting (which is better
in almost every way).

minitest-reporters breaks a lot of normal minitest functionality because
it replaces the runner in subtly incompatible ways. What it does provide is
a way to add or set reporters *outside* of the nominal plug-in/setup system.

On Thu, Apr 7, 2016 at 8:24 PM, Ed Howland <ed.howland@gmail.com> wrote:

minitest-reporters-json_reporter version 0.1.1 has been released.

Home, Code: GitHub - edhowland/minitest-reporters-json_reporter: Source of minitest-reporters-json_reporter Gem. An extension to the minitest-reporters Gem that provides JSON output for test runs.
Bugs:
Issues · edhowland/minitest-reporters-json_reporter · GitHub

minitest-reporters-json_reporter is an extension to the Ruby
minitest-reporters gem that provides JSON output from your MiniTest or
MiniSpec test run.
JSON has become the lingua franca of interprogram communications. There
are several very good output formats
for MiniTest, but no JSON format until now. If you want to hook up an IDE
or some other automatic
framework, it is natural to send output in some parsable format like XML
or JSON. This extension summarizes the result of
a test run in format like JSON.a test run in an easible consumable
format.

Usage is very simple. Just require 'minitest/reporters/json_reporter' in
your test_helper.rb or spec_helper.rb.
This gem also depends on 'minitest' and 'minitest/reporters'. Then, you
should also invoke the use! method of MiniTest::Reporters class with a call
to MiniTest::Reporters::JsonReporter.new.
Here is an example test_helper.rb:

require 'minitest/reporters/json_reporter'
# ... Other requires
require 'minitest/autorun'

MiniTest::Reporters.use! [MiniTest::Reporters::JsonReporter.new]

Here is the URLs for the dependent gems:

gem list minitest

*** LOCAL GEMS ***

minitest (5.7.0, 5.4.3) : minitest | RubyGems.org | your community gem host
minitest-reporters (1.1.8) : minitest-reporters | RubyGems.org | your community gem host
minitest-reporters-json_reporter (0.1.1) :
minitest-reporters-json_reporter | RubyGems.org | your community gem host

The JSON format is sent to stdout when you run a test or all tests.
There are 5 objects contained within this output:

1. metadata - General information about this test run. Includes generator
name, version and the test run completion time in UTC formatted in ISO8601
format.
2. status - Status of the overall test run. Can be 'Success', 'Passed
with skipped tests' or 'Failed'. There also a color attribute: green,
yellow or red - respectfully.
3. statistics - Contains counts of the status of test cases. Includes:
Total, Skipped, Failed Errored and Passed.
4. fails - Array of failed or errored tests. Each object herein contains
the information of each test, its name, class, error message and location
of the test.
5. skips - Array of each skipped. Each object contains the information as
a failed test, including the skip message, if any.

Here is a sample test run:

vagrant ruby report_spec.rb
{"status":{"code":"Failed","color":"red"},"metadata":{"generated_by":"Minitest::Reporters::JsonReporter","version":"0.1.1","time":"2016-04-07T23:31:04Z"},"statistics":{"total":5,"failed":2,"errored":1,"skipped":1,"passed":1},"fails":[{"type":"error","class":"Error","name":"test_0001_anonymous","message":"RuntimeError:
should fail\n report_spec.rb:6:in `block (2 levels) in
<main>'","location":"report_spec.rb:6"},{"type":"failure","class":"second
failure","name":"test_0001_anonymous","message":"Expected: 9\n Actual:
3","location":"report_spec.rb:20"},{"type":"failure","class":"failure","name":"test_0001_anonymous","message":"Expected:
2\n Actual:
1","location":"report_spec.rb:14"}],"skips":[{"type":"skipped","class":"skipped
test","name":"test_0001_anonymous","message":"what a
layabout","location":"report_spec.rb:32"}]}vagrant

You can use JSON parsing tools to get elements or subsets of this format.
One such tool is JQ: Redirecting to jqlang.github.io

Here is how you might view only the color of the test:

$ ruby blog_test.rb | jq -r .status.color
green

Cheers,
Ed

Ed Howland
http://greenprogrammer.wordpress.com
http://twitter.com/ed_howland

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

--
Austin Ziegler • halostatue@gmail.com • austin@halostatue.ca
http://www.halostatue.ca/http://twitter.com/halostatue

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Thanks Austin for the feedback.
I have been asked to fold this separate class into the MiniTest::Reporters
gem via a PR. I am also looking into the normal MiniTest plugin system.
After I do the PR, I will start on creating a general plugin for JSON
reporting.

Thank you. It isn’t that hard to do; you can look at Tim Pope’s fivemat (
https://github.com/tpope/fivemat/\) or any repo by teoljungberg (
https://github.com/teoljungberg/minitest-color\) for a
non-Minitest::Reporters reporter plug-in.

I would like to know more about packaging MT plugins. Via RubyGems? or
just via GitHub cloning. Also, I am curious about how to handle the actual
output stream. A file? A stdout?

Minitest looks for plug-ins exclusively through the $LOAD_PATH. It’s a
fairly easy plug-in system. While I think you could use stdout for
something like minitest-json (for how you use it with viper), I would find
it most useful as a file.

-a

···

On Tue, Apr 12, 2016 at 8:27 PM, Ed Howland <ed.howland@gmail.com> wrote:
--
Austin Ziegler • halostatue@gmail.com • austin@halostatue.ca
http://www.halostatue.ca/http://twitter.com/halostatue

Also worth looking at minitest-ci: GitHub - bhenderson/minitest-ci: This has been moved to circleci.

Minitest::Reporters was built in the pre-Minitest 5.x days, when it wasn’t
possible to have multiple reporters. Unfortunately, there are
incompatibilities and you have to choose *either* Minitest::Reporters or
normal Minitest reporters, and I found some end-of-run incompatibilities
introduced by Minitest::Reporters—which I used for a long time, but have
recently stopped using. I do wish that Ryan would add an easier way to
*manually* configure Minitest at run-time, rather than just relying on
plug-in discovery and command-line argument handling, but there’s a
specific design decision that makes that hard, especially for reporters.

-a

···

On Tue, Apr 12, 2016 at 9:46 PM, Austin Ziegler <halostatue@gmail.com> wrote:

On Tue, Apr 12, 2016 at 8:27 PM, Ed Howland <ed.howland@gmail.com> wrote:

Thanks Austin for the feedback.
I have been asked to fold this separate class into the
MiniTest::Reporters gem via a PR. I am also looking into the normal
MiniTest plugin system. After I do the PR, I will start on creating a
general plugin for JSON reporting.

Thank you. It isn’t that hard to do; you can look at Tim Pope’s fivemat (
https://github.com/tpope/fivemat/\) or any repo by teoljungberg (
https://github.com/teoljungberg/minitest-color\) for a
non-Minitest::Reporters reporter plug-in.

--
Austin Ziegler • halostatue@gmail.com • austin@halostatue.ca
http://www.halostatue.ca/http://twitter.com/halostatue