Arvados

master @ 9887260

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    1m34.001s
user    0m43.247s
sys     0m10.625s

simplecov @ 6a7ffd9

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/services/api/coverage. 4823 / 5400 LOC (89.31%) covered.

real    1m41.563s
user    0m49.247s
sys     0m10.865s

Comments:

a) test files should probably be excluded from coverage results?

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

d) I think I want this https://github.com/fguillen/simplecov-rcov

Without Workbench coverage

apps/workbench$ time NO_COVERAGE_TEST=true RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    3m31.547s
user    1m55.643s
sys     0m26.122s

With Workbench coverage

apps/workbench$ time RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/apps/workbench/coverage. 1544 / 2195 LOC (70.34%) covered.

real    3m30.293s
user    1m56.131s
sys     0m25.458s

Ward Vandewege wrote:

a) test files should probably be excluded from coverage results?

Mostly useless, yes, although it can point out skipped tests, and the "#times hit" statistic is potentially interesting. Would be nice if the web report could hide these by default.

Fixed (ccdaeb4)

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

Currently (17d83ff) you can do

• Just do tests → NO_COVERAGE_TEST=1 bundle exec rake test
• Generate coverage report for test suite → bundle exec rake test

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

Added (17d83ff) an env var wrapper to the workbench side just like apiserver, and added a rescue for good measure. But "require" might not mean what you think it means in that context: it means "install it during bundle install, but don't load it automatically when I start my app; I'll do require 'foo' if/when I need it."

Ward Vandewege wrote:

d) I think I want this https://github.com/fguillen/simplecov-rcov

Added in 611223d

Review comments:

• everything under vendor/ should most definitely be excluded from coverage report

Otherwise, good to merge I think.

Ward Vandewege wrote:

• everything under vendor/ should most definitely be excluded from coverage report

Yes. That filter gets applied by default (as does "anything outside project root"). According to docs and my local results.