Feature #2733

Generate test coverage report in CI pipeline

Added by Tom Clegg over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
-
Start date:
05/06/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.50 h)
Story points:
1.0

Subtasks

Task #2745: Review 2733-coverage-report branchResolvedTom Clegg

Associated revisions

Revision b2979b0d
Added by Tom Clegg over 7 years ago

Merge branch '2733-coverage-report'

closes #2733

History

#1 Updated by Tom Clegg over 7 years ago

master @ 9887260

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    1m34.001s
user    0m43.247s
sys     0m10.625s

simplecov @ 6a7ffd9

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/services/api/coverage. 4823 / 5400 LOC (89.31%) covered.

real    1m41.563s
user    0m49.247s
sys     0m10.865s

#2 Updated by Ward Vandewege over 7 years ago

Comments:

a) test files should probably be excluded from coverage results?

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

d) I think I want this https://github.com/fguillen/simplecov-rcov

#3 Updated by Tom Clegg over 7 years ago

Without Workbench coverage

apps/workbench$ time NO_COVERAGE_TEST=true RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    3m31.547s
user    1m55.643s
sys     0m26.122s

With Workbench coverage

apps/workbench$ time RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/apps/workbench/coverage. 1544 / 2195 LOC (70.34%) covered.

real    3m30.293s
user    1m56.131s
sys     0m25.458s

#4 Updated by Tom Clegg over 7 years ago

Ward Vandewege wrote:

a) test files should probably be excluded from coverage results?

Mostly useless, yes, although it can point out skipped tests, and the "#times hit" statistic is potentially interesting. Would be nice if the web report could hide these by default.

Fixed (ccdaeb4)

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

Currently (17d83ff) you can do

  • Just do tests → NO_COVERAGE_TEST=1 bundle exec rake test
  • Generate coverage report for test suite → bundle exec rake test

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

Added (17d83ff) an env var wrapper to the workbench side just like apiserver, and added a rescue for good measure. But "require" might not mean what you think it means in that context: it means "install it during bundle install, but don't load it automatically when I start my app; I'll do require 'foo' if/when I need it."

#5 Updated by Tom Clegg over 7 years ago

Ward Vandewege wrote:

d) I think I want this https://github.com/fguillen/simplecov-rcov

Added in 611223d

#6 Updated by Tom Clegg over 7 years ago

  • Target version set to 2014-05-07 Storing and Organizing Data

#7 Updated by Ward Vandewege over 7 years ago

Review comments:

  • everything under vendor/ should most definitely be excluded from coverage report

Otherwise, good to merge I think.

#8 Updated by Tom Clegg over 7 years ago

Ward Vandewege wrote:

  • everything under vendor/ should most definitely be excluded from coverage report

Yes. That filter gets applied by default (as does "anything outside project root"). According to docs and my local results.

#9 Updated by Anonymous over 7 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:b2979b0db0b6b5ee7ad3001ffcc8f76ff4d96f05.

Also available in: Atom PDF