Project

General

Profile

Actions

Feature #2733

closed

Generate test coverage report in CI pipeline

Added by Tom Clegg almost 9 years ago. Updated almost 9 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
-
Start date:
05/06/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.50 h)
Story points:
1.0

Subtasks 1 (0 open1 closed)

Task #2745: Review 2733-coverage-report branchResolvedTom Clegg05/06/2014

Actions
Actions #1

Updated by Tom Clegg almost 9 years ago

master @ 9887260

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    1m34.001s
user    0m43.247s
sys     0m10.625s

simplecov @ 6a7ffd9

services/api$ time RAILS_ENV=test bundle exec rake test
...
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/services/api/coverage. 4823 / 5400 LOC (89.31%) covered.

real    1m41.563s
user    0m49.247s
sys     0m10.865s
Actions #2

Updated by Ward Vandewege almost 9 years ago

Comments:

a) test files should probably be excluded from coverage results?

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

d) I think I want this https://github.com/fguillen/simplecov-rcov

Actions #3

Updated by Tom Clegg almost 9 years ago

Without Workbench coverage

apps/workbench$ time NO_COVERAGE_TEST=true RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]

real    3m31.547s
user    1m55.643s
sys     0m26.122s

With Workbench coverage

apps/workbench$ time RAILS_ENV=test bundle exec rake test
ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-linux]
Coverage report generated for Functional Tests, Integration Tests, Unit Tests to /home/tom/arvados/apps/workbench/coverage. 1544 / 2195 LOC (70.34%) covered.

real    3m30.293s
user    1m56.131s
sys     0m25.458s
Actions #4

Updated by Tom Clegg almost 9 years ago

Ward Vandewege wrote:

a) test files should probably be excluded from coverage results?

Mostly useless, yes, although it can point out skipped tests, and the "#times hit" statistic is potentially interesting. Would be nice if the web report could hide these by default.

Fixed (ccdaeb4)

b) I want to be able to run coverage tests as a separate step in the build pipeline; rake coverage

Currently (17d83ff) you can do

  • Just do tests → NO_COVERAGE_TEST=1 bundle exec rake test
  • Generate coverage report for test suite → bundle exec rake test

c) the 'require simplecov' should be wrapped in a begin/rescue/end that we can trap the exception when it's not installed (you have it listed as not required in Gemfile)

Added (17d83ff) an env var wrapper to the workbench side just like apiserver, and added a rescue for good measure. But "require" might not mean what you think it means in that context: it means "install it during bundle install, but don't load it automatically when I start my app; I'll do require 'foo' if/when I need it."

Actions #5

Updated by Tom Clegg almost 9 years ago

Ward Vandewege wrote:

d) I think I want this https://github.com/fguillen/simplecov-rcov

Added in 611223d

Actions #6

Updated by Tom Clegg almost 9 years ago

  • Target version set to 2014-05-07 Storing and Organizing Data
Actions #7

Updated by Ward Vandewege almost 9 years ago

Review comments:

  • everything under vendor/ should most definitely be excluded from coverage report

Otherwise, good to merge I think.

Actions #8

Updated by Tom Clegg almost 9 years ago

Ward Vandewege wrote:

  • everything under vendor/ should most definitely be excluded from coverage report

Yes. That filter gets applied by default (as does "anything outside project root"). According to docs and my local results.

Actions #9

Updated by Anonymous almost 9 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:b2979b0db0b6b5ee7ad3001ffcc8f76ff4d96f05.

Actions

Also available in: Atom PDF