Story #4754

[Tests] Establish performance test benchmarks.

Added by Radhika Chippada over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
-
Target version:
Start date:
12/09/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.50 h)
Story points:
0.5

Description

Add a performance test task to jenkins testing.

Establish performance testing benchmarks for home page and search.


Subtasks

Task #4822: Review 4754-performance-TCResolvedTim Pierce

Task #4757: Do what is needed to run the browsing_test.rb (and also add search_test)ResolvedRadhika Chippada

Task #4758: Add a jenkins task to run workbench performance testsResolvedRadhika Chippada

Task #4765: Add "performance" configuration similar to that of "diagnostics" so that performance benchmarking can be run against other environments such as production.ResolvedRadhika Chippada

Task #4796: Configure a jenkins task to run workbench performance test against production workbenchResolvedWard Vandewege

Task #4797: Review branches: 4754-performance-benchmarks (arvados) and 4754-add-workbench-performance-test (arvados-dev)ResolvedTom Clegg

Associated revisions

Revision 3fc2ff90
Added by Radhika Chippada over 6 years ago

refs #4754
Merge branch '4754-performance-benchmarks'

Revision 511d0877
Added by Radhika Chippada over 6 years ago

refs #4754
Merge branch '4754-add-workbench-performance-test'

Revision 511d0877
Added by Radhika Chippada over 6 years ago

refs #4754
Merge branch '4754-add-workbench-performance-test'

Revision 896d01ea
Added by Tom Clegg over 6 years ago

Merge branch '4754-performance-TC' closes #4754

Revision 525d5d63 (diff)
Added by Ward Vandewege over 6 years ago

Make sure the performance testing gems are available when workbench is
run in the performance rails environment.

refs #4754

Revision c882575c (diff)
Added by Ward Vandewege over 6 years ago

We need tmp/cache to exist before we can run performance tests

refs #4754

Revision abcdf588 (diff)
Added by Ward Vandewege over 6 years ago

Add a script to run our performance tests.

refs #4754

Revision abcdf588 (diff)
Added by Ward Vandewege over 6 years ago

Add a script to run our performance tests.

refs #4754

Revision c9f5db97 (diff)
Added by Ward Vandewege over 6 years ago

Revert "We need tmp/cache to exist before we can run performance tests"

refs #4754

This reverts commit c882575c856e01313cf2caf2e4ead1f27bfb33ae.

Revision d65b683a (diff)
Added by Ward Vandewege over 6 years ago

Also add minitest and friends to the performance group.

Refs #4754

Revision 41d0494a (diff)
Added by Ward Vandewege over 6 years ago

Run the performance tests the documented way. This is possible after
commit d65b683af52e072b3d179b6f32edfbf37e108011 has been merged in the
arvados repository.

refs #4754

Revision 41d0494a (diff)
Added by Ward Vandewege over 6 years ago

Run the performance tests the documented way. This is possible after
commit d65b683af52e072b3d179b6f32edfbf37e108011 has been merged in the
arvados repository.

refs #4754

Revision 6c63e6c0 (diff)
Added by Ward Vandewege over 6 years ago

Make sure tmp/cache exists before we run our performance test suite.

refs #4754

Revision 6c63e6c0 (diff)
Added by Ward Vandewege over 6 years ago

Make sure tmp/cache exists before we run our performance test suite.

refs #4754

Revision fc8e5729 (diff)
Added by Ward Vandewege over 6 years ago

Fix typo.

refs #4754

History

#1 Updated by Radhika Chippada over 6 years ago

  • Tracker changed from Bug to Story
  • Description updated (diff)
  • Status changed from New to In Progress
  • Assigned To set to Radhika Chippada
  • Target version set to 2014-12-10 sprint

#2 Updated by Radhika Chippada over 6 years ago

  • Use the following command to run the performance test
RAILS_ENV=test bundle exec rake test:benchmark
  • The following is the benchmark report after running the tests
BrowsingTest#test_homepage (9.95 sec warmup)
           wall_time: 2.01 sec
SearchTest#test_search (3.65 sec warmup)
           wall_time: 3.64 sec

#3 Updated by Ward Vandewege over 6 years ago

  • Story points set to 0.5

#4 Updated by Ward Vandewege over 6 years ago

  • Target version changed from 2014-12-10 sprint to 2015-01-07 sprint

#5 Updated by Radhika Chippada over 6 years ago

There are two modes in which performance benchmark testing can be done: one to run in test mode and one to run against a running workbench instance (such as production).

  1. Similar to other integration tests using the following command. This will run tests located in the arvados/apps/workbench/test/performance directory in RAILS_ENV=test mode. A jenkins task is also added for this.
    RAILS_ENV=test bundle exec rake test:benchmark
  2. Against a configured workbench url using "RAILS_ENV=performance". Edit the performance block in application.default.yml to configure the workbench url and user token to be used.
    RAILS_ENV=performance bundle exec rake TEST=test/performance/*.rb
  • After test is completed, the benchmark data will be stored in tmp/performance directory.

#6 Updated by Radhika Chippada over 6 years ago

Merged code into master after getting Ward's green signal on the arvados-dev update.

#7 Updated by Tim Pierce over 6 years ago

4754-performance-TC LGTM at 0b2005c4d

#8 Updated by Anonymous over 6 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 83 to 100

Applied in changeset arvados|commit:896d01ea7b3fed8b621ff930893d3ec806931dc9.

Also available in: Atom PDF