Feature #2660

[Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop

Added by Tom Clegg over 5 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
Tests
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Desirable improvements
  • Move "run_test_server.py" out of the Python SDK directory and make it a package installable in its own right.
    • Probably needs a better name. "arvados-service"?
    • Callable like "start", "stop", and "reset" (reset to initial fixture state -- see below)
    • Manage services individually and as a group (e.g., "start all", "start keep0 keep1 keep2", "reset api")
    • Ideally, can (optionally) use the config file from /docker to configure the services. (Perhaps make this an explicit step: "configure all", "configure keep0"?)
  • Pattern for using factories in addition to (eventually instead of) fixtures in API server. "For this test, I need the following set of objects."
    • FactoryGirl is a place to start.
  • Rollback apiserver to initial fixture state after each Workbench test case.
    • Also: apply a factory in preparation for a Workbench (or other) integration test.
    • The database_cleaner gem seems to be aimed at this sort of thing. Perhaps apiserver just needs a button that uses database_cleaner (or something) to reset to "just the fixtures" state. (refs #2596)
    • Probably safer (and easier) to make the mechanism for calling factories go out-of-band, i.e., use a rake task or a signal instead of adding an API call.
    • Whatever the mechanism, it should only function when RAILS_ENV=test or RAILS_ENV=development.
CI test script will end up looking like
  1. Install each component (keepproxy, api server, python sdk, ...)
  2. Install arvados-service
  3. arvados-service start
  4. Run test suite for each component, calling arvados-service reset after each one
    • arvados-service stop api while running api test suite, etc.

Related issues

Related to Arvados - Story #3021: [Tests] Make jenkins tests run faster.Resolved01/05/2015

History

#1 Updated by Tom Clegg over 5 years ago

  • Description updated (diff)

#2 Updated by Tom Clegg about 5 years ago

  • Target version set to Arvados Future Sprints

#3 Updated by Tom Clegg about 5 years ago

  • Subject changed from Rollback apiserver to initial fixture state after each Workbench test case. to [Tests] [Draft] Clean up testing infrastructure
  • Description updated (diff)
  • Category set to Tests

#4 Updated by Tom Clegg almost 5 years ago

  • Subject changed from [Tests] [Draft] Clean up testing infrastructure to [Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop

#5 Updated by Tom Clegg almost 5 years ago

  • Story points changed from 1.0 to 2.0

#6 Updated by Tom Clegg almost 5 years ago

  • Story points deleted (2.0)

#7 Updated by Brett Smith over 4 years ago

#3021 implemented much of the end of the description, but didn't accomplish some of the higher-minded implementation details in here like more cleanly separating run_test_server.py.

#8 Updated by Tom Clegg over 2 years ago

  • Status changed from New to Resolved
  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF