Project

General

Profile

Actions
Copy link

Feature #2660

closed

[Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop

Added by Tom Clegg about 10 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
Tests
Target version:
-
Story points:
-

Description

Desirable improvements
  • Move "run_test_server.py" out of the Python SDK directory and make it a package installable in its own right.
    • Probably needs a better name. "arvados-service"?
    • Callable like "start", "stop", and "reset" (reset to initial fixture state -- see below)
    • Manage services individually and as a group (e.g., "start all", "start keep0 keep1 keep2", "reset api")
    • Ideally, can (optionally) use the config file from /docker to configure the services. (Perhaps make this an explicit step: "configure all", "configure keep0"?)
  • Pattern for using factories in addition to (eventually instead of) fixtures in API server. "For this test, I need the following set of objects."
    • FactoryGirl is a place to start.
  • Rollback apiserver to initial fixture state after each Workbench test case.
    • Also: apply a factory in preparation for a Workbench (or other) integration test.
    • The database_cleaner gem seems to be aimed at this sort of thing. Perhaps apiserver just needs a button that uses database_cleaner (or something) to reset to "just the fixtures" state. (refs #2596)
    • Probably safer (and easier) to make the mechanism for calling factories go out-of-band, i.e., use a rake task or a signal instead of adding an API call.
    • Whatever the mechanism, it should only function when RAILS_ENV=test or RAILS_ENV=development.
CI test script will end up looking like
  1. Install each component (keepproxy, api server, python sdk, ...)
  2. Install arvados-service
  3. arvados-service start
  4. Run test suite for each component, calling arvados-service reset after each one
    • arvados-service stop api while running api test suite, etc.

Related issues

Related to Arvados - Idea #3021: [Tests] Make jenkins tests run faster.Resolved01/05/2015Actions
Actions
Copy link
 #1

Updated by Tom Clegg about 10 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Tom Clegg almost 10 years ago

  • Target version set to Arvados Future Sprints
Actions
Copy link
 #3

Updated by Tom Clegg over 9 years ago

  • Subject changed from Rollback apiserver to initial fixture state after each Workbench test case. to [Tests] [Draft] Clean up testing infrastructure
  • Description updated (diff)
  • Category set to Tests
Actions
Copy link
 #4

Updated by Tom Clegg over 9 years ago

  • Subject changed from [Tests] [Draft] Clean up testing infrastructure to [Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop
Actions
Copy link
 #5

Updated by Tom Clegg over 9 years ago

  • Story points changed from 1.0 to 2.0
Actions
Copy link
 #6

Updated by Tom Clegg over 9 years ago

  • Story points deleted (2.0)
Actions
Copy link
 #7

Updated by Brett Smith about 9 years ago

#3021 implemented much of the end of the description, but didn't accomplish some of the higher-minded implementation details in here like more cleanly separating run_test_server.py.

Actions
Copy link
 #8

Updated by Tom Clegg about 7 years ago

  • Status changed from New to Resolved
  • Target version deleted (Arvados Future Sprints)
Actions
Copy link

Also available in: Atom PDF