Feature #2660: [Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop - Arvados

Actions

Feature #2660

closed

Status:

Resolved

Priority:

Normal

Assigned To:

Category:

Tests

Target version:

Story points:

Description

Desirable improvements

Move "run_test_server.py" out of the Python SDK directory and make it a package installable in its own right.
- Probably needs a better name. "arvados-service"?
- Callable like "start", "stop", and "reset" (reset to initial fixture state -- see below)
- Manage services individually and as a group (e.g., "start all", "start keep0 keep1 keep2", "reset api")
- Ideally, can (optionally) use the config file from /docker to configure the services. (Perhaps make this an explicit step: "configure all", "configure keep0"?)
Pattern for using factories in addition to (eventually instead of) fixtures in API server. "For this test, I need the following set of objects."
- FactoryGirl is a place to start.
Rollback apiserver to initial fixture state after each Workbench test case.
- Also: apply a factory in preparation for a Workbench (or other) integration test.
- The database_cleaner gem seems to be aimed at this sort of thing. Perhaps apiserver just needs a button that uses database_cleaner (or something) to reset to "just the fixtures" state. (refs #2596)
- Probably safer (and easier) to make the mechanism for calling factories go out-of-band, i.e., use a rake task or a signal instead of adding an API call.
- Whatever the mechanism, it should only function when RAILS_ENV=test or RAILS_ENV=development.

CI test script will end up looking like

Install each component (keepproxy, api server, python sdk, ...)
Install arvados-service
arvados-service start
Run test suite for each component, calling arvados-service reset after each one
- arvados-service stop api while running api test suite, etc.

Related issues

Actions

Also available in: Atom PDF