[Tests] [Draft] Make entire test suite run faster by eliminating excess start/stop
- Move "run_test_server.py" out of the Python SDK directory and make it a package installable in its own right.
- Probably needs a better name. "arvados-service"?
- Callable like "start", "stop", and "reset" (reset to initial fixture state -- see below)
- Manage services individually and as a group (e.g., "start all", "start keep0 keep1 keep2", "reset api")
- Ideally, can (optionally) use the config file from
/dockerto configure the services. (Perhaps make this an explicit step: "configure all", "configure keep0"?)
- Pattern for using factories in addition to (eventually instead of) fixtures in API server. "For this test, I need the following set of objects."
- FactoryGirl is a place to start.
- Rollback apiserver to initial fixture state after each Workbench test case.
- Also: apply a factory in preparation for a Workbench (or other) integration test.
database_cleanergem seems to be aimed at this sort of thing. Perhaps apiserver just needs a button that uses
database_cleaner(or something) to reset to "just the fixtures" state. (refs #2596)
- Probably safer (and easier) to make the mechanism for calling factories go out-of-band, i.e., use a rake task or a signal instead of adding an API call.
- Whatever the mechanism, it should only function when
- Install each component (keepproxy, api server, python sdk, ...)
- Install arvados-service
- Run test suite for each component, calling
arvados-service resetafter each one
arvados-service stop apiwhile running api test suite, etc.