Feature #2660

Updated by Tom Clegg about 5 years ago

Desirable improvements
* Move "run_test_server.py" out of the Python SDK directory and make it a package installable in its own right.
** Probably needs a better name. "arvados-service"?
** Callable like "start", "stop", and "reset" (reset to initial fixture state -- see below)
** Manage services individually and as a group (e.g., "start all", "start keep0 keep1 keep2", "reset api")
** Ideally, can (optionally) use the config file from @/docker@ to configure the services. (Perhaps make this an explicit step: "configure all", "configure keep0"?)
* Pattern for using factories in addition to (eventually instead of) fixtures in API server. "For this test, I need the following set of objects."
** FactoryGirl is a place to start.
* Rollback apiserver to initial fixture state after each Workbench test case.
** Also: apply a factory in preparation for a Workbench (or other) integration test.
The @database_cleaner@ gem seems to be aimed at this sort of thing. Perhaps apiserver just needs a button (which only functions when @RAILS_ENV=test@) that uses @database_cleaner@ (or something) to reset to "just the fixtures" state. (refs #2596)
** Probably safer (and easier) to make the mechanism for calling factories go out-of-band, i.e., use a rake task or a signal instead of adding an API call.
** Whatever the mechanism, it should only function when @RAILS_ENV=test@ or @RAILS_ENV=development@.

CI test script will end up looking like
# Install each component (keepproxy, api server, python sdk, ...)
# Install arvados-service
# @arvados-service start@
# Run test suite for each component, calling @arvados-service reset@ after each one
#* @arvados-service stop api@ while running api test suite, etc.
(From #2596 notes)