Declarative test framework¶

Problem description¶

Currently, running tests in Arvados means running the monolithic run-tests.sh script. The current script has several problems:

• Script is long, complex and hard to maintain
• Script sets up its own environment to run tests is substantially different from production environment
• Setting up the test environment adds overhead, even when there is nothing to do
• Definition of setting up and executing tests is separate from the directory containing the component under test.
• Related tasks such as building and testing packages are entirely separate.

Solution¶

I think we want a declarative way to describe our components for the purposes of building, testing and packaging. This would provide the metadata necessary to drive a framework which is able to build, test and package each component based on that metadata.