Arvados

Added the job in the Arvados Build Pipeline, as a downstream job after build-packages-debian10 success.

Modified the "Execute Shell" step to:

• use the development release (so it the formula uses the arvados-dev repo)
• set VERSION to "latest" (so it picks the newest package in the repo

The jenkins run is failing because the job is misconfigured:

• To test packages built in the development pipeline (https://ci.arvados.org/job/test-saltstack-arvados-formula/, triggered after https://ci.arvados.org/job/build-packages-debian10/), it should:
  • pull packages from the -dev repo
  • install latest package available.
• To test the release pipeline (to test release candidates, jenkins job to be created, to be triggered after https://ci.arvados.org/view/Release%20Pipeline/job/build-publish-packages-testing-debian10/), it should:
  • pull packages from the -testing repo
  • install latest package available.

Fixed the development run (sed regex issue in the job configuration), rerun latest and passed

• Added a new cloud of instances in Jenkins, with "vanilla images" for the OSes we support, named gce2-vanilla
• Added a Centos 7 job https://ci.arvados.org/view/All/job/test-provision-centos7/

All is working now

Added a multi-job pipeline, test-provision, which accepts two parameters:

• git_hash, arvados commit (as we do with all our other jobs) and
• ARVADOS_FORMULA_BRANCH, to pass the arvados-formula branch to pass to the provision.sh script.

• centos7
• debian10
• ubuntu1804
• ubuntu2004

after "build-packages-multijob" completes successfully.

Added the jobs to the "release pipeline"

84e52fd52, branch 17535-test-provision-jenkins

I'm wondering about the duplication of all those nginx sls files: there are three copies of each under `tools/salt-install/config_examples`, and another copy in the arvados-formula repo under `test/salt/pillar/examples`. I looked at some of the differences, I think there may be some room to make all copies more consistent with each other.

And maybe some trimming/consolidation is also in order. Anyway, none of that should block this, LGTM.

I had the same thoughts regarding duplication. I just hate it.

The examples in the formula are there for those that expect to use the formula without using the provision scripts.

And the duplication in the provision scripts are because each directory is self-contained and to simplify the deployment process, I suggested users just choose a configuration variant and copy the dir to a 'working dir'.

De-duplicating means we will not allow users to just 'copy the whole dir to a place, modify it and use it', unless we find an alternative way to do this?