Arvados

Before #22489, the shell script(s) that we used to call from jenkins/packer-images/jenkins-image-with-docker.json installed tools on the image so they could run the build/run-build-packages*.sh scripts, and the build-packages-* Jenkins jobs were configured to use build images.

As part of #22489, I made sure these tools were installed from source:tools/ansible/install-test-env.yml — developers should be able to build packages as part of their work, after all. But this means the tools are no longer installed in the build image, instead they're installed in the tests image. After #22489 merged, build-packages-multijob started failing because make isn't installed on the image anymore.

For a quick fix, I have addressed the issue by reconfiguring these jobs to use the tests images instead. In general, I don't understand why we have so many different images. The test-provision images should be as small as possible to provide the best test of installer completeness, sure. Other than that exception, I don't see what we gain. The only technical reason to split up the images that I can see is if we want to keep the disk size of Jenkins instances down, and we need different sets of packages for different tasks to leave enough space available for work. That's at least understandable. But it would cost significant ops time and overhead to make that happen.

We should try to be clear about why we install different pieces of software on the image; e.g., what specific Jenkins job(s) need them. Ansible can help us structure our deployment in a way that makes that clearer. But that can be done in our devops code. We don't need to split the output images to make it clear.

To discuss: are we cool with just having all our internal development jobs run from the same image with all of the tools we need installed? Packer, Docker, Arvados development libraries, package build tools, etc.?

• If yes: merge Ansible playbooks, configure Jenkins to make that happen, retire the jobs to build other images.
• If no: Update jenkins-arvados-packer.yml to install the package build tools (make, createrepo-c, dpkg-dev)
• Either way: Update the ops wiki that documents the different Packer builds with this information.