Feature #22489
closed
Convert packer-build-jenkins-image-arvados-tests to use Ansible
Added by Brett Smith 2 months ago.
Updated about 1 month ago.
Release relationship:
Auto
Description
This Packer template currently runs a shell script that runs, among other things, arvados-server install -test. A lot of this can be replaced with the install-test-env.yml playbook. The rest of the script (user setup, caching) should be straightforward to port to Ansible. Ideally the Packer template should run a single Ansible playbook that does:
- initial user and repository setup
- run
install-test-env.yml from the arvados repository via include
- test setup and caching
Doing this is a prerequisite to removing arvados-server install.
- Target version set to Development 2025-02-12
- Assigned To set to Brett Smith
- Blocked by Bug #22556: `run-tests.sh --only install` makes a bad GEMHOME by uninstalling arvados-google-api-client out from under arvados added
- Status changed from New to In Progress
arvados-dev branch 22489-jenkins-tests-ansible @ commit:a0cdea4008019c2d269f5843282a0a0646eabe7a
arvados branch 22489-install-test-env @ a558a28d3011adefd1348b5710da9e89f11c7801
Base image build: packer-build-jenkins-image-build: #64 
Test image build: packer-build-jenkins-image-arvados-tests: #111 
developer-run-tests on the new test image: developer-run-tests: #4655 
- run-test-remainder retry: run-tests-remainder: #4957 
- All agreed upon points are implemented / addressed.
- Yes. Because
jenkins-image-arvados-tests.sh includes jenkins-image-with-docker.sh, I converted both into Ansible playbooks, and kept the same relationship between them.
- Anything not implemented (discovered or discussed during work) has a follow-up story.
- In the process of doing this, I found a lot of things that I think could be cleaned up. These branches are already big enough that I have filed separate follow-up tickets for all of these. See #22557, #22558, #22559, #22560.
- In order to get these new builds going, I had to do some weird configuration to the Jenkins shell for the Packer image builds. I can clean that up after the branches are merged, but it doesn't make sense to do it before that.
- Code is tested and passing, both automated and manual, what manual testing was done is described
- Documentation has been updated.
- Strictly internal. If you're happy with these results I will probably remove the "this is a prototype" comment in
install-test-env.yml (since this represents us using it in production) and update the developer wiki to match. But again, I don't want to put the cart before the horse.
- Behaves appropriately at the intended scale (describe intended scale).
- Considered backwards and forwards compatibility issues between client and server.
- Follows our coding standards and GUI style guidelines.
- N/A (no style guide for shell, YAML, or JSON)
Note I have reverted our Jenkins configuration to use the previous image for new test runs, so other branches don't trip over the #22556 issue. The fix for that is in this branch in build/run-tests.sh.
- Target version changed from Development 2025-02-12 to Development 2025-02-26
Both branches LGTM, thanks.
- Related to Bug #22579: Have most Jenkins jobs use a single shared image jenkins-image-arvados-tests added
- Status changed from In Progress to Resolved
I was leaving this open to actually switch Jenkins over to the new compute image, but now as a practical matter that's going to happen as part of #22509 instead, so this is done.
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by