Arvados

1. These are the CI jobs in play, and the changes I've made:

• docker-arvbox: was configured to push the ${git_hash} and 'latest' tags. I removed the 'latest' tag from the push list.
• arvados-cwl-conformance-tests: was already configured to use the ${git_hash} parameter passed through the build pipeline
• arvados-federation-migrate-test: was using the 'latest' tag, I converted it to use ${git_hash} instead

2. Docker hub:

• I have deleted the 'latest' tag on the arvados/arvbox-demo docker image
• Since we are not in the habit of running the 'docker-arvbox' jenkins job as part of the release pipeline, there is no arvados/arvbox-demo docker image for git commit 95594ca8821d6c8f7e13fa788b4dc160fc3bbb53 (that's the 2.0.4 release). I've kicked off a build to push that version to dockerhub: docker-arvbox: #3758

And... apparently that build is simply broken, cf. docker-arvbox: #3758 /console:

++ flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server
# git.arvados.org/arvados.git/sdk/go/arvados
sdk/go/arvados/keep_service.go:155:14: undefined: http.NewRequestWithContext
note: module requires Go 1.13
# github.com/Azure/go-autorest/autorest/adal
/var/lib/gopath/pkg/mod/github.com/!azure/go-autorest/autorest/adal@v0.9.2/token.go:1193:12: undefined: http.NewRequestWithContext
# github.com/msteinert/pam
/var/lib/gopath/pkg/mod/github.com/msteinert/pam@v0.0.0-20190215180659-f29b9f28d6f9/transaction.go:4:31: fatal error: security/pam_appl.h: No such file or directory
 //#include <security/pam_appl.h>
                               ^
compilation terminated.
The command '/bin/sh -c sudo -u arvbox /var/lib/arvbox/service/keep-web/run-service --only-deps' returned a non-zero code: 2

********** !!!!!! docker BUILD FAILED !!!!!! **********

That's probably because 'tools/arvbox/lib/arvbox/docker/go-setup.sh' does this:

flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server

which means it always pulls master, not the desired git version. That's not good.

The correct syntax here would be (this works with go get; need to verify that it also works with go install):

flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server@$GIT_COMMIT

but afaict we don't have the desired git commit exposed inside the build step.

I pushed a small change to the build/run-build-docker-images.sh script so that it can build and push only the 'localdemo' image.

I added a new Jenkins job to the 'Release Pipeline' view: https://ci.arvados.org/view/Release%20Pipeline/job/build-and-release-arvbox-image/. This job builds and pushes the 'arvbox/localdemo' docker image for a specific commit, and will tag that image as the git commit, the 'latest' tag, and the release version number.

I ran it on the 2.1-dev branch here: build-and-release-arvbox-image: #3

We now have an 'arvados/arvbox-demo' image published on dockerhub with the 'latest' and '2.1.0' tags.

Applied in changeset arvados|b699f6a0005d2d692b0394fb4bad26c46df6b498.