Bug #16930

Don't push arvbox latest, fix jenkins to not depend on latest

Added by Peter Amstutz 22 days ago. Updated about 8 hours ago.

Status:
Feedback
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Only latest stable of "arvbox-demo" should be latest.

Associated revisions

Revision 34cb1075 (diff)
Added by Ward Vandewege 1 day ago

Make run-build-docker-images.sh a little smarter, allow building just
one type of arvbox images (part 2).

refs #16930

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <>

Revision 95dbb33e (diff)
Added by Ward Vandewege about 7 hours ago

Make run-build-docker-images.sh a little smarter, allow building just
one type of arvbox images.

refs #16930

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <>

History

#1 Updated by Peter Amstutz 22 days ago

  • Description updated (diff)

#2 Updated by Ward Vandewege 22 days ago

  • Assigned To set to Ward Vandewege

#3 Updated by Ward Vandewege 22 days ago

  • Status changed from New to In Progress

1. These are the CI jobs in play, and the changes I've made:

  • docker-arvbox: was configured to push the ${git_hash} and 'latest' tags. I removed the 'latest' tag from the push list.
  • arvados-cwl-conformance-tests: was already configured to use the ${git_hash} parameter passed through the build pipeline
  • arvados-federation-migrate-test: was using the 'latest' tag, I converted it to use ${git_hash} instead

2. Docker hub:

  • I have deleted the 'latest' tag on the arvados/arvbox-demo docker image
  • Since we are not in the habit of running the 'docker-arvbox' jenkins job as part of the release pipeline, there is no arvados/arvbox-demo docker image for git commit 95594ca8821d6c8f7e13fa788b4dc160fc3bbb53 (that's the 2.0.4 release). I've kicked off a build to push that version to dockerhub: https://ci.arvados.org/job/docker-arvbox/3758/

And... apparently that build is simply broken, cf. https://ci.arvados.org/job/docker-arvbox/3758/console:

++ flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server
# git.arvados.org/arvados.git/sdk/go/arvados
sdk/go/arvados/keep_service.go:155:14: undefined: http.NewRequestWithContext
note: module requires Go 1.13
# github.com/Azure/go-autorest/autorest/adal
/var/lib/gopath/pkg/mod/github.com/!azure/go-autorest/autorest/adal@v0.9.2/token.go:1193:12: undefined: http.NewRequestWithContext
# github.com/msteinert/pam
/var/lib/gopath/pkg/mod/github.com/msteinert/pam@v0.0.0-20190215180659-f29b9f28d6f9/transaction.go:4:31: fatal error: security/pam_appl.h: No such file or directory
 //#include <security/pam_appl.h>
                               ^
compilation terminated.
The command '/bin/sh -c sudo -u arvbox /var/lib/arvbox/service/keep-web/run-service --only-deps' returned a non-zero code: 2

********** !!!!!! docker BUILD FAILED !!!!!! **********

That's probably because 'tools/arvbox/lib/arvbox/docker/go-setup.sh' does this:

flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server

which means it always pulls master, not the desired git version. That's not good.

The correct syntax here would be (this works with go get; need to verify that it also works with go install):

flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/cmd/arvados-server@$GIT_COMMIT

but afaict we don't have the desired git commit exposed inside the build step.

#4 Updated by Ward Vandewege 16 days ago

  • Target version changed from 2020-10-07 Sprint to 2020-10-21 Sprint

#5 Updated by Ward Vandewege 7 days ago

Okay, now that 2.1 is out, the plan is to add a job to the release pipeline to build and push arvbox docker images with the release tag. We can then also make it tag those as latest.

#6 Updated by Ward Vandewege 1 day ago

  • Target version changed from 2020-10-21 Sprint to 2020-11-04 Sprint

#7 Updated by Ward Vandewege 1 day ago

  • Status changed from In Progress to Resolved

I pushed a small change to the build/run-build-docker-images.sh script so that it can build and push only the 'localdemo' image.

I added a new Jenkins job to the 'Release Pipeline' view: https://ci.arvados.org/view/Release%20Pipeline/job/build-and-release-arvbox-image/. This job builds and pushes the 'arvbox/localdemo' docker image for a specific commit, and will tag that image as the git commit, the 'latest' tag, and the release version number.

I ran it on the 2.1-dev branch here: https://ci.arvados.org/view/Release%20Pipeline/job/build-and-release-arvbox-image/3/

We now have an 'arvados/arvbox-demo' image published on dockerhub with the 'latest' and '2.1.0' tags.

#8 Updated by Peter Amstutz about 9 hours ago

  • Status changed from Resolved to Feedback

#9 Updated by Peter Amstutz about 9 hours ago

Looking at https://hub.docker.com/r/arvados/arvbox-demo/tags the 'latest' is a development release and not the 2.1.0 release so the fix didn't stick.

#10 Updated by Ward Vandewege about 8 hours ago

Peter Amstutz wrote:

Looking at https://hub.docker.com/r/arvados/arvbox-demo/tags the 'latest' is a development release and not the 2.1.0 release so the fix didn't stick.

The 'latest' tag was pushed by this job

https://ci.arvados.org/view/Arvados%20build%20pipeline/job/docker-arvbox/3813/consoleFull

and it shouldn't have done that.

Also available in: Atom PDF