Project

General

Profile

Actions

Bug #21367

closed

build-dev-docker-jobs-image.sh won't build

Added by Peter Amstutz 6 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Docker
Story points:
-
Release:
Release relationship:
Auto

Description

I tried to run this on main recently and it didn't build.

The purpose of this script is to facilitate building a development arvados/jobs image from a local branch, primarily for arvados-cwl-runner testing.

First, it decided it needed lxml. This required some new package dependencies. I'm not sure why, it may be a new transitive dependency of something.

Next, it failed with an error related to 'importlib.metadata'.

Then I realized it was still building based on debian 10 'buster' so I changed it the base image to "FROM debian:bullseye-slim".

When I did that, now I can't install anything because of this error:

The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793

Finally, we had discussed migrating things to using virtual environments instead of installing globally. That would be fine here as long as arvados-cwl-runner is in the default $PATH for the container, but that's probably an additional tweak that needs to be made.


Subtasks 1 (0 open1 closed)

Task #21368: Review 21367-arvados-jobs-updateResolvedPeter Amstutz01/09/2024Actions

Related issues

Related to Arvados - Bug #21087: Python 3.7 deprecationResolvedBrett SmithActions
Actions #1

Updated by Peter Amstutz 6 months ago

  • Description updated (diff)
Actions #2

Updated by Brett Smith 6 months ago

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

Actions #3

Updated by Brett Smith 6 months ago

  • Target version changed from Development 2024-01-31 sprint to Development 2024-01-17 sprint
  • Assigned To set to Brett Smith
Actions #4

Updated by Brett Smith 6 months ago

  • Related to Bug #21087: Python 3.7 deprecation added
Actions #5

Updated by Peter Amstutz 6 months ago

Brett Smith wrote in #note-2:

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

So, the build-dev-docker-jobs-image.sh script doesn't actually build arvados/docker/jobs/Dockerfile, it builds arvados/sdk/dev-jobs.dockerfile. So arvados/docker/jobs/Dockerfile did also need to be updated, but that wasn't the one I was stuck on.

Actions #6

Updated by Peter Amstutz 6 months ago

Peter Amstutz wrote in #note-5:

Brett Smith wrote in #note-2:

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

So, the build-dev-docker-jobs-image.sh script doesn't actually build arvados/docker/jobs/Dockerfile, it builds arvados/sdk/dev-jobs.dockerfile. So arvados/docker/jobs/Dockerfile did also need to be updated, but that wasn't the one I was stuck on.

Turns out for the dev jobs image, if I use FROM debian:bullseye instead of bullseye-slim then it works.

Building the jobs image from packages (the part you fixed) also looks good:

docker-jobs-image: #3087

LGTM.

Actions #7

Updated by Brett Smith 6 months ago

  • Status changed from New to In Progress

Peter Amstutz wrote in #note-6:

So, the build-dev-docker-jobs-image.sh script doesn't actually build arvados/docker/jobs/Dockerfile, it builds arvados/sdk/dev-jobs.dockerfile. So arvados/docker/jobs/Dockerfile did also need to be updated, but that wasn't the one I was stuck on.

That's my bad for not reading carefully, since we were moving fast I just leaned on fuzzy matching a little too much and got the wrong thing. Sorry.

build-dev needs more updates anyway for #21230. Since I'm swapped in on all this anyway give me a bit and I'll update the branch with more there too.

Actions #8

Updated by Brett Smith 6 months ago

Brett Smith wrote in #note-7:

build-dev needs more updates anyway for #21230. Since I'm swapped in on all this anyway give me a bit and I'll update the branch with more there too.

21367-arvados-jobs-update @ acf1fa4559c007f127a12c4da3ae644a5ad3a6ac

Actions #9

Updated by Peter Amstutz 6 months ago

Brett Smith wrote in #note-8:

Brett Smith wrote in #note-7:

build-dev needs more updates anyway for #21230. Since I'm swapped in on all this anyway give me a bit and I'll update the branch with more there too.

21367-arvados-jobs-update @ acf1fa4559c007f127a12c4da3ae644a5ad3a6ac

I'm still getting this error with bullseye-slim:

W: GPG error: http://deb.debian.org/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793 0.680 
E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.

With just "bullseye" it doesn't error out.

Since it's a development image there isn't so much incentive to optimize for size.

Actions #10

Updated by Peter Amstutz 6 months ago

Full log:

+ docker build --no-cache -f /home/peter/work/arvados/sdk/dev-jobs.dockerfile -t arvados/jobs:2.8.0.dev20240108214801 /tmp/dev-jobs.lawOlicx
[+] Building 0.8s (6/11)                                                                                                      docker:default
 => [internal] load .dockerignore                                                                                                       0.0s
 => => transferring context: 2B                                                                                                         0.0s
 => [internal] load build definition from dev-jobs.dockerfile                                                                           0.0s
 => => transferring dockerfile: 1.26kB                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                                 0.0s
 => CACHED [1/7] FROM docker.io/library/debian:bullseye-slim                                                                            0.0s
 => [internal] load build context                                                                                                       0.0s
 => => transferring context: 335.70kB                                                                                                   0.0s
 => ERROR [2/7] RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-de  0.7s
------                                                                                                                                       
 > [2/7] RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential:                                                                                 
0.383 Get:1 http://security.debian.org/debian-security bullseye-security InRelease [48.4 kB]                                                 
0.405 Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]                                                                         
0.452 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]                                                                
0.502 Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [264 kB]
0.566 Err:2 http://deb.debian.org/debian bullseye InRelease
0.566   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793
0.640 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
0.640   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
0.654 Reading package lists...
0.680 W: GPG error: http://deb.debian.org/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793
0.680 E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.
0.680 W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
0.680 E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed.
------
dev-jobs.dockerfile:16
--------------------
  15 |     
  16 | >>> RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends \
  17 | >>>     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential
  18 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential" did not complete successfully: exit code: 100
Actions #11

Updated by Brett Smith 6 months ago

Peter Amstutz wrote in #note-9:

I'm still getting this error with bullseye-slim:

[...]

Since it's a development image there isn't so much incentive to optimize for size.

I understand and agree but it's weird that I can't reproduce. Especially since this is a pure apt error, this happens before we even really get the chance to put anything in the Dockerfile.

I won't go ten rounds on this but does it persist if you docker pull debian:bullseye-slim and try again? Is it possible you have a stale image with outdated GPG keys maybe?

Actions #12

Updated by Peter Amstutz 6 months ago

Brett Smith wrote in #note-11:

Peter Amstutz wrote in #note-9:

I'm still getting this error with bullseye-slim:

[...]

Since it's a development image there isn't so much incentive to optimize for size.

I understand and agree but it's weird that I can't reproduce. Especially since this is a pure apt error, this happens before we even really get the chance to put anything in the Dockerfile.

I won't go ten rounds on this but does it persist if you docker pull debian:bullseye-slim and try again? Is it possible you have a stale image with outdated GPG keys maybe?

ahhh, that was it. Yes, after pulling a new bullseye-slim it works.

I ran a workflow with the image and it worked as expected.

LGTM.

Actions #13

Updated by Brett Smith 6 months ago

  • Status changed from In Progress to Resolved
Actions #14

Updated by Peter Amstutz 3 months ago

  • Release set to 70
Actions

Also available in: Atom PDF