Arvados

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

Brett Smith wrote in #note-2:

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

So, the build-dev-docker-jobs-image.sh script doesn't actually build arvados/docker/jobs/Dockerfile, it builds arvados/sdk/dev-jobs.dockerfile. So arvados/docker/jobs/Dockerfile did also need to be updated, but that wasn't the one I was stuck on.

Peter Amstutz wrote in #note-5:

Brett Smith wrote in #note-2:

21367-arvados-jobs-update @ 623b01534a135a964cc473b45cf8333a780a1b74 - I could run the Jenkins build, but I get the impression you'd prefer to test locally, at least to start.

So, the build-dev-docker-jobs-image.sh script doesn't actually build arvados/docker/jobs/Dockerfile, it builds arvados/sdk/dev-jobs.dockerfile. So arvados/docker/jobs/Dockerfile did also need to be updated, but that wasn't the one I was stuck on.

Turns out for the dev jobs image, if I use FROM debian:bullseye instead of bullseye-slim then it works.

Building the jobs image from packages (the part you fixed) also looks good:

docker-jobs-image: #3087

LGTM.

Brett Smith wrote in #note-7:

build-dev needs more updates anyway for #21230. Since I'm swapped in on all this anyway give me a bit and I'll update the branch with more there too.

21367-arvados-jobs-update @ acf1fa4559c007f127a12c4da3ae644a5ad3a6ac

Brett Smith wrote in #note-8:

Brett Smith wrote in #note-7:

build-dev needs more updates anyway for #21230. Since I'm swapped in on all this anyway give me a bit and I'll update the branch with more there too.

21367-arvados-jobs-update @ acf1fa4559c007f127a12c4da3ae644a5ad3a6ac

I'm still getting this error with bullseye-slim:

W: GPG error: http://deb.debian.org/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793 0.680 
E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.

With just "bullseye" it doesn't error out.

Since it's a development image there isn't so much incentive to optimize for size.

Full log:

+ docker build --no-cache -f /home/peter/work/arvados/sdk/dev-jobs.dockerfile -t arvados/jobs:2.8.0.dev20240108214801 /tmp/dev-jobs.lawOlicx
[+] Building 0.8s (6/11)                                                                                                      docker:default
 => [internal] load .dockerignore                                                                                                       0.0s
 => => transferring context: 2B                                                                                                         0.0s
 => [internal] load build definition from dev-jobs.dockerfile                                                                           0.0s
 => => transferring dockerfile: 1.26kB                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                                 0.0s
 => CACHED [1/7] FROM docker.io/library/debian:bullseye-slim                                                                            0.0s
 => [internal] load build context                                                                                                       0.0s
 => => transferring context: 335.70kB                                                                                                   0.0s
 => ERROR [2/7] RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-de  0.7s
------                                                                                                                                       
 > [2/7] RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential:                                                                                 
0.383 Get:1 http://security.debian.org/debian-security bullseye-security InRelease [48.4 kB]                                                 
0.405 Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]                                                                         
0.452 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]                                                                
0.502 Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [264 kB]
0.566 Err:2 http://deb.debian.org/debian bullseye InRelease
0.566   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793
0.640 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
0.640   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
0.654 Reading package lists...
0.680 W: GPG error: http://deb.debian.org/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 605C66F00D6C9793
0.680 E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.
0.680 W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
0.680 E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed.
------
dev-jobs.dockerfile:16
--------------------
  15 |     
  16 | >>> RUN DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends \
  17 | >>>     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential
  18 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get update -q && apt-get install -qy --no-install-recommends     git python3-dev python3-venv libcurl4-gnutls-dev libgnutls28-dev nodejs build-essential" did not complete successfully: exit code: 100

Peter Amstutz wrote in #note-9:

I'm still getting this error with bullseye-slim:

[...]

Since it's a development image there isn't so much incentive to optimize for size.

I understand and agree but it's weird that I can't reproduce. Especially since this is a pure apt error, this happens before we even really get the chance to put anything in the Dockerfile.

I won't go ten rounds on this but does it persist if you docker pull debian:bullseye-slim and try again? Is it possible you have a stale image with outdated GPG keys maybe?

Brett Smith wrote in #note-11:

Peter Amstutz wrote in #note-9:

I'm still getting this error with bullseye-slim:

[...]

Since it's a development image there isn't so much incentive to optimize for size.

I understand and agree but it's weird that I can't reproduce. Especially since this is a pure apt error, this happens before we even really get the chance to put anything in the Dockerfile.

I won't go ten rounds on this but does it persist if you docker pull debian:bullseye-slim and try again? Is it possible you have a stale image with outdated GPG keys maybe?

ahhh, that was it. Yes, after pulling a new bullseye-slim it works.

I ran a workflow with the image and it worked as expected.

LGTM.