Bug #11222

Docker load time on compute node is too long

Added by Bryan Cosca over 4 years ago. Updated 3 months ago.

Assigned To:
Target version:
Start date:
Due date:
% Done:


Estimated time:
Story points:


using a-c-r --local, it took 10 minutes to load a 3GB Docker image

2017-03-07_20:59:33 e51c5-8i9sb-th04z0sonfpxwma 59269  stderr starting: ['srun','--nodelist=compute1','/bin/bash','-o','pipefail','-ec',' if /usr/bin/docker.io images -q --no-trunc --all | grep -xF 9df946de46be6418fa9b82e906981677b24ea3206bc7561c517d1a7dbdbeebca >/dev/null; then     exit 0 fi declare -a exit_codes=("${PIPESTATUS[@]}") if [ 0 != "${exit_codes[0]}" ]; then    exit "${exit_codes[0]}"  # `docker images` failed elif [ 1 != "${exit_codes[1]}" ]; then    exit "${exit_codes[1]}"  # `grep` encountered an error else    # Everything worked fine, but grep didn\'t find the image on this host.    arv-get 62925a53810ef8a5d57b54e49f6628c4\\+2058\\/9df946de46be6418fa9b82e906981677b24ea3206bc7561c517d1a7dbdbeebca\\.tar | /usr/bin/docker.io load fi ']
2017-03-07_21:09:03 e51c5-8i9sb-th04z0sonfpxwma 59269  load docker image: exit 0

I don't really have an idea for a long term fix, but I think having the compute nodes up for ~ 15 minutes instead of shutting them down immediately would really help since the docker image would already be loaded on it.


#1 Updated by Bryan Cosca over 4 years ago

  • Project changed from Support to Arvados Private

#2 Updated by Bryan Cosca over 4 years ago

  • Project changed from Arvados Private to Arvados

#3 Updated by Tom Morris about 4 years ago

  • Target version set to Arvados Future Sprints

#4 Updated by Peter Amstutz 3 months ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF