Crunch2 installation¶

(DRAFT -- when ready, this will move to doc.arvados.org→install)

• Table of contents
• Crunch2 installation
  • Set up a crunch-dispatch service
  • Install crunch-run on all compute nodes
  • Enable cgroup accounting on all compute nodes
  • Configure docker
  • Test the dispatcher

Set up a crunch-dispatch service¶

Currently, dispatching containers via SLURM is supported.

Install crunch-dispatch-slurm on a node that can submit SLURM jobs. This can be the slurm controller node, a worker node, or any other node that has the appropriate SLURM/munge configuration.

sudo apt-get install crunch-dispatch-slurm

Create a privileged token for use by the dispatcher. If you have multiple dispatch processes, you should give each one a different token.

apiserver:~$ cd /var/www/arvados-api/current
apiserver:/var/www/arvados-api/current$ sudo -u webserver-user RAILS_ENV=production bundle exec script/create_superuser_token.rb
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

Save the token on the dispatch node, in /etc/sv/crunch-dispatch-slurm/env/ARVADOS_API_TOKEN

Example runit script (/etc/sv/crunch-dispatch-slurm/run):

#!/bin/sh
set -e
exec 2>&1

export ARVADOS_API_HOST=uuid_prefix.your.domain

exec chpst -e ./env -u crunch crunch-dispatch-slurm

Example runit logging script (/etc/sv/crunch-dispatch-slurm/log/run):

#!/bin/sh
set -e
[ -d main ] || mkdir main
exec svlogd -tt ./main

Ensure the crunch user exists -- and has the same UID, GID, and home directory -- on the dispatch node and all SLURM compute nodes. Ensure the crunch user can run docker containers on SLURM compute nodes.

Install crunch-run on all compute nodes¶

sudo apt-get install crunch-run

Enable cgroup accounting on all compute nodes¶

(This requirement isn't new for crunch2/containers, but it seems to be a FAQ. The Docker install guide mentions it's optional and performance-degrading, so it's not too surprising if people skip it. Perhaps we should say why/when it's a good idea to enable it?)

Check https://docs.docker.com/engine/installation/linux/ for instructions specific to your distribution.

For example, on Ubuntu:

1. Update /etc/default/grub to include:

   GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" 
   

2. sudo update-grub
3. Reboot

Configure docker¶

Unchanged from current docs.

Test the dispatcher¶

On the dispatch node, monitor the crunch-dispatch logs.

dispatch-node$ tail -F /etc/sv/crunch-dispatch-slurm/log/main/current

On a shell VM, install a docker image for testing.

user@shellvm:~$ arv-keepdocker busybox

On a shell VM, run a trivial container.

user@shellvm:~$ arv container_request create --container-request '{
  "name":            "test",
  "state":           "Committed",
  "priority":        1,
  "container_image": "busybox",
  "command":         ["echo", "OK"]
}'