Arvados

Unlike the arvados-dispatch-cloud case, the dispatcher doesn't know which HPC compute node will run the container, and the HPC compute node isn't necessarily even reachable from controller. To work around this, we will make an initial connection in the opposite direction and set up a tunnel.

• crunch-run connects to new controller API arvados/v1/containers/{uuid}/gateway_tunnel, authenticated using the container key (GatewayAuthSecret)
• controller registers its own internalURL as the container’s GatewayAddress, and uses the tunnel to route incoming container_ssh connections to crunch-run through the tunnel
• there can be multiple controller hosts/processes; the container_ssh API on controller A will sometimes need to proxy through the same API on controller B