Project

General

Profile

Actions

Feature #19166

open

Container shell support for SLURM and LSF dispatchers

Added by Peter Amstutz about 1 month ago. Updated 5 days ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
06/24/2022
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

Unlike the arvados-dispatch-cloud case, the dispatcher doesn't know which HPC compute node will run the container, and the HPC compute node isn't necessarily even reachable from controller. To work around this, we will make an initial connection in the opposite direction and set up a tunnel.

  • crunch-run connects to new controller API arvados/v1/containers/{uuid}/gateway_tunnel, authenticated using the container key (GatewayAuthSecret)
  • controller registers its own internalURL as the container’s GatewayAddress, and uses the tunnel to route incoming container_ssh connections to crunch-run through the tunnel
  • there can be multiple controller hosts/processes; the container_ssh API on controller A will sometimes need to proxy through the same API on controller B

Subtasks 1 (1 open0 closed)

Task #19184: Review 19166-gateway-tunnelIn ProgressPeter Amstutz06/24/2022

Actions

Related issues

Related to Arvados Epics - Story #17207: External access to web services running in containersNew12/01/202203/31/2023

Actions
Actions #1

Updated by Peter Amstutz about 1 month ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 23 days ago

  • Target version changed from 2022-07-20 to 2022-06-22 Sprint
Actions #4

Updated by Peter Amstutz 21 days ago

  • Assigned To set to Tom Clegg
Actions #5

Updated by Tom Clegg 13 days ago

  • Related to Story #17207: External access to web services running in containers added
Actions #6

Updated by Tom Clegg 13 days ago

  • Status changed from New to In Progress
  • Description updated (diff)
Actions #8

Updated by Peter Amstutz 7 days ago

  • Target version changed from 2022-06-22 Sprint to 2022-07-06
Actions #9

Updated by Tom Clegg 6 days ago

19166-gateway-tunnel @ 3fae0f0626c5152a5aa6f39f0874f0190f2131db -- developer-run-tests: #3196

Includes a doc page about HPC with a description of how the multiplex-tunnel setup works, and an update to the InternalURLs info in the install docs to reflect that it relies on controller-to-controller connections.

Actions

Also available in: Atom PDF