Federation implementation roadmap » History » Version 9

« Previous - Version 9/16 (diff) - Next » - Current version
Tom Clegg, 08/21/2018 06:30 PM

Federation implementation roadmap

Use a remote-hosted workflow #13493

  • When retrieving a workflow record for arvados-cwl-runner, API server notices workflow UUID is remote, and fetches it with salted token (instead of looking in local DB)

Mount remote-hosted collection (read-only) - Federated collections

  • (controller) fetch remote-hosted container by UUID (similar to workflow case) (#13993)
  • (controller) rewrite locators in manifest to indicate remote cluster (#13993)
  • (keepstore) if locator indicates remote cluster and data does not exist locally (or local signature is not supplied), fetch from remote using salted token (#13994)

Mount remote-hosted collection (read-only) in a container

  • When creating a container to satisfy a CR, Rails API assumes any PDH appearing in the CR mounts is OK.
  • When running the container, arv-mount (and any other client) will go through the controller to get the collection, and the controller will search the federation for any PDH not found (or not readable by the acting user) in the local DB.

Run some portions of a CWL workflow on a remote cluster

  • When submitting container requests, a-c-r follows workflow/commandline/input UUID hints to choose a cluster.
  • a-c-r submits CRs to its default cluster with "submit to cluster X" flag, and the controller proxies those requests to remote clusters.