Federation implementation roadmap » History » Version 14
Peter Amstutz, 09/12/2018 02:36 PM
| 1 | 1 | Tom Clegg | h1. Federation implementation roadmap |
|---|---|---|---|
| 2 | |||
| 3 | 6 | Tom Clegg | Use a remote-hosted workflow #13493 |
| 4 | 1 | Tom Clegg | |
| 5 | * When retrieving a workflow record for arvados-cwl-runner, API server notices workflow UUID is remote, and fetches it with salted token (instead of looking in local DB) |
||
| 6 | |||
| 7 | 5 | Tom Clegg | Mount remote-hosted collection (read-only) - [[Federated collections]] |
| 8 | 1 | Tom Clegg | |
| 9 | 7 | Peter Amstutz | * (controller) fetch remote-hosted container by UUID (similar to workflow case) (#13993) |
| 10 | * (controller) rewrite locators in manifest to indicate remote cluster (#13993) |
||
| 11 | * (keepstore) if locator indicates remote cluster and data does not exist locally (or local signature is not supplied), fetch from remote using salted token (#13994) |
||
| 12 | 5 | Tom Clegg | |
| 13 | Mount remote-hosted collection (read-only) in a container |
||
| 14 | |||
| 15 | 8 | Tom Clegg | * When creating a container to satisfy a CR, Rails API assumes any PDH appearing in the CR mounts is OK. |
| 16 | 10 | Tom Clegg | * When running the container, arv-mount (and any other client) will go through the controller to get the collection. |
| 17 | * The controller will search the federation for any PDH not found (or not readable by the acting user) in the local DB. (#14087) |
||
| 18 | 12 | Peter Amstutz | * API server issues v2 format auth tokens (#14196) |
| 19 | 1 | Tom Clegg | |
| 20 | Run some portions of a CWL workflow on a remote cluster |
||
| 21 | |||
| 22 | 12 | Peter Amstutz | * When submitting container requests, a-c-r follows workflow/commandline/input UUID hints to choose a cluster. |
| 23 | * a-c-r submits CRs to its home cluster with owner_uuid, if owner_uuid is a remote cluster, the controller proxies those requests to remote the cluster (#14197) |
||
| 24 | 14 | Peter Amstutz | * When creating a container request on a remote cluster, submitter can specify a container token with appropriate privileges |
| 25 | 11 | Tom Clegg | * When a-c-r requests a batch of containers, the controller proxies requests to remotes as needed to get a full response (#13619) |
| 26 | 13 | Peter Amstutz | * To prepare final output collection, must be able to copy files from a remote hosted collection to a local one. |
| 27 | ** Requires converting blocks with +R hints to +A hints |
||
| 28 | ** Involves copying blocks from remote cluster to local storage to get local signed locator to create collection. (if necessary, could be done on client / cwl-runner side.) |