Container secret mounts » History » Version 5

Peter Amstutz, 03/01/2018 04:06 PM

1 1 Tom Clegg
h1. Container secret mounts
2 1 Tom Clegg
3 2 Tom Clegg
This is a proposed modification to [[Containers API]].
4 2 Tom Clegg
5 2 Tom Clegg
Add a "secret_mounts" serialized field to containers and container requests.
6 2 Tom Clegg
7 2 Tom Clegg
"secret_mounts" has the same form and behavior as "mounts", except:
8 2 Tom Clegg
* Only literal content is allowed (kind=text or kind=json)
9 2 Tom Clegg
* Current value is never returned in a container request or container API response
10 2 Tom Clegg
* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_mounts@) which must be authenticated by the container's own runtime token
11 1 Tom Clegg
* Never appears in container logs
12 1 Tom Clegg
* Never appears in the Arvados logs table
13 1 Tom Clegg
* Never appears in websocket updates
14 1 Tom Clegg
* Never appears in API server request logs
15 1 Tom Clegg
16 2 Tom Clegg
It is an error for the same key (mount path) to appear in both mounts and secret_mounts. It should not be possible to _commit_ a container request with this error condition, although it might be possible to _save._
17 2 Tom Clegg
18 2 Tom Clegg
A secret_mounts key (path) cannot be
19 2 Tom Clegg
* equal to the container's output path,
20 2 Tom Clegg
* a descendant of the container's output path, or
21 2 Tom Clegg
* an ancestor of the container's output path (currently, this is a moot point because secret mounts are not directories)
22 2 Tom Clegg
23 4 Peter Amstutz
(PA) I would modify this restriction.  Secret file mounts should be allowed to appear under the output directory, but must be _excluded_ when capturing output.  The reasoning is that this aligns better with the CWL InitialWorkDir feature which pre-populates the output directory.  Pre-populating other directories is technically legal but less portable (only works if you are using containers, which is an ok assumption for Arvados but not for CWL generally).
24 3 Peter Amstutz
25 2 Tom Clegg
For clarity, some ways in which secret_mounts behaves like mounts are:
26 2 Tom Clegg
* Non-identical secret_mounts disqualifies a container for reuse. The mere existence of secret_mounts does not disqualify.
27 2 Tom Clegg
* secret_mounts can be set via container_requests#create and container_requests#update APIs
28 2 Tom Clegg
* secret_mounts cannot be null, but can be an empty hash
29 2 Tom Clegg
* keys of secret_mounts are paths in the container's filesystem, and always begin with "/"
30 5 Peter Amstutz
31 5 Peter Amstutz
h3. Secret environment
32 5 Peter Amstutz
33 5 Peter Amstutz
Add a "secret_environment" serialized field to containers and container requests.
34 5 Peter Amstutz
35 5 Peter Amstutz
"secret_environment" has the same form and behavior as "environment", except:
36 5 Peter Amstutz
37 5 Peter Amstutz
* Current value is never returned in a container request or container API response
38 5 Peter Amstutz
* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_environment@) which must be authenticated by the container's own runtime token
39 5 Peter Amstutz
* Never appears in container logs
40 5 Peter Amstutz
* Never appears in the Arvados logs table
41 5 Peter Amstutz
* Never appears in websocket updates
42 5 Peter Amstutz
* Never appears in API server request logs
43 5 Peter Amstutz
44 5 Peter Amstutz
It is an error for the same key (environment var) to appear in both environment and secret_environment. It should not be possible to _commit_ a container request with this error condition, although it might be possible to _save._
45 5 Peter Amstutz
46 5 Peter Amstutz
For clarity, some ways in which secret_environment behaves like environment are:
47 5 Peter Amstutz
* Non-identical secret_environment disqualifies a container for reuse. The mere existence of secret_environment does not disqualify.
48 5 Peter Amstutz
* secret_environment can be set via container_requests#create and container_requests#update APIs
49 5 Peter Amstutz
* secret_environment cannot be null, but can be an empty hash
50 5 Peter Amstutz
* keys of secret_environment are names of environment variables
51 5 Peter Amstutz
52 5 Peter Amstutz
h3. Secret command
53 5 Peter Amstutz
54 5 Peter Amstutz
55 5 Peter Amstutz
Add a "secret_environment" serialized field to containers and container requests.
56 5 Peter Amstutz
57 5 Peter Amstutz
"secret_command" has the same form and behavior as "command", except:
58 5 Peter Amstutz
59 5 Peter Amstutz
* Current value is never returned in a container request or container API response
60 5 Peter Amstutz
* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_command@) which must be authenticated by the container's own runtime token
61 5 Peter Amstutz
* Never appears in container logs
62 5 Peter Amstutz
* Never appears in the Arvados logs table
63 5 Peter Amstutz
* Never appears in websocket updates
64 5 Peter Amstutz
* Never appears in API server request logs
65 5 Peter Amstutz
66 5 Peter Amstutz
To run, "secret_command" is merged with "command".  If secret_command is empty, it is ignored, otherwise they must be arrays of the same size.  Entries in "command" which are "null" take their value from the corresponding entry in "secret_command".  It is an error for the same entry in command and secret_command to to both have a value or both be null.
67 5 Peter Amstutz
68 5 Peter Amstutz
For clarity, some ways in which secret_command behaves like command are:
69 5 Peter Amstutz
* Non-identical secret_command disqualifies a container for reuse. The mere existence of secret_command does not disqualify.
70 5 Peter Amstutz
* secret_command can be set via container_requests#create and container_requests#update APIs
71 5 Peter Amstutz
* secret_command cannot be null, but can be an empty list