Arvados

h1. Vault

Second pass at proposal on how to use Vault from an Arvados container, based on Feb 13 meeting:

* Credentials are entered into Vault

* Arvados user is granted permission to read/write certain credentials.  Need to determine how this works.

** Who can grant it?

** How is it represented?

** Some ideas: associated with a specific token (like "scopes"); create a new "secrets" table and use permission links

** Needs to be carefully engineered so that users can't manipulate policies to claim secrets they shouldn't have access to.

* Container, when run, logs into Vault using its container run token

* Vault uses Arvados plugin to associate container token with an Arvados user and determine policies

* Container queries vault for the credentials it needs

* Vault allows/denies query based on polices previously returned by Arvados plugin

Notes:

* Credentials may be passed in via environment, config file or command line like any other input, so it would be nice to generalize this as a CWL feature where some inputs are simply marked "sensitive" and automatically handled by Vault.  However, while arvados-cwl-runner is responsible for constructing the container request it doesn't run on the compute node, only crunch-run, which is "dumb", so its not clear how we go about allowing for arbitrary substitution into the environment, command line, or config file on the compute node.

* Response wrapping is another mechanism provided by Vault for delegating access.  A wrapped response can be passed through and accessed exactly once (would imply max_container_count: 1).

* Another solution is to develop an AuthMethod for vault with Arvados Token proivded by the container. 

h2. Vault plugins

We can implement Arvados authentication as a plugin.  A plugin is a service which is executed by Vault and communicates with Vault over RPC.  Vault provides infrastructure to do this easily in Go.

https://www.hashicorp.com/blog/building-a-vault-secure-plugin

Authentication returns a list of Vault policies to apply to the resulting login.  This means policies need to be stored in Arvados (see above discussion).

If the plugin can return arbitrary policies, this suggests that Arvados theoretically has unlimited ability to grant access to the Vault database.  We should consider how to restrict it to just those secrets / policies intended to be managed by Arvados.