Project

General

Profile

Multi-cluster user database » History » Version 4

Tom Clegg, 07/25/2019 02:17 PM

1 1 Tom Clegg
h1. Multi-cluster user database
2
3
It is sometimes desirable to share a single user database across multiple Arvados clusters. For example:
4
* Clusters aaaaa, bbbbb, ccccc are on different continents.
5
* A down/unreachable cluster should not prevent a user from accessing _other_ clusters -- even if the down/unreachable cluster is normally the best/default one from that user's perspective.
6
7
This requires some changes to authentication (obtaining and validating API tokens).
8
9
h2. Obtaining tokens
10
11 2 Tom Clegg
Each user should be able to log in to their account using any cluster, regardless of where/whether they have logged in previously. This contrasts with the current setup, where each user account has a "home cluster" which must be used to log in.
12 1 Tom Clegg
13
To achieve this (without depending real-time communication between clusters) we need all of the participating clusters to agree on a mapping of upstream authentication results to Arvados user UUIDs. For example, if the upstream authentication result is @"ldap://ldap.example foo@bar.example"@ ("ldap://ldap.example assures us this user is foo@bar.example"):
14
# If a row already exists in the users table with <code>upstream == "ldap://ldap.example foo@bar.example"</code> then use that row
15 4 Tom Clegg
# Otherwise, create a new row with user UUID "eeeee-tpzed-${sha1part(upstream)}" (where eeeee is a common prefix used by all participating clusters and sha1part() is the first 15 chars of base-36-encoded sha1())
16 1 Tom Clegg
17 4 Tom Clegg
To avoid changing existing user accounts' UUIDs to @eeeee-*@, we would do a one-time synchronization of user accounts (and their upstreams) across all participating clusters. For example, if aaaaa-tpzed-012340123401234 exists on cluster aaaaa, we would add that row to bbbbb and ccccc as well. Next time a user logs in to bbbbb with an upstream account matching aaaaa-tpzed-012340123401234, bbbbb would issue a token itself, rather than deferring to aaaaa.
18 1 Tom Clegg
19 2 Tom Clegg
Untrusted remote accounts (the kind that we already have in the users table with foreign UUIDs) have a null upstream field.
20
21
|uuid                        |upstream                            |significance                                |
22
|aaaaa-tpzed-aaaaaaaaaaaaaaa |google:// foo@bar.example           |Imported/migrated from remote cluster aaaaa |
23 4 Tom Clegg
|eeeee-tpzed-012340123401234 |ldap://ldap.example foo@baz.example |User didn't exist before the multi-cluster user db system arrived |
24 2 Tom Clegg
|ooooo-tpzed-ooooooooooooooo |NULL                                |Remote user from cluster ooooo (not part of our multi-cluster group) |
25
26 1 Tom Clegg
h2. Validating tokens
27
28 2 Tom Clegg
Each cluster should be able to validate a token that was issued by a different, currently unreachable, cluster. This contrasts with the current setup, where aaaaa validates tokens issued by bbbbb by doing a callback to bbbbb.
29 1 Tom Clegg
30 4 Tom Clegg
This seems easy enough: instead of random strings, tokens can be [like] JWT, signed by a private key whose public part is known by all clusters. (This would also be more efficient than callbacks, benefiting the mutually-untrusted cluster scenario too.)
31 2 Tom Clegg
32
h2. Configuration
33
34 3 Tom Clegg
Each cluster needs to know
35 4 Tom Clegg
* the uuid prefix to use when creating a new account, e.g., "eeeee"
36 3 Tom Clegg
* additional user uuid prefixes that remote clusters are trusted to validate
37 2 Tom Clegg
38
<pre><code class="yaml">
39
Clusters:
40
  aaaaa:
41 1 Tom Clegg
    RemoteClusters:
42
      bbbbb:
43
        Proxy: true
44
        Authenticate:
45
          aaaaa: {} # accept tokens issued by bbbbb for users with uuid aaaaa-*
46
          bbbbb: {} # (implied)
47 4 Tom Clegg
          eeeee: {} # accept tokens issued by bbbbb for users with uuid eeeee-*
48 1 Tom Clegg
</code></pre>
49 4 Tom Clegg
50
Example: aaaaa needs to validate a token issued by bbbbb.
51
* Do a callback to bbbbb (or check JWT signature) to confirm bbbbb really issued this token (result: yes, user uuid is eeeee-tpzed-012340123401234)
52
* If config Clusters.aaaaa.RemoteClusters.bbbbb.Authenticate.eeeee is present, accept the token
53
* Otherwise, fetch eeeee's config; if RemoteClusters.bbbbb.Authenticate.eeeee is present, accept the token
54
* Otherwise, reject the token