Multi-cluster user database » History » Revision 8
Revision 7 (Tom Morris, 07/25/2019 03:09 PM) → Revision 8/18 (Tom Clegg, 07/25/2019 08:55 PM)
h1. Multi-cluster user database It is sometimes desirable to share a single user database across multiple Arvados clusters. For example: * Clusters aaaaa, bbbbb, ccccc, ddddd, eeeee are on different continents, but they use the same upstream authentication providers (ldap/google). * A down/unreachable cluster should not prevent any user from using _other_ clusters in the group -- even if the down/unreachable cluster is the one where the user's account was initially created. This requires some changes to login and token validation. (Currently, any given user account has a single "home cluster" that can issue or validate tokens for it.) h2. Logging in Each user should be able to log in to their account using any cluster, regardless of where/whether they have logged in previously. To achieve this (without depending real-time communication between clusters) we need all of the participating clusters need to agree on a mapping of upstream authentication results to Arvados user UUIDs. For example, if the upstream authentication result is @"ldap://ldap.example foo@bar.example"@ ("ldap://ldap.example assures us this user is foo@bar.example"): # Generate If a row already exists in the users table with <code>upstream == "ldap://ldap.example foo@bar.example"</code> then use that row # Otherwise, create a new row with user UUID "eeeee-tpzed-${sha1part(upstream)}" (where eeeee is a common prefix used by all participating clusters and sha1part() is the first 15 chars of base-36-encoded sha1()) # If it doesn't already exist, add To avoid changing existing user accounts' UUIDs to @eeeee-*@, we would do a row to the users table with this UUID # If another row one-time synchronization of user accounts (and their upstreams) across all participating clusters. For example, if aaaaa-tpzed-012340123401234 exists in the users table with the same upstream (or same identity_url) but a different UUID, [offer to] merge the old account's data/objects/permissions into the new account (it isn't possible to log into the old account any more, but on cluster aaaaa, we know it belongs would add that row to the same person bbbbb and ccccc as the new account). Notes * the "upstream" field is similar well. Next time a user logs in to identity_url as initially conceived. Since #4601, identity_url has been bbbbb with an opaque SSO-generated UUID, with no info about upstream -- so we will rely on it to detect "same upstream as old account that needs matching aaaaa-tpzed-012340123401234, bbbbb would issue a token itself, rather than deferring to be migrated" but we can't use it to generate the same user UUID as other clusters, hence the need for a new "upstream" field * "remote" aaaaa. Untrusted remote accounts (the kind that we already have in the users table with foreign UUIDs) have a null identity_url field, and will also have a null upstream field field. |uuid |upstream |identity_url |significance | |eeeee-tpzed-012340123401234 |ldap://ldap.example |aaaaa-tpzed-aaaaaaaaaaaaaaa |google:// foo@bar.example |login-tpzed-aaaaaaaaaaaaaaa |Newly created user account |Imported/migrated from remote cluster aaaaa | |aaaaa-tpzed-aaaaaaaaaaaaaaa |NULL |login-tpzed-aaaaaaaaaaaaaaa |Old |eeeee-tpzed-012340123401234 |ldap://ldap.example foo@baz.example |User didn't exist before the multi-cluster user account (can't log in to this any more - contents should be migrated to eeeee-*) db system arrived | |ooooo-tpzed-ooooooooooooooo |NULL |NULL |Remote user from cluster ooooo (not part of our multi-cluster group) | h2. Configuration Each cluster needs to know * the uuid prefix to use when creating a new account, e.g., "eeeee" * additional user uuid prefixes that remote clusters are trusted to validate <pre><code class="yaml"> Clusters: aaaaa: Login: AssignUUIDPrefix: eeeee RemoteClusters: bbbbb: Proxy: true Authenticate: aaaaa: {} # accept tokens issued by bbbbb for users with uuid aaaaa-* bbbbb: {} # (implied) eeeee: {} # accept tokens issued by bbbbb for users with uuid eeeee-* </code></pre> Example: aaaaa needs to validate a token issued by bbbbb. * Do a callback to bbbbb (or check JWT signature) to confirm bbbbb really issued this token and get the relevant user UUID (result: yes, user uuid is eeeee-tpzed-012340123401234) * If config Clusters.aaaaa.RemoteClusters.bbbbb.Authenticate.eeeee is present, accept the token * Otherwise, fetch eeeee's config; if RemoteClusters.bbbbb.Authenticate.eeeee is present, accept the token * Otherwise, reject the token h2. Validating tokens (...even when the issuing cluster is unreachable) Each cluster should be able to validate a token that was issued by a different, currently unreachable, cluster. This contrasts with the current setup, where aaaaa validates tokens issued by bbbbb by doing a callback to bbbbb. This seems easy enough: instead of random strings, tokens can be [like] "JSON Web Tokens":https://jwt.io/, signed by a private key whose public part is known by all clusters. (This would also be more efficient than callbacks, benefiting the mutually-untrusted cluster scenario too.)