Idea #15720
closed
[API] Unified user listing across all clusters in a federation
Description
When choosing to share a project with another user, the contents of the dialog box is populated from the user database of the local cluster. Non-local users only appear if they have previously accessed that cluster in some way (causing a user record to be created). This means when attempting to share a project with another user, that user may or may not appear in the list based on whether the user has ever accessed that cluster in the past. (It is also invalid to create a sharing involving a user uuid that doesn't appear in the users table). This is confusing.
For the case where LoginCluster is set, this can be solved by having arvados controller proxy user record listing requests to the LoginCluster. The API server also needs to allow permission links that point to users uuids that are not already cached in the users table -- possibly by automatically creating a stub record for those users.
For the general case, controller would need to query the user listings of all known remote clusters and merge them into a single response. (we could probably defer this)
Kind of related: what happens when a user merges accounts on the main cluster? All the other clusters should be synchronized and merge the old user to the new user. Should the API server merge account feature do this?
Agreed solution for the LoginCluster case:¶
When LoginCluster is set, Controller proxies requests to "get" or "list" user records to the LoginCluster.
Controller uses the response from the LoginCluster to create or update user records in the local database before returning the response to the client.
If the query contains 'select' only update the fields in the response (must include 'uuid').
Updated by Peter Amstutz about 5 years ago
- Status changed from New to In Progress
Updated by Peter Amstutz about 5 years ago
- Related to Feature #15531: [SDK] Migrate federation to central LoginCluster added
Updated by Peter Amstutz about 5 years ago
- Status changed from In Progress to New
Updated by Peter Amstutz about 5 years ago
- Subject changed from Unified user listing across all clusters in a federation to [API] Unified user listing across all clusters in a federation
- Description updated (diff)
Updated by Tom Morris
Updated by Peter Amstutz about 5 years ago
- Description updated (diff)
- Target version changed from To Be Groomed to Arvados Future Sprints
- Story points set to 3.0
Updated by Peter Amstutz about 5 years ago
- Target version changed from Arvados Future Sprints to 2019-11-20 Sprint
Updated by Tom Clegg about 5 years ago
15720-fed-user-list @ b047cc92acab06d4c1f1d80173486f951b63d729 -- https://ci.curoverse.com/view/Developer/job/developer-run-tests/1651/
I think this is functional, but has a flaw (which I figure is a blocker) that it involves N controller-to-RailsAPI update requests for each response page of N items. Ways to fix:- do a local query with filters=[uuid,in,[...]], select=[...] to compare, then update only the items that have changed (might need multiple local queries if the local max page size is smaller than the remote max page size)
- add a batch-update API to RailsAPI, perhaps requiring SystemRootToken in order to avoid permission-related nuances
Updated by Tom Clegg about 5 years ago
- add a batch-update API to RailsAPI, perhaps requiring SystemRootToken in order to avoid permission-related nuances
15720-fed-user-list @ 233a2b6bd23a3e2054cfc0690f2bc06c0f9f7323 -- https://ci.curoverse.com/view/Developer/job/developer-run-tests/1658/
Updated by Tom Clegg about 5 years ago
- Target version changed from 2019-11-20 Sprint to 2019-12-04 Sprint
Updated by Peter Amstutz about 5 years ago
EnableBetaController14287 needs to be on for this feature to work. Should this be the branch that enables it by default?
If you contact a cluster with a token issued by that cluster (eg the superuser token, rather than the login cluster), it won't be accepted by the login cluster and you get this:
Error: request failed: https://172.17.0.2:8000/arvados/v1/users?cluster_id=&count=exact&filters=%5B%5B%22email%22%2C%22%3D%22%2C%22a%40a%22%5D%5D&limit=100&offset=0: 401 Unauthorized: Not logged in (req-9fdv9jdhlz7b298vo3t3)
Next try, talking to cluster (B) using a token issued by cluster (A):
Error: error updating local user records: request failed: http://localhost:8004/arvados/v1/users/batch_update: 401 Unauthorized: Not logged in (req-oagajjudmev71bd4u28r)
On futher research, it looks like arvbox has fallen into one of our circular dependency setup traps. The SystemRootToken is set in the config file, you can't start the API server without a config file, so after starting the API server, I would need to create an api_client_authorization record for the system root token, rewrite the the config file and then restart controller. Uhg.
I think we discussed having API accept SystemRootToken as a special case, could we sneak this in? I really really want to stamp out all the circular dependencies between the config file and API. (Perhaps I'll just go ahead and make that branch over the weekend).
Updated by Tom Clegg about 5 years ago
#15795 "Accept configured SystemRootToken ..."
Updated by Tom Clegg about 5 years ago
- Related to Idea #15795: [API] Accept configured SystemRootToken without doing a database lookup added
Updated by Tom Clegg about 5 years ago
Updated by Peter Amstutz about 5 years ago
Would it be more RESTful to PATCH on /arvados/v1/users instead of /arvados/v1/users/batch_update? In which case the format should be a list of user objects (like the index query) instead of a uuid->object map. I guess making it a special-purpose endpoint avoids these questions.
Looks like batch_update is not intended to be a public API, there's no EndpointUserBatchUpdate APIEndpoint in the Go SDK. Looks like it doesn't appear in the discovery document, so that's fine.
It is a little weird that the code generation starts from ContainerList and rewrites it for other variants instead of starting from a fully generic template.
I was very confused that new user records were appearing on one of the remote clusters even though there was no obvious user interaction. It turns out that workbench1 does session login on each remote cluster even when not on the multi-site search page, even if multi-site search is not enabled. (I expected something like this with workbench2, but not workbench1).
However, even though it turns out that workbench causes users to be automatically cached at other cluster as a side effect, the user listing is still necessary to ensure the correct user list is displayed (based on group membership) and correct user info (like first name / last name).
With that in mind, I made changes to user's group membership and user's name on the login cluster and those changes were reflected in the sharing dialog of a federate cluster's workbench. Success!
This LGTM.
Updated by Tom Clegg about 5 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|c371555926e3f361497b2bb1c9531fcaf7283ed4.