Bug #17785
closed[controller/api] "Forbidden: this API client cannot manipulate other clients' access tokens." on federated login clusters (2.2.0 regression)
Description
This happens on tordo (2.3.0~dev20210608145247) (login federation with ce8i5) but not on 2xpu4 (2.2.0) (directly configured for login through google).
Bug observed in multiple places:
- go to workbench.tordo, log in as admin, view a user in the admin user list, and click the "Log in as ..." button. The result is a fiddlesticks with the error "Forbidden: this API client cannot manipulate other clients' access tokens.", e.g.:
{ ":errors":[ "Forbidden: this API client cannot manipulate other clients' access tokens. (req-ckw5smn0dfhygvcgk5h6)" ], ":error_token":"1625590529+e5031a85" }
- on shell.ce8i5, the `arvados-login-sync` script (which runs with a token belonging to an admin user) throws this output on every iteration:
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-ddhir3er6zg31hszw9o1)"] Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-u61v42jybvqur0ygz5x3)"] Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1dtnzyr2oo2sfp6e8pjz)"] Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-h05j0ififv2t8ksfekhd)"] Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1w89ttespwuf51azgjt1)"] Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1773gy0yhdoo11t74emp)"]
Related issues
Updated by Ward Vandewege about 3 years ago
- Target version changed from 2021-06-09 sprint to 2021-06-23 sprint
- Description updated (diff)
Updated by Ward Vandewege about 3 years ago
- Related to Bug #17754: [wb] merge account problem added
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2021-06-23 sprint to 2021-07-07 sprint
Updated by Nico César about 3 years ago
- File 2021-06-24_10-57.png added
- File 17785_error.png added
- File 2021-06-24_10-53.png added
Updated by Ward Vandewege about 3 years ago
- Description updated (diff)
- Subject changed from [workbench] log in as another user broken to [controller/api] "Forbidden: this API client cannot manipulate other clients' access tokens." on federated login clusters (2.2.0 regression)
Updated by Ward Vandewege about 3 years ago
- Related to Feature #17583: Remote controller forwards trusted client aware calls on a federated scenario added
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2021-07-07 sprint to 2021-07-21 sprint
Updated by Peter Amstutz about 3 years ago
- Related to deleted (Bug #17754: [wb] merge account problem)
Updated by Peter Amstutz about 3 years ago
- Blocks Bug #17754: [wb] merge account problem added
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2021-07-21 sprint to 2021-08-04 sprint
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2021-08-04 sprint to 2021-08-18 sprint
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2021-08-18 sprint to 2021-09-01 sprint
Updated by Peter Amstutz about 3 years ago
- Assigned To changed from Peter Amstutz to Lucas Di Pentima
Updated by Lucas Di Pentima about 3 years ago
- Status changed from New to In Progress
Updated by Lucas Di Pentima almost 3 years ago
While trying to reproduce this error, I kept getting another error message when asking tordo
for its user's list:
$ arv user list Error: error updating local user records: //railsapi.internal/arvados/v1/users/batch_update: 422 Unprocessable Entity: #<ActiveRecord::RecordInvalid: Validation failed: Username has already been taken> (req-1iksissnw490d1thn1q9)
After re-requesting the list with -b
(to bypass federation) and comparing users with ce8i5
, I realized an issue with a couple of users sharing the same ward2
username. The one cached on tordo
's side doesn't exist on ce8i5
(even though its UUID is from that cluster), so when tordo
's controller attempts to refresh the local cache, the username is already taken by a user record that won't get updated anymore.
Updated by Lucas Di Pentima almost 3 years ago
- Target version changed from 2021-09-01 sprint to 2021-09-15 sprint
Updated by Lucas Di Pentima almost 3 years ago
For the "Login as <user>
" case, the issue is that wb1 tries to create an ApiClientAuthorization resource owned by the target user, and that fails because the current api client record isn't trusted because:
$ arv user current { "created_at":"2020-01-21T14:39:17.181611000Z", "first_name":"Lucas ", "full_name":"Lucas Di Pentima", "identity_url":"", "is_active":true, "is_admin":true, "is_invited":true, "kind":"arvados#user", "last_name":"Di Pentima", "uuid":"ce8i5-tpzed-o4njwilpp4ov286", [...] } $ arv api_client_authorization current { "href":"/api_client_authorizations/ce8i5-gj3su-vobk909hrbmmsrl", "uuid":"ce8i5-gj3su-vobk909hrbmmsrl", "owner_uuid":"ce8i5-tpzed-o4njwilpp4ov286", "api_token":"xxxxtokenxxxx", "api_client_id":0, [...] }
Updated by Lucas Di Pentima almost 3 years ago
Updates at deca285 - branch 17785-federated-token-regression
Test run: developer-run-tests-remainder: #2789
- Adds an integration test that exposes the issue.
Updated by Lucas Di Pentima almost 3 years ago
For the arvados-login-sync
case, maybe what's needed is to set up the LOGINCLUSTER_ARVADOS_API_HOST
and LOGINCLUSTER_ARVADOS_API_TOKEN
envvars so that the api client authorization creation is done on the LoginCluster?
Updated by Lucas Di Pentima almost 3 years ago
- Target version changed from 2021-09-15 sprint to 2021-09-29 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2021-09-29 sprint to 2021-10-13 sprint
Updated by Lucas Di Pentima almost 3 years ago
- Target version changed from 2021-10-13 sprint to 2021-10-27 sprint
Updated by Lucas Di Pentima almost 3 years ago
- Target version changed from 2021-10-27 sprint to 2021-11-10 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2021-11-10 sprint to 2021-11-24 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2021-11-24 sprint to 2021-11-10 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2021-11-10 sprint to 2021-11-24 sprint
Updated by Lucas Di Pentima almost 3 years ago
Updates at a98916d - branch 17785-federated-token-regression
Test run: developer-run-tests: #2812
- Migrates the
api_client_authorization
handling to controller's "new code path", forwardingCREATE
andUPDATE
calls to remote clusters if needed.
This fixes the test commented on #17785-28
Updated by Tom Clegg almost 3 years ago
- The
*string
fields should bestring
since there is no semantic difference between "" and NULL - I don't think we need the
Href
field. It looks like we are currently using two different approaches for this. Container and Group have Href fields, but User and Collection don't -- instead "href" is exempt from the response-comparison test for User and Collection in lib/controller/handler_test.go. Unless something actually needs this, I think we should drop it.
- Looks like this could be simplified
- for cls := range s.testClusters { - if cls == "z1111" || cls == "z3333" { + for _, cls := range []string{"z1111", "z3333"} {
- After creating the "log in as..." token, might as well check that it works... if resp is an APIClientAuthorization then resp.TokenV2() might be convenient.
Updated by Lucas Di Pentima over 2 years ago
- Target version changed from 2021-11-24 sprint to 2021-12-08 sprint
Updated by Lucas Di Pentima over 2 years ago
Updates at 47982d3
Test run: developer-run-tests: #2814
- Enhances
IntegrationSuite.TestFederatedApiClientAuthHandling
test by confirming that the created token does work. - Fixes
arvados.ApiClientAuthorization
's member types. Updates tests & related code.
Updated by Lucas Di Pentima over 2 years ago
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados-private:commit:arvados|9c8a812148e6b989fd7ab6aac49168276f5d5b9f.