Bug #17785
closed
[controller/api] "Forbidden: this API client cannot manipulate other clients' access tokens." on federated login clusters (2.2.0 regression)
Added by Ward Vandewege over 3 years ago.
Updated almost 3 years ago.
Release relationship:
Auto
Description
This happens on tordo (2.3.0~dev20210608145247) (login federation with ce8i5) but not on 2xpu4 (2.2.0) (directly configured for login through google).
Bug observed in multiple places:
- go to workbench.tordo, log in as admin, view a user in the admin user list, and click the "Log in as ..." button. The result is a fiddlesticks with the error "Forbidden: this API client cannot manipulate other clients' access tokens.", e.g.:
{
":errors":[
"Forbidden: this API client cannot manipulate other clients' access tokens. (req-ckw5smn0dfhygvcgk5h6)"
],
":error_token":"1625590529+e5031a85"
}
- on shell.ce8i5, the `arvados-login-sync` script (which runs with a token belonging to an admin user) throws this output on every iteration:
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-ddhir3er6zg31hszw9o1)"]
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-u61v42jybvqur0ygz5x3)"]
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1dtnzyr2oo2sfp6e8pjz)"]
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-h05j0ififv2t8ksfekhd)"]
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1w89ttespwuf51azgjt1)"]
Error setting token for STRIPPED: ["Forbidden: this API client cannot manipulate other clients' access tokens. (req-1773gy0yhdoo11t74emp)"]
- Description updated (diff)
- Target version changed from 2021-06-09 sprint to 2021-06-23 sprint
- Description updated (diff)
- Related to Bug #17754: [wb] merge account problem added
- Description updated (diff)
- Target version changed from 2021-06-23 sprint to 2021-07-07 sprint
- Assigned To set to Nico César
- File 2021-06-24_10-57.png added
- File 17785_error.png added
- File 2021-06-24_10-53.png added
- File deleted (
2021-06-24_10-57.png)
- File deleted (
17785_error.png)
- File deleted (
2021-06-24_10-53.png)
- Description updated (diff)
- Subject changed from [workbench] log in as another user broken to [controller/api] "Forbidden: this API client cannot manipulate other clients' access tokens." on federated login clusters (2.2.0 regression)
- Description updated (diff)
- Related to Feature #17583: Remote controller forwards trusted client aware calls on a federated scenario added
- Target version changed from 2021-07-07 sprint to 2021-07-21 sprint
- Assigned To deleted (
Nico César)
- Assigned To set to Peter Amstutz
- Related to deleted (Bug #17754: [wb] merge account problem)
- Blocks Bug #17754: [wb] merge account problem added
- Target version changed from 2021-07-21 sprint to 2021-08-04 sprint
- Target version changed from 2021-08-04 sprint to 2021-08-18 sprint
- Target version changed from 2021-08-18 sprint to 2021-09-01 sprint
- Assigned To changed from Peter Amstutz to Lucas Di Pentima
- Status changed from New to In Progress
While trying to reproduce this error, I kept getting another error message when asking tordo
for its user's list:
$ arv user list
Error: error updating local user records: //railsapi.internal/arvados/v1/users/batch_update: 422 Unprocessable Entity: #<ActiveRecord::RecordInvalid: Validation failed: Username has already been taken> (req-1iksissnw490d1thn1q9)
After re-requesting the list with -b
(to bypass federation) and comparing users with ce8i5
, I realized an issue with a couple of users sharing the same ward2
username. The one cached on tordo
's side doesn't exist on ce8i5
(even though its UUID is from that cluster), so when tordo
's controller attempts to refresh the local cache, the username is already taken by a user record that won't get updated anymore.
- Target version changed from 2021-09-01 sprint to 2021-09-15 sprint
For the "Login as <user>
" case, the issue is that wb1 tries to create an ApiClientAuthorization resource owned by the target user, and that fails because the current api client record isn't trusted because:
$ arv user current
{
"created_at":"2020-01-21T14:39:17.181611000Z",
"first_name":"Lucas ",
"full_name":"Lucas Di Pentima",
"identity_url":"",
"is_active":true,
"is_admin":true,
"is_invited":true,
"kind":"arvados#user",
"last_name":"Di Pentima",
"uuid":"ce8i5-tpzed-o4njwilpp4ov286",
[...]
}
$ arv api_client_authorization current
{
"href":"/api_client_authorizations/ce8i5-gj3su-vobk909hrbmmsrl",
"uuid":"ce8i5-gj3su-vobk909hrbmmsrl",
"owner_uuid":"ce8i5-tpzed-o4njwilpp4ov286",
"api_token":"xxxxtokenxxxx",
"api_client_id":0,
[...]
}
For the arvados-login-sync
case, maybe what's needed is to set up the LOGINCLUSTER_ARVADOS_API_HOST
and LOGINCLUSTER_ARVADOS_API_TOKEN
envvars so that the api client authorization creation is done on the LoginCluster?
- Target version changed from 2021-09-15 sprint to 2021-09-29 sprint
- Target version changed from 2021-09-29 sprint to 2021-10-13 sprint
- Target version changed from 2021-10-13 sprint to 2021-10-27 sprint
- Target version changed from 2021-10-27 sprint to 2021-11-10 sprint
- Target version changed from 2021-11-10 sprint to 2021-11-24 sprint
- Target version changed from 2021-11-24 sprint to 2021-11-10 sprint
- Target version changed from 2021-11-10 sprint to 2021-11-24 sprint
Updates at a98916d - branch 17785-federated-token-regression
Test run: developer-run-tests: #2812
- Migrates the
api_client_authorization
handling to controller's "new code path", forwarding CREATE
and UPDATE
calls to remote clusters if needed.
This fixes the test commented on #17785-28
In sdk/go/arvados/api_client_authorization.go
- The
*string
fields should be string
since there is no semantic difference between "" and NULL
- I don't think we need the
Href
field. It looks like we are currently using two different approaches for this. Container and Group have Href fields, but User and Collection don't -- instead "href" is exempt from the response-comparison test for User and Collection in lib/controller/handler_test.go. Unless something actually needs this, I think we should drop it.
In test
- Target version changed from 2021-11-24 sprint to 2021-12-08 sprint
Updates at 47982d3
Test run: developer-run-tests: #2814
- Enhances
IntegrationSuite.TestFederatedApiClientAuthHandling
test by confirming that the created token does work.
- Fixes
arvados.ApiClientAuthorization
's member types. Updates tests & related code.
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados-private:commit:arvados|9c8a812148e6b989fd7ab6aac49168276f5d5b9f.
Also available in: Atom
PDF