"sync-groups" tool doesn't find groups when running on a LoginCluster federation
arvados-sync-groups on a federated cluster with
Login.LoginCluster pointing to other cluster, the tool assumes that the needed groups should be owned by the LoginCluster's system user instead of the local system user.
This affects external groups that were synchronized before setting up the federation, because it will try to create a group named "Externally synchronized groups" and it will collide with the preexisting one.
#1 Updated by Lucas Di Pentima 3 months ago
Fix at ab9833a2d - branch
Test run: https://ci.arvados.org/job/developer-run-tests/2134/
- Fixed the problem by retrieving the local cluster ID from the exported config instead of assuming it from the user's uuid.
#2 Updated by Lucas Di Pentima 3 months ago
Manual testing was done by setting up a LoginCluster federation with a couple of arvbox instances A (main cluster) and B (satellite cluster)
Previous to setting up the federation, created some users and ran the tool with some groups on the B instance.
Then, set up the federation and ran the migrate users script to migrate B users to A.
As a last step into confirming the bug, ran again the
arvados-sync-groups tool with the A's admin user credentials from cluster B. Got the following error:
(venv-arvados) lucas@buster:~/arvados/tools/sync-groups$ ./sync-groups -user-id=username ../../test-groups.csv 2020/10/08 15:11:14 error creating system user owned group named "Externally synchronized groups": request failed: https://172.17.0.3:8000//arvados/v1/groups: 422 Unprocessable Entity: #<ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_groups_on_owner_uuid_and_name" DETAIL: Key (owner_uuid, name)=(x1ejm-tpzed-000000000000000, Externally synchronized groups) already exists. : INSERT INTO "groups" ("uuid", "owner_uuid", "created_at", "modified_by_user_uuid", "modified_at", "name", "updated_at", "group_class") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id"> (req-28280i3qiygt1p4asl8e)
After the fix, the tool started to work again. Did some changes on the CSV files and re-applied; the changes were done correctly.
#9 Updated by Lucas Di Pentima 11 days ago
Updates at 59e8b47be - branch
Test run: https://ci.arvados.org/job/developer-run-tests/2254/
arvados-bootbased helper functions from
- Adds federation test on
arvados-sync-groupsthat confirms the tool works on a federated cluster with admin credentials from the LoginCluster.
#10 Updated by Lucas Di Pentima 11 days ago
Updates at c271137b5
Test run: https://ci.arvados.org/job/developer-run-tests/2255/
API Test re-run: https://ci.arvados.org/job/developer-run-tests-services-api/2362/
Fixes go import cycle problems.