Idea #13146
closed
[API] Endpoint to get projects shared with me
Description
To get "projects shared with me" efficiently requires a new API endpoint. The way arv-mount currently determines what to list in "shared" currently requires looking at all projects and finding the ones where owner_uuid is not another project which is visible to us (meaning: users, non-project groups, or shared subprojects where the parent is not visible). This is expensive to compute on
the client, but can probably be accomplished with a single query on the API server.
Proposed endpoint: /arvados/v1/groups/shared
Return in "items" a list of toplevel projects and collections which are shared with me. Toplevel projects are ones which are either owned by a user, owned by a non-project group, or owned by a group which the user cannot read.
Support "includes" to return the set of owner objects as well (users, groups).
Possible query (abbreviated):
- select groups that are readable by current user AND
- the owner_uuid is a user (not a group) OR
- the owner_uuid is a group is not a project or is not readable by the current user
select uuid from groups where exists(select 1 from materialized_permission_view where user_uuid='2tlax-tpzed-51vcbmac4uv6bpb' and target_uuid=groups.uuid) and (groups.owner_uuid in (select uuid from users) or not exists(select 1 from materialized_permission_view where user_uuid='2tlax-tpzed-51vcbmac4uv6bpb' and target_uuid=groups.owner_uuid)) and owner_uuid != '2tlax-tpzed-51vcbmac4uv6bpb';
Updated by Peter Amstutz almost 7 years ago
- Status changed from New to In Progress
Updated by Peter Amstutz almost 7 years ago
- Related to Bug #12990: [FUSE] Access shared/ is inefficient added
Updated by Peter Amstutz almost 7 years ago
- Related to Idea #13111: [WebDAV] Support browsing of project hierarchies added
Updated by Tom Morris almost 7 years ago
- Tracker changed from Feature to Idea
- Status changed from In Progress to New
Updated by Tom Morris almost 7 years ago
- Related to Idea #13218: Support browsing of projects shared with me in WebDAV added
Updated by Peter Amstutz over 6 years ago
- Description updated (diff)
The current algorithm used in the FUSE driver is:
- get all projects
- go through the list of projects and find projects which have an owner_uuid that isn't in the list and isn't the current user. This has the effect of finding the projects for which the owner is either a user, a group which is not a project, or a group which is not readable by the current user.
- The list of "roots" consists of each owner (if the owner is readable but not a project) or the project itself (if not).
Updated by Peter Amstutz over 6 years ago
- Description updated (diff)
- Target version changed from To Be Groomed to Arvados Future Sprints
- Story points set to 3.0
Updated by Tom Morris over 6 years ago
- Target version changed from Arvados Future Sprints to 2018-08-15 Sprint
Updated by Peter Amstutz over 6 years ago
- Assigned To set to Peter Amstutz
- Target version changed from 2018-08-15 Sprint to Arvados Future Sprints
Updated by Peter Amstutz over 6 years ago
- Target version changed from Arvados Future Sprints to 2018-08-15 Sprint
Updated by Peter Amstutz over 6 years ago
- Status changed from New to In Progress
Updated by Peter Amstutz over 6 years ago
- Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint
Updated by Peter Amstutz over 6 years ago
13146-shared-rails @ 0ba8a53ba145475153b01dc498f85dbd2f03228b
https://ci.curoverse.com/view/Developer/job/developer-run-tests/850/
I spent a while mulling over alternate APIs for "shared" but kept coming back the original definition. It is a bit ugly but it should still be much more efficient than listing every project and user and working it out on the client side.
Updated by Tom Clegg over 6 years ago
- rename the "include" response field to "included" like http://jsonapi.org/format/#document-compound-documents
- ...and rename "includes" to "included" in the comment in shared()
- skip the owners unless the client asks for them with
?include=ownerlike http://jsonapi.org/format/#fetching-includes - apply
@selectto the related objects as well
Have you tried these queries on a production-size database to make sure performance is OK with the existing indices?
Updated by Peter Amstutz over 6 years ago
Tom Clegg wrote:
Suggestions
- rename the "include" response field to "included" like http://jsonapi.org/format/#document-compound-documents
- ...and rename "includes" to "included" in the comment in shared()
- skip the owners unless the client asks for them with
?include=ownerlike http://jsonapi.org/format/#fetching-includes>- apply
@selectto the related objects as well
Done.
Have you tried these queries on a production-size database to make sure performance is OK with the existing indices?
See previous note.
Now 13146-shared-rails @ a41c0f6aa41b658c8f2947c46cb90778894f5cf3
https://ci.curoverse.com/view/Developer/job/developer-run-tests/855/
Updated by Tom Clegg over 6 years ago
Experiments & include/included changes LGTM.
In the docs...- The second paragraph should have some reference to the "include" flag, perhaps just the word "optionally" -- as written it implies "always".
- It feels like we're saying the same thing 4 different ways, including some pseudocode that has "a and b or c or d", which makes me think about operator precedence, and an "intended use" paragraph that seems to amount to "this API exists to support clients which wish to do what this API does". Could we pare this back to one or two ways of explaining it?
Updated by Peter Amstutz over 6 years ago
Tom Clegg wrote:
Experiments & include/included changes LGTM.
In the docs...
- The second paragraph should have some reference to the "include" flag, perhaps just the word "optionally" -- as written it implies "always".
- It feels like we're saying the same thing 4 different ways, including some pseudocode that has "a and b or c or d", which makes me think about operator precedence, and an "intended use" paragraph that seems to amount to "this API exists to support clients which wish to do what this API does". Could we pare this back to one or two ways of explaining it?
Sure, documentation updated @ 489aed58cb0d8bd816e07128cfcb9f5a06224083
Updated by Tom Clegg over 6 years ago
LGTM, thanks.
(Nit: seems like "this is useful for clients which wish to browse the list of" could be replaced with "i.e.," since it's implicit that an API is useful for clients that wish to get the information it returns... but this is gold-plated enough and I'm ready to move on)
Updated by Tom Morris over 6 years ago
It looks like a branch got merged a few days ago, but this ticket is still marked as being "In Progress." Is there additional work to be done? We were looking at it during the Workbench2 review to see what it's status was.
Updated by Peter Amstutz over 6 years ago
- Status changed from In Progress to Resolved
Tom Morris wrote:
It looks like a branch got merged a few days ago, but this ticket is still marked as being "In Progress." Is there additional work to be done? We were looking at it during the Workbench2 review to see what it's status was.
This is done. The workbench2 folks should be able to use it (provided they are accessing one of the dev clusters with the latest code).
Updated by Peter Amstutz over 6 years ago
And it's documented at the bottom of the page here:
Updated by Tom Clegg over 6 years ago
- Related to Feature #14201: [API] Accept "exclude_home_project" flag in groups#contents added
Updated by
Updated by Tom Morris over 5 years ago
- Related to Support #15238: [Workbench] Change the content inside the owner column from uuid to Projects and user names added