Project

General

Profile

Actions

Idea #13146

closed

[API] Endpoint to get projects shared with me

Added by Peter Amstutz about 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
3.0
Release:
Release relationship:
Auto

Description

To get "projects shared with me" efficiently requires a new API endpoint. The way arv-mount currently determines what to list in "shared" currently requires looking at all projects and finding the ones where owner_uuid is not another project which is visible to us (meaning: users, non-project groups, or shared subprojects where the parent is not visible). This is expensive to compute on
the client, but can probably be accomplished with a single query on the API server.

Proposed endpoint: /arvados/v1/groups/shared

Return in "items" a list of toplevel projects and collections which are shared with me. Toplevel projects are ones which are either owned by a user, owned by a non-project group, or owned by a group which the user cannot read.

Support "includes" to return the set of owner objects as well (users, groups).

Possible query (abbreviated):

  • select groups that are readable by current user AND
    • the owner_uuid is a user (not a group) OR
    • the owner_uuid is a group is not a project or is not readable by the current user
select uuid from groups where 
  exists(select 1 from materialized_permission_view where user_uuid='2tlax-tpzed-51vcbmac4uv6bpb' and target_uuid=groups.uuid) and 
  (groups.owner_uuid in (select uuid from users) or 
    not exists(select 1 from materialized_permission_view where user_uuid='2tlax-tpzed-51vcbmac4uv6bpb' and target_uuid=groups.owner_uuid)) and 
  owner_uuid != '2tlax-tpzed-51vcbmac4uv6bpb';

Subtasks 2 (0 open2 closed)

Task #13950: Review 13146-shared-railsResolvedPeter Amstutz08/16/2018Actions
Task #14060: shared endpointResolvedPeter Amstutz08/15/2018Actions

Related issues

Related to Arvados - Bug #12990: [FUSE] Access shared/ is inefficientResolvedPeter AmstutzActions
Related to Arvados - Idea #13111: [WebDAV] Support browsing of project hierarchiesResolvedTom Clegg03/27/2018Actions
Related to Arvados - Idea #13218: Support browsing of projects shared with me in WebDAVNewActions
Related to Arvados - Feature #14201: [API] Accept "exclude_home_project" flag in groups#contentsResolvedPeter Amstutz10/01/2018Actions
Related to Arvados Workbench 2 - Support #15238: [Workbench] Change the content inside the owner column from uuid to Projects and user namesResolvedActions
Actions #1

Updated by Peter Amstutz about 6 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz about 6 years ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz about 6 years ago

  • Related to Bug #12990: [FUSE] Access shared/ is inefficient added
Actions #4

Updated by Peter Amstutz about 6 years ago

  • Related to Idea #13111: [WebDAV] Support browsing of project hierarchies added
Actions #5

Updated by Tom Morris about 6 years ago

  • Tracker changed from Feature to Idea
  • Status changed from In Progress to New
Actions #6

Updated by Tom Morris about 6 years ago

  • Related to Idea #13218: Support browsing of projects shared with me in WebDAV added
Actions #7

Updated by Peter Amstutz almost 6 years ago

  • Description updated (diff)
Actions #8

Updated by Peter Amstutz almost 6 years ago

  • Description updated (diff)

The current algorithm used in the FUSE driver is:

  • get all projects
  • go through the list of projects and find projects which have an owner_uuid that isn't in the list and isn't the current user. This has the effect of finding the projects for which the owner is either a user, a group which is not a project, or a group which is not readable by the current user.
  • The list of "roots" consists of each owner (if the owner is readable but not a project) or the project itself (if not).
Actions #9

Updated by Peter Amstutz almost 6 years ago

  • Description updated (diff)
  • Target version changed from To Be Groomed to Arvados Future Sprints
  • Story points set to 3.0
Actions #10

Updated by Tom Morris over 5 years ago

  • Target version changed from Arvados Future Sprints to 2018-08-15 Sprint
Actions #11

Updated by Peter Amstutz over 5 years ago

  • Assigned To set to Peter Amstutz
  • Target version changed from 2018-08-15 Sprint to Arvados Future Sprints
Actions #12

Updated by Peter Amstutz over 5 years ago

  • Target version changed from Arvados Future Sprints to 2018-08-15 Sprint
Actions #13

Updated by Peter Amstutz over 5 years ago

  • Status changed from New to In Progress
Actions #14

Updated by Peter Amstutz over 5 years ago

  • Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint
Actions #15

Updated by Peter Amstutz over 5 years ago

13146-shared-rails @ 0ba8a53ba145475153b01dc498f85dbd2f03228b

https://ci.curoverse.com/view/Developer/job/developer-run-tests/850/

I spent a while mulling over alternate APIs for "shared" but kept coming back the original definition. It is a bit ugly but it should still be much more efficient than listing every project and user and working it out on the client side.

Actions #16

Updated by Tom Clegg over 5 years ago

Suggestions

Have you tried these queries on a production-size database to make sure performance is OK with the existing indices?

Actions #18

Updated by Peter Amstutz over 5 years ago

Tom Clegg wrote:

Suggestions

Done.

Have you tried these queries on a production-size database to make sure performance is OK with the existing indices?

See previous note.

Now 13146-shared-rails @ a41c0f6aa41b658c8f2947c46cb90778894f5cf3

https://ci.curoverse.com/view/Developer/job/developer-run-tests/855/

Actions #19

Updated by Tom Clegg over 5 years ago

Experiments & include/included changes LGTM.

In the docs...
  • The second paragraph should have some reference to the "include" flag, perhaps just the word "optionally" -- as written it implies "always".
  • It feels like we're saying the same thing 4 different ways, including some pseudocode that has "a and b or c or d", which makes me think about operator precedence, and an "intended use" paragraph that seems to amount to "this API exists to support clients which wish to do what this API does". Could we pare this back to one or two ways of explaining it?
Actions #20

Updated by Peter Amstutz over 5 years ago

Tom Clegg wrote:

Experiments & include/included changes LGTM.

In the docs...
  • The second paragraph should have some reference to the "include" flag, perhaps just the word "optionally" -- as written it implies "always".
  • It feels like we're saying the same thing 4 different ways, including some pseudocode that has "a and b or c or d", which makes me think about operator precedence, and an "intended use" paragraph that seems to amount to "this API exists to support clients which wish to do what this API does". Could we pare this back to one or two ways of explaining it?

Sure, documentation updated @ 489aed58cb0d8bd816e07128cfcb9f5a06224083

Actions #21

Updated by Tom Clegg over 5 years ago

LGTM, thanks.

(Nit: seems like "this is useful for clients which wish to browse the list of" could be replaced with "i.e.," since it's implicit that an API is useful for clients that wish to get the information it returns... but this is gold-plated enough and I'm ready to move on)

Actions #22

Updated by Tom Morris over 5 years ago

It looks like a branch got merged a few days ago, but this ticket is still marked as being "In Progress." Is there additional work to be done? We were looking at it during the Workbench2 review to see what it's status was.

Actions #23

Updated by Peter Amstutz over 5 years ago

  • Status changed from In Progress to Resolved

Tom Morris wrote:

It looks like a branch got merged a few days ago, but this ticket is still marked as being "In Progress." Is there additional work to be done? We were looking at it during the Workbench2 review to see what it's status was.

This is done. The workbench2 folks should be able to use it (provided they are accessing one of the dev clusters with the latest code).

Actions #24

Updated by Peter Amstutz over 5 years ago

And it's documented at the bottom of the page here:

http://doc.arvados.org/api/methods/groups.html

Actions #25

Updated by Tom Clegg over 5 years ago

  • Related to Feature #14201: [API] Accept "exclude_home_project" flag in groups#contents added
Actions #26

Updated by Ward Vandewege over 5 years ago

  • Release set to 13
Actions #27

Updated by Tom Morris almost 5 years ago

  • Related to Support #15238: [Workbench] Change the content inside the owner column from uuid to Projects and user names added
Actions

Also available in: Atom PDF