Feature #12320
closedAccess Arvados projects under the "by_id" directory in keep mount
Description
One can access collections either by their portable data hash or UUID from the "by_id" directory in the keep mount. Trying to do the same for an Arvados project will fail with a "no such file or directory".
It would be nice to be able to access Arvados projects by their UUID under the keep "by_id" directory.
Implementation details:
- The code is in arvados/services/fuse
- The mount point configuration is initialized in arvados_fuse/command.py
- The "by_id" directory is implemented by the MagicDirectory class in arvados_fuse/fusedir.py
- The MagicDirectory is special, subdirectories only appear in the listing after they are first accessed
- The MagicDirectory.__contains__ method is called to determine if a subdirectory exists (named by portable data hash or uuid) and creates on demand it if necessary
- Currently it creates CollectionDirectory object for collections.
- The goal of this task is to add support to MagicDirectory for accessing projects (ProjectDirectory) by uuid.
Tests are in arvados/services/fuse/tests
The simplest way to run them is using arvbox on Linux (note, won't work on Windows/MacOS so in that case you will need to use a VM).
$ arvbox start test --only services/fuse
Alternately you can set up your workstation or a VM following these instructions: https://dev.arvados.org/projects/arvados/wiki/Hacking_prerequisites (but personally I use arvbox for all my development).
The test framework starts up the API server in testing mode with a known database, so some tests rely on the API server returning known values. These test values are defined in arvados/services/api/test/fixtures. You can probably use an existing record, such as "aproject" (uuid zzzzz-j7d0g-v955i6s2oi1cbso) defined groups.yml.
As a starting point, try looking at FuseMagicTest in arvados/services/fuse/tests/test_mount.py
To run a single test:
$ arvbox start test --only services/fuse services/fuse_test=--test-suite=tests.test_mount.FuseMagicTest
Updated by Tom Morris over 7 years ago
- Target version set to Arvados Future Sprints
Updated by Tom Morris almost 7 years ago
- Assigned To set to Fuad Muhic
- Target version changed from Arvados Future Sprints to 2018-03-28 Sprint
Updated by Tom Morris almost 7 years ago
- Status changed from New to In Progress
- Target version changed from 2018-03-28 Sprint to 2018-04-11 Sprint
Updated by Lucas Di Pentima almost 7 years ago
Reviewing branch 12320-access-projects-from-by_id
at 1f15c5b6060b0d82c9b17fb724b7128374f65747
Just one question:
Should all types of groups be accessible on fuse mount? Maybe we’re only looking for the ones with [‘group_class’, ‘=‘, ‘project’]
(just in case, double check with Tom or Peter)
Updated by Fuad Muhic almost 7 years ago
Lucas Di Pentima wrote:
Reviewing branch
12320-access-projects-from-by_id
at 1f15c5b6060b0d82c9b17fb724b7128374f65747Just one question:
Should all types of groups be accessible on fuse mount? Maybe we’re only looking for the ones with
[‘group_class’, ‘=‘, ‘project’]
(just in case, double check with Tom or Peter)
You are right, I fixed it.
Updated by Lucas Di Pentima almost 7 years ago
Reviewing fcdea9d131e4ba823ac8b69224d90b8eb5f4ae2e
Do you think wouldn't be better to add a filter to the api call so that it isn't retrieved in the first place?
Updated by Fuad Muhic almost 7 years ago
Lucas Di Pentima wrote:
Reviewing fcdea9d131e4ba823ac8b69224d90b8eb5f4ae2e
Do you think wouldn't be better to add a filter to the api call so that it isn't retrieved in the first place?
Do you mean to do a api group list with filter [‘group_class’, ‘=‘, ‘project’] and [‘uuid’, ‘=‘, k]?
Updated by Tom Morris almost 7 years ago
- Target version changed from 2018-04-11 Sprint to 2018-04-25 Sprint
Updated by Lucas Di Pentima almost 7 years ago
Fuad Muhic wrote:
Do you think wouldn't be better to add a filter to the api call so that it isn't retrieved in the first place?
Do you mean to do a api group list with filter [‘group_class’, ‘=‘, ‘project’] and [‘uuid’, ‘=‘, k]?
Yes, I believe it would be better in terms of bandwidth usage, for example if some user tries to access lots of group uuids.
Updated by Lucas Di Pentima almost 7 years ago
Updated by Fuad Muhic almost 7 years ago
- Status changed from In Progress to Resolved