Bug #3261
closed[Keep] arv-put fails if permission system is enabled
Added by Tom Clegg about 10 years ago. Updated almost 10 years ago.
Description
In production, when Keep and the API server are configured to enforce permissions: after storing a collection in Keep, arv-put attempts to write the new collection to the API server but it fails a permission signature validation check.
Related issues
Updated by Tim Pierce almost 10 years ago
Reviewing tomclegg's 3261-keep-docker-config branch at 7305149:
- docker/api/omniauth.rb.in
- I don't think that
SSO_HOSTNAME.ARVADOS_DOMAIN
is going to resolve to the local SSO container, which is whatENV['SSO_PORT_443_TCP_ADDR']
did.
- I don't think that
- docker/build_tools/Makefile
- This line:
KEEP_GENERATED := $(shell ls -1 $(KEEP_GENERATED_IN) | perl -pe 's:^keep:keep/generated:')
- ... I think also needs
s/\.in$//
.
- This line:
- docker/keep/run-keep.in
- -z is the wrong test: this looks like it should be
if [ -s "$pkf" ] then permisson_key_arg="-permission-key-file=$pkf" else permisson_key_arg="" fi
- -z is the wrong test: this looks like it should be
Updated by Tom Clegg almost 10 years ago
Tim Pierce wrote:
Ah, I wondered where that came from. I've added that as a fallback, and changed the config.yml default to empty, so we have two main options:
- docker/api/omniauth.rb.in
- I don't think that
SSO_HOSTNAME.ARVADOS_DOMAIN
is going to resolve to the local SSO container, which is whatENV['SSO_PORT_443_TCP_ADDR']
did.
- Leave empty in config.yml = use sso docker container
- Specify https://auth.curoverse.com in config.yml = use some sso service elsewhere
- docker/build_tools/Makefile
- This line:
[...]- ... I think also needs
s/\.in$//
.
Ah yes. (Now obsoleted by your replacement.)
- docker/keep/run-keep.in
- -z is the wrong test: this looks like it should be
[...]
Yes. Fixed, thanks.
With these and other misc fixes, I've confirmed that #3261 is not so easily reproduced. :/
Updated by Tom Clegg almost 10 years ago
After turning on --enforce-permissions, both keep and api server seem to work as expected:
keep
root@compute0:/# curl -H 'Authorization: OAuth2 4mqswa8myo058hxxfyx0r9zt9mm5m91brpviuvaoun4d857ewd' http://keep_server_0.keep.dev.arvados:25107/acbd18db4cc2f85cedef654fccc4a4d8+3+A8dd47f55a2815eb149887938d3416e5fdb6f9813@53ed4abf foo root@compute0:/# curl -H 'Authorization: OAuth2 4mqswa8myo058hxxfyx0r9zt9mm5m91brpviuvaoun4d857ewd' http://keep_server_0.keep.dev.arvados:25107/acbd18db4cc2f85cedef654fccc4a4d8+3+A8dd47f55a2815eb149887938d3416e5fdb6fffff@53ed4abf Forbidden
api
root@compute0:/tmp# arv collection create --collection '{"uuid":"1f4b0bc7583c2a7f9102c395f4ffc5e3+45","manifest_text":". acbd18db4cc2f85cedef654fccc4a4d8+3+Afa79eed615b0cadac0a86dcd99c155c49d654c43@53ed418c 0:3:foo\n"}' # => success root@compute0:/tmp# arv collection create --collection '{"uuid":"1f4b0bc7583c2a7f9102c395f4ffc5e3+45","manifest_text":". acbd18db4cc2f85cedef654fccc4a4d8+3+A1dcc4d36024c3c4166088ba32686ffb428c33fc2@53ed4163 0:3:foo\n"}' Error: #<ArvadosModel::PermissionDeniedError: ArvadosModel::PermissionDeniedError>
Updated by Tom Clegg almost 10 years ago
Turning on permissions on 4xphq also seems fine:
4xphq:~# echo -n bar > bar 4xphq:~# arv-put bar arv-put: No --project-uuid or --name specified. This data will be cached in Keep. You will need to find this upload by its locator(s) later. 0M / 0M 100.0% fa7aeb5140e2848d39b416daeef4ffc5+45 4xphq:~# arv-get fa7aeb5140e2848d39b416daeef4ffc5+45 . 37b51d194a7513e45b56f6524f2d51f2+3+Af291968a5e126295afeef53e71e4537a2e6d7fa3@53ed4fd8 0:3:bar 4xphq:~# arv collection create --collection '{"uuid":"fa7aeb5140e2848d39b416daeef4ffc5+45","manifest_text":". 37b51d194a7513e45b56f6524f2d51f2+3+Af291968a5e126295afeef53e71e4537a2e6d7fa3@53ed4fd8 0:3:bar\n"}' { "href":"/collections/fa7aeb5140e2848d39b416daeef4ffc5+45", "kind":"arvados#collection", ... 4xphq:~# arv collection create --collection '{"uuid":"fa7aeb5140e2848d39b416daeef4ffc5+45","manifest_text":". 37b51d194a7513e45b56f6524f2d51f2+3+Af291968a5e126295afeef53e71e4537a2e6dffff@53ed4fd8 0:3:bar\n"}' Error: #<ArvadosModel::PermissionDeniedError: ArvadosModel::PermissionDeniedError>
Updated by Tom Clegg almost 10 years ago
Meanwhile, more docker patches in 3261-keep-docker-config @ 5ce0e0f
Updated by Tom Clegg almost 10 years ago
- Category set to Keep
- Status changed from New to In Progress
- Assigned To changed from Tim Pierce to Ward Vandewege
Updated by Ward Vandewege almost 10 years ago
- Target version changed from 2014-08-06 Sprint to 2014-08-27 Sprint
Updated by Tom Clegg almost 10 years ago
- Bug: CollectionWriter.finish() (as used by well-behaved crunch scripts, but not crunch-job) threw away its permission hints before showing manifest_text to anyone, so of course arvados.v1.collections.create failed.
- Fix: Don't do that. 3bd1f8da
- Unfortunately this means docker images with affected versions of the Python SDK can't be used when the permission system is turned up.
Updated by Tom Clegg almost 10 years ago
- Target version changed from 2014-08-27 Sprint to 2014-09-17 sprint
Updated by Tom Clegg almost 10 years ago
- Target version changed from 2014-09-17 sprint to Arvados Future Sprints
Updated by Tom Clegg almost 10 years ago
- Target version changed from Arvados Future Sprints to 2014-10-08 sprint
Updated by Ward Vandewege almost 10 years ago
- Status changed from In Progress to Resolved
Updated by Ward Vandewege almost 10 years ago
- Status changed from Resolved to In Progress
Updated by Ward Vandewege almost 10 years ago
- Status changed from In Progress to Resolved