Bug #5744
closed
PermissionDenied error writing job output of pipeline_instance qr1hi-d1hrv-4zczpddlf3m9uh4
Description
This pipeline instance https://cloud.curoverse.com/pipeline_instances/qr1hi-d1hrv-4zczpddlf3m9uh4 failed in the end while writing the output.
According to the logs, the error is a Permission Denied error.
Looking at the API logs, I see the following:
Missing signature on locator e2c9d041e8f3fa360d3efe0af631ca16+67108864
I attached the complete log for this CollectionController#create request found the API server log.
Files
Updated by Radhika Chippada over 9 years ago
- File PermissionDenied error log.rtf PermissionDenied error log.rtf added
- Subject changed from PermissionDenied error writing job output to PermissionDenied error writing job output of pipeline_instance qr1hi-d1hrv-4zczpddlf3m9uh4
- Description updated (diff)
Updated by Bryan Cosca over 9 years ago
Some notes, this has run succesfully before here: qr1hi-d1hrv-dl3bjgemakkt5aw
Here are some failed instances:
qr1hi-d1hrv-90r2fwvith54p9u
qr1hi-d1hrv-4zczpddlf3m9uh4
qr1hi-d1hrv-y5xjsqr4b32j50h
These users are given read access to the project. They probably tried to run it in their home project but I dont know how to check that.
Updated by Brett Smith over 9 years ago
- Status changed from New to Closed
- Target version deleted (
Bug Triage)
This bug is effectively a duplicate of #3261. The Arvados SDK in the Docker image does not retain Keep permission tokens in manifests, and will be rejected by modern API server deployments which require this. Very specifically, they're missing 3bd1f8dad. See #3261-10 note 10, which specifically mentions, "Unfortunately this means docker images with affected versions of the Python SDK can't be used when the permission system is turned up."
qr1hi-d1hrv-dl3bjgemakkt5aw worked because it reused a job from September 2014, before we began enforcing this rule on manifests. The failing pipelines created new jobs and got bit by the bug.
The fix is to either upgrade the Docker image with a newer Arvados SDK version, or update the pipeline to specify a version that at least includes 3bd1f8dad.