PermissionDenied error writing job output of pipeline_instance qr1hi-d1hrv-4zczpddlf3m9uh4
This pipeline instance https://cloud.curoverse.com/pipeline_instances/qr1hi-d1hrv-4zczpddlf3m9uh4 failed in the end while writing the output.
According to the logs, the error is a Permission Denied error.
Looking at the API logs, I see the following:
Missing signature on locator e2c9d041e8f3fa360d3efe0af631ca16+67108864
I attached the complete log for this CollectionController#create request found the API server log.
#1 Updated by Radhika Chippada almost 5 years ago
#2 Updated by Bryan Cosca almost 5 years ago
Some notes, this has run succesfully before here: qr1hi-d1hrv-dl3bjgemakkt5aw
Here are some failed instances:
These users are given read access to the project. They probably tried to run it in their home project but I dont know how to check that.
#3 Updated by Brett Smith almost 5 years ago
- Status changed from New to Closed
- Target version deleted (
This bug is effectively a duplicate of #3261. The Arvados SDK in the Docker image does not retain Keep permission tokens in manifests, and will be rejected by modern API server deployments which require this. Very specifically, they're missing 3bd1f8dad. See #3261-10 note 10, which specifically mentions, "Unfortunately this means docker images with affected versions of the Python SDK can't be used when the permission system is turned up."
qr1hi-d1hrv-dl3bjgemakkt5aw worked because it reused a job from September 2014, before we began enforcing this rule on manifests. The failing pipelines created new jobs and got bit by the bug.
The fix is to either upgrade the Docker image with a newer Arvados SDK version, or update the pipeline to specify a version that at least includes 3bd1f8dad.