Arvados

Context: experimenting with a 3rd party workflow engine, which supports S3 for storing files in between workflow steps.

One of the things it wants to do is create an S3 bucket for staging files.

It would be neat if we supported creating S3 buckets. I think the main question is how to manage the namespace.

Proposal:

• Create an Arvados project owned by the system called "S3 Buckets"
• A "create bucket" API call creates a subproject under "S3 Buckets" with the appropriate name, and adds a can_manage link for the requesting user
• Subsequent S3 API requests that don't match the PDH or UUID pattern are looked up by name under "S3 Buckets"