Feature #8570

[Crunch2] Impure access to object store

Added by Peter Amstutz over 5 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

As a transitional feature for deploying Crunch v2 into existing cloud environments, support staging files by downloading/uploading from/to object store such as S3 (alternately, use something like https://github.com/s3fs-fuse/s3fs-fuse). Will make container impure (not reusable/reproducible) since contents of files are not guaranteed to remain the same.

Related issues

Related to Arvados - Bug #10216: [Crunch2] [API] Allow container request to bypass reuse (ensure a new container is created)Resolved10/10/2016

Related to Arvados Epics - Story #15960: Computing on external dataNew01/01/202206/30/2022

History

#1 Updated by Peter Amstutz over 5 years ago

  • Description updated (diff)

#2 Updated by Peter Amstutz over 5 years ago

  • Subject changed from [Crunch2] Impure staging files from object store to [Crunch2] Impure access to object store

#3 Updated by Peter Amstutz over 5 years ago

  • Description updated (diff)

#4 Updated by Peter Amstutz over 5 years ago

  • Description updated (diff)

#5 Updated by Tom Morris over 3 years ago

  • Release deleted (11)

#6 Updated by Peter Amstutz almost 2 years ago

Also available in: Atom PDF