Project

General

Profile

Actions

Feature #10191

open

Arvados should be able to cat/zcat files in collections easily

Added by Bryan Cosca about 8 years ago. Updated 10 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

Use case:

I have 8 lanes of paired end fastq files, and I would like to combine all the lanes together into one big sample. I don't want to manually cat them, as that could take upwards of 12 hours.

Actions #1

Updated by Tom Morris about 8 years ago

While server-side concatenation could be valuable in certain circumstances, experiments for this case showed that the actual time was 13 min., not 12 hours, so we're going to defer looking at this for the time being.

Actions #2

Updated by Joshua Randall about 8 years ago

Support for server-side (manifest-based) concatenation seems like a potentially useful feature to have, although I'm wary of the `zcat` support unless it is also just an underlying manifest concatenation. If non-trivial operations such (re)compression need to be exposed via the FUSE interface, I'd suggest the use of more generic support for running pipelines via keep such as that proposed in story 10388.

Actions #3

Updated by Tom Morris about 8 years ago

  • Tracker changed from Bug to Feature
  • Target version set to Arvados Future Sprints
Actions #4

Updated by Ward Vandewege over 3 years ago

  • Target version deleted (Arvados Future Sprints)
Actions #5

Updated by Peter Amstutz almost 2 years ago

  • Release set to 60
Actions #6

Updated by Peter Amstutz 10 months ago

  • Target version set to Future
Actions

Also available in: Atom PDF