Feature #10191
open
Arvados should be able to cat/zcat files in collections easily
Description
Use case:
I have 8 lanes of paired end fastq files, and I would like to combine all the lanes together into one big sample. I don't want to manually cat them, as that could take upwards of 12 hours.
Updated by Tom Morris over 7 years ago
While server-side concatenation could be valuable in certain circumstances, experiments for this case showed that the actual time was 13 min., not 12 hours, so we're going to defer looking at this for the time being.
Updated by Joshua Randall over 7 years ago
Support for server-side (manifest-based) concatenation seems like a potentially useful feature to have, although I'm wary of the `zcat` support unless it is also just an underlying manifest concatenation. If non-trivial operations such (re)compression need to be exposed via the FUSE interface, I'd suggest the use of more generic support for running pipelines via keep such as that proposed in story 10388.
Updated by Tom Morris over 7 years ago
- Tracker changed from Bug to Feature
- Target version set to Arvados Future Sprints
Updated by Ward Vandewege almost 3 years ago
- Target version deleted (
Arvados Future Sprints)
Updated by