Story #4030

[SDK] Command line tool to manipulate collection manifests

Added by Peter Amstutz over 4 years ago. Updated over 4 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

The science team has expressed the need for a command line tool to provide comprehensive editing of collections/manifests. Some possible features include:

  • Delete files or streams from a collection
  • Filter collection (include/exclude files by regex)
  • Rename files or streams (by name or by regex)
  • Move files/streams from one collection to another
  • Combine manifests to get a new manifest
  • Add file(s) from another collection
  • Upload files using arvados.command.put module and add to collection
  • Optimize manifest by re-packing small files (or files that are split across many more blocks than necessary) into continuous ranges that fills whole blocks (possibly with some logic to avoid starting new files at the very end of a block)

User should be able to specify whether to modify collection in place, create new collection, or operate on manifest text read in from stdin.

History

#1 Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)

#2 Updated by Peter Amstutz over 4 years ago

  • Subject changed from Command line tools to manipulate collection manifests to Command line tool to manipulate collection manifests
  • Description updated (diff)

#3 Updated by Ward Vandewege over 4 years ago

  • Subject changed from Command line tool to manipulate collection manifests to [SDK] Command line tool to manipulate collection manifests

#4 Updated by Ward Vandewege over 4 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints

Also available in: Atom PDF