Build docker images as part of a workflow » History » Version 1
Tom Clegg, 07/12/2018 02:02 PM
| 1 | 1 | Tom Clegg | h1. Build docker images as part of a workflow |
|---|---|---|---|
| 2 | |||
| 3 | (draft) |
||
| 4 | |||
| 5 | Background |
||
| 6 | |||
| 7 | Container images provide a well-defined execution environment for doing reproducible work. As long as the image is runnable by a container engine, a job can be repeated. However, the point of reproducibility isn't just to allow repetition of the same computation -- it's to make it possible to use prior work as the starting point for future work. Much of this opportunity is lost if the provenance trail ends at a giant binary image. |
||
| 8 | |||
| 9 | Example: When a bug is discovered in an analysis tool or library, it should be tractable to identify which existing results are affected and re-run those analyses with the updated software. |
||
| 10 | |||
| 11 | Requirements |
||
| 12 | |||
| 13 | Users should have the option of building container images... |
||
| 14 | * ...as part of a CWL workflow (so they can update the image-building instructions and hit one "re-run" button to see the result) |
||
| 15 | * ...in Arvados containers (so the build environment is controlled, build logs are saved, etc.) |
||
| 16 | * ...without having docker on the client side (so build-and-run workflows can be initiated from browsers, non-Linux workstations, and shared VM environments) |