Idea #9397
Updated by Tom Clegg over 7 years ago
This is needed to support the CWL InitialWorkDirRequirement which is needed for -full CWL compliance- Crunch1 feature parity. h3. Minimum required functionality Design: When a container's output_path is a tmp mount backed by local disk, this output directory can be pre-populated with content from existing collections. * Initial content is specified in the container request by mounting collections at mount points that are subdirectories of output_path. * Mount points underneath output_path must _not_ have @"writable":true@ -- if any of them do, the API refuses # We want to create/update the container request, and (just support entries in case the API does not catch this problem) crunch-run fails the container. * When the container starts, these existing collections and files "mounts" which are readable at targeted inside the specified mount points. * When the container finishes, the mounted collections/files are included in the output collection at the specified mount points. IOW, the container's output is equal to what the container sees in output_path just before it exits. *Except:* If a mount has @"exclude_from_output":true@ then it is omitted from the container's output collection. directory. * If a process in the container tries to modify, remove, or rename these mount points or anything underneath them, the operation fails and the container output is unaffected (as are the underlying collections used to pre-populate). h3. Implementation # In crunchrun.go SetupMounts(), sort the keys in "runner.Container.Mounts" such so that parents they are processed in order. This is so that the volume bind for the output directory is listed before children (e.g., alphanumerically or by length). any binds for files within the output directory # The inconsistency between the crunch-run and the spec noted in #note-10 needs to be fixed to follow the spec # In crunchrun.go CaptureOutput(), after getting @manifestText@, go through runner.Container.Mounts and search for mount points beginning mounts that begin with runner.Container.OutputPath that do not have @"exclude_from_output":true@. runner.Container.OutputPath. # For each such file and or directory that was mounted inside the output directory, add the relevant manifest fragment it to the container output manifest, modifying stream/file names as needed. @manifestText@. The last one may be the most complicated part of the ticket just due to the fact that there is much less infrastructure for manipulating collections in Go than compared to the Python SDK.