Actions
Feature #9964
closedarvados-cwl-runner limits output data to keep using output_glob
Added by Bryan Cosca about 8 years ago. Updated 5 months ago.
Status:
Resolved
Priority:
Normal
Assigned To:
Category:
CWL
Target version:
Story points:
-
Release:
Release relationship:
Auto
Description
Once we have a feature to tell crunch-run to limit which files to be uploaded, have arvados-cwl-runner start using it.
Related issues
Updated by Tom Morris almost 8 years ago
- Assigned To set to Tom Morris
- Target version set to Arvados Future Sprints
Updated by Ward Vandewege over 3 years ago
- Target version deleted (
Arvados Future Sprints)
Updated by Peter Amstutz over 1 year ago
- Release deleted (
60) - Target version set to Future
- Assigned To deleted (
Tom Morris) - Category set to CWL
- Subject changed from [CWL][Crunch2][Crunch] crunchrunner should use CWL globs to output data to keep to arvados-cwl-runner limits output data to keep using output_glob
- Tracker changed from Bug to Feature
Updated by Peter Amstutz over 1 year ago
- Related to deleted (Feature #12430: Crunch2 limit output collection to glob patterns)
Updated by Peter Amstutz over 1 year ago
- Blocked by Feature #12430: Crunch2 limit output collection to glob patterns added
Updated by Peter Amstutz over 1 year ago
- Related to Bug #20561: Log when files from input are being propagated to output in crunch-run finalization added
Updated by Peter Amstutz about 1 year ago
- Target version changed from Future to Development 2024-01-03 sprint
Updated by Peter Amstutz about 1 year ago
- Target version changed from Development 2024-01-03 sprint to Development 2024-01-17 sprint
Updated by Peter Amstutz 11 months ago
- Target version changed from Development 2024-01-17 sprint to Development 2024-01-31 sprint
Updated by Peter Amstutz 11 months ago
- Target version changed from Development 2024-01-31 sprint to Development 2024-02-14 sprint
Updated by Peter Amstutz 10 months ago
- Target version changed from Development 2024-02-14 sprint to Development 2024-02-28 sprint
Updated by Peter Amstutz 9 months ago
- Target version changed from Development 2024-02-28 sprint to Development 2024-03-27 sprint
Updated by Peter Amstutz 9 months ago
- Target version changed from Development 2024-03-27 sprint to Development 2024-04-10 sprint
Updated by Peter Amstutz 8 months ago
- Target version changed from Development 2024-04-10 sprint to Development 2024-04-24 sprint
Updated by Peter Amstutz 8 months ago
- Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Updated by Peter Amstutz 8 months ago
- Target version changed from Development 2024-05-08 sprint to Development 2024-04-24 sprint
Updated by Peter Amstutz 7 months ago
- Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Updated by Peter Amstutz 7 months ago
- Target version changed from Development 2024-05-08 sprint to Development 2024-06-05 sprint
Updated by Peter Amstutz 6 months ago
- Target version changed from Development 2024-06-05 sprint to Development 2024-06-19 sprint
Updated by Peter Amstutz 6 months ago
9964-output-glob-acr @ 59ac77450a978b8bcfcbfdcc01b191ea91a2bc86
- All agreed upon points are implemented / addressed.
- Translates CWL
outputBinding->glob
tooutput_glob
- Translates CWL
- Anything not implemented (discovered or discussed during work) has a follow-up story.
- One corner case came up in development. If both outputBinding glob and cwl.output.json are used in the same CommandLineTool and cwl.output.json refers to files that don't match any globs, those files won't be uploaded. This situation cannot be detected automatically, but also may not be something that ever appears in practice. I discuss this in a comment.
- Code is tested and passing, both automated and manual, what manual testing was done is described
- added a unit test and ran the CWL 1.2 test suite
- Documentation has been updated.
- improves the implementation of existing CWL functionality, doesn't require any documentation changes.
- Behaves appropriately at the intended scale (describe intended scale).
- will improve performance in situations where there are a lot of files in the working directory and only a small subset are considered output by the workflow, by excluding those extra files from the output collection
- Considered backwards and forwards compatibility issues between client and server.
- Updated the discovery doc
revision
to 20240502 (this is the date theoutput_glob
feature was merged, it really should have been updated then) and made the feature conditional on the API server advertising the minimum version
- Updated the discovery doc
- Follows our coding standards and GUI style guidelines.
- yes
Updated by Peter Amstutz 6 months ago
Not sure why this is failing, re-run:
Updated by Peter Amstutz 6 months ago
9964-output-glob-acr @ 917e8925b29567a9324988767667235d5540f878
Updated by Peter Amstutz 6 months ago
9964-output-glob-acr @ 9364cbd0174e9fd8f4394721ff575c59f8d42ff1
Fixed some tests
Updated by Peter Amstutz 5 months ago
- Status changed from In Progress to Resolved
Actions