Git strategy for pipeline development¶

• You have a project that involves one or two pipelines
• Each pipeline have many components
• These pipelines and components make use of a common code base

Example:

crunch_scripts/align
crunch_scripts/call
crunch_scripts/compare

align(1)    align(2)
   |           |
call(1)     call(2)
   |____   ____|
        | |
      compare

align(1)    align(2)    align(3)
   |           |           |
call(1)     call(2)     call(3)
   |_________  |  _________|
             | | |
            compare

• Fix a bug in compare that was making it fail when given 3 inputs.
• Update the code, commit, push, and re-run. (Note: results from previous runs that succeeded are still valid.)
• Find a bug in compare that was making it produce incorrect output when given 3 inputs.
• Update the code, commit, push, and re-run. Update pipeline template B to prevent the broken jobs (the ones that are marked "success" but produced incorrect outputs) from being re-used in future pipeline runs. (Note: results from previous jobs from pipeline A are still OK.)

A strategy¶