Arvados

arvados-cwl-runner has a different help listings depending on whether arvados-cwl-runner or arvados-cwl-runner --help is being called. These probably should be the same.

When running a CWL job, I wanted to 'force' the job to rerun without reusing any previously saved state. Executing arvados-cwl-runner without any options brought out a listing of commands as expected:

usage: arvados-cwl-runner [-h] [--basedir BASEDIR] [--outdir OUTDIR]
                          [--no-container] [--preserve-environment ENVVAR]
                          [--preserve-entire-environment]
                          [--rm-container | --leave-container]
                          [--tmpdir-prefix TMPDIR_PREFIX]
                          [--tmp-outdir-prefix TMP_OUTDIR_PREFIX | --cachedir CACHEDIR]
                          [--rm-tmpdir | --leave-tmpdir]
                          [--move-outputs | --leave-outputs | --copy-outputs]
                          [--enable-pull | --disable-pull]
                          [--rdf-serializer RDF_SERIALIZER]
                          [--eval-timeout EVAL_TIMEOUT]
                          [--print-rdf | --print-dot | --print-pre | --print-deps | --print-input-deps | --pack | --version | --validate | --print-supported-versions]
                          [--strict | --non-strict] [--skip-schemas]
                          [--verbose | --quiet | --debug] [--tool-help]
                          [--relative-deps {primary,cwd}] [--enable-dev]
                          [--enable-ext]
                          [--default-container DEFAULT_CONTAINER]
                          [--no-match-user] [--disable-net]
                          [--custom-net CUSTOM_NET]
                          [--enable-ga4gh-tool-registry | --disable-ga4gh-tool-registry]
                          [--add-ga4gh-tool-registry GA4GH_TOOL_REGISTRIES]
                          [--on-error {stop,continue}] [--compute-checksum]
                          [--no-compute-checksum] [--relax-path-checks]
                          [--make-template] [--force-docker-pull]
                          [workflow] ...

Reference executor for Common Workflow Language

positional arguments:
  workflow
  job_order

optional arguments:
  -h, --help            show this help message and exit
  --basedir BASEDIR
  --outdir OUTDIR       Output directory, default current directory
  --no-container        Do not execute jobs in a Docker container, even when
                        specified by the CommandLineTool
  --preserve-environment ENVVAR
                        Preserve specific environment variable when running
                        CommandLineTools. May be provided multiple times.
...

but looking at the command list, I did not see any option for forcing reuse.

After talking with other people, it was clear there was a --disable-reuse option. I had missed it because I didn't provide the --help option. Listing the options via arvados-cwl-runner --help shows the option that I was looking for:

usage: arvados-cwl-runner [-h] [--basedir BASEDIR] [--outdir OUTDIR]
                          [--eval-timeout EVAL_TIMEOUT]
                          [--print-dot | --version | --validate]
                          [--verbose | --quiet | --debug] [--metrics]
                          [--tool-help] [--enable-reuse | --disable-reuse]
                          [--project-uuid UUID] [--output-name OUTPUT_NAME]
                          [--output-tags OUTPUT_TAGS]
                          [--ignore-docker-for-reuse]
                          [--submit | --local | --create-template | --create-workflow | --update-workflow UUID]
                          [--wait | --no-wait]
                          [--log-timestamps | --no-log-timestamps]
                          [--api {jobs,containers}] [--compute-checksum]
                          [--submit-runner-ram SUBMIT_RUNNER_RAM]
                          [--submit-runner-image SUBMIT_RUNNER_IMAGE]
                          [--name NAME] [--on-error {stop,continue}]
                          [--enable-dev] [--intermediate-output-ttl N]
                          [--trash-intermediate | --no-trash-intermediate]
                          [workflow] ...

Arvados executor for Common Workflow Language

positional arguments:
  workflow              The workflow to execute
  job_order             The input object to the workflow.

optional arguments:
  -h, --help            show this help message and exit
  --basedir BASEDIR     Base directory used to resolve relative references in
                        the input, default to directory of input object file
                        or current directory (if inputs piped/provided on
                        command line).
  --outdir OUTDIR       Output directory, default current directory
  --eval-timeout EVAL_TIMEOUT
                        Time to wait for a Javascript expression to evaluate
                        before giving an error, default 20s.
  --print-dot           Print workflow visualization in graphviz format and
                        exit
  --version             Print version and exit
  --validate            Validate CWL document only.
  --verbose             Default logging
  --quiet               Only print warnings and errors.
  --debug               Print even more logging
  --metrics             Print timing metrics
  --tool-help           Print command line help for tool
  --enable-reuse        Enable job or container reuse (default)
  --disable-reuse       Disable job or container reuse
...

arvados-cwl-runner should probably have consistency between help listings.