Project

General

Profile

Actions

Bug #13849

closed

[CWL] secondaryFiles checks failed in single container

Added by Jiayong Li over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

single container
https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-1vziek0s8udmajx

non-single container
https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-g0u996tfq07d4mp

Note that the non-single container run failed, with the secondaryFiles checks prompted errors, as they should.

cwltool ERROR: Cannot make scatter job: Missing required secondary file 'chr1.1kg.phase3.v5a.vcf.gz.tbi' from file object: {
    "basename": "chr1.1kg.phase3.v5a.vcf.gz", 
    "nameroot": "chr1.1kg.phase3.v5a.vcf", 
    "nameext": ".gz", 
    "location": "keep:ba7cd392bc4aa229c3c771b496e79628+9990/chr1.1kg.phase3.v5a.vcf.gz", 
    "secondaryFiles": [], 
    "class": "File" 
}

This means the secondaryFiles checks failed in the single container version. See the attached tar ball for cwl. Run with

arvados-cwl-runner phasing-wf.cwl phasing-NA12878.yml


Files

Phasing.tar.gz (1.49 KB) Phasing.tar.gz Jiayong Li, 07/18/2018 05:52 PM
Actions #1

Updated by Jiayong Li over 5 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Jiayong Li over 5 years ago

Actions #3

Updated by Tom Morris over 5 years ago

  • Assigned To set to Peter Amstutz
  • Target version set to 2018-08-15 Sprint
Actions #4

Updated by Peter Amstutz over 5 years ago

The expectation is that it would fail when run with RunInSingleContainer, but it didn't fail.

The diagnosis is that RunInSingleContainer uses cwltool inside the container. If the version of cwltool is older, in isn't enforcing those checks.

The solution is to rebuild the container image with a newer arvados-cwl-runner / cwltool.

Actions #5

Updated by Peter Amstutz over 5 years ago

  • Status changed from New to Resolved
Actions #6

Updated by Tom Morris over 5 years ago

  • Release set to 13
Actions

Also available in: Atom PDF