Project

General

Profile

Actions

Bug #19699

closed

HTTP download creates collections with too-long names, needs flag to run in runner process after submission

Added by Peter Amstutz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
CWL
Target version:
Story points:
-
Release relationship:
Auto

Description

Customer issues that came up:

  • Customer needs to run on a lot of samples which means launching a whole bunch of a-c-r processes. The feature that automatically transfers from HTTP to Keep happens before submitting the workflow. This will scale much better if the data transfer happens on the compute node when the workflow actually launches.
  • The collection is named "Downloaded from http://..." and if the URL is too long, it will exceed the 255 character limit on collection names. a-c-r needs to account for the limit (probably also including the timestamp that gets added by ensure_unique_name) and trim the name to a valid length so it won't get rejected.

Files


Subtasks 2 (0 open2 closed)

Task #19703: Review 19699-cwl-http-dl ResolvedPeter Amstutz11/14/2022Actions
Task #19812: Review 19699-cwl-dl-docsResolvedLucas Di Pentima11/14/2022Actions

Related issues

Related to Arvados - Bug #19688: Launch registered workflows fasterResolvedPeter Amstutz11/14/2022Actions
Actions

Also available in: Atom PDF