Bug #4524

[Workbench] Error running pipeline with File input data type.

Added by Radhika Chippada about 6 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Brett Smith
Category:
Workbench
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
0.5

Description

Run pipeline with File input (pipeline_templates/qr1hi-p5p6p-xt9iw6rmhvm93q5) -> Select a file (selected into a clippy before hand) -> Run it -> Pipeline failing with 404 (pipeline_instances/qr1hi-d1hrv-udkzowfb2qr65nb)

stderr arvados.errors.ApiError: <HttpError 404 when requesting https://qr1hi.arvadosapi.com/arvados/v1/collections/qr1hi-4zz18-pudu13a9iuwvgwg%2Ffile1_in_main.txt?alt=json returned "Path not found”

History

#1 Updated by Radhika Chippada about 6 years ago

  • Category set to Workbench

Run pipeline with File input (pipeline_templates/qr1hi-p5p6p-xt9iw6rmhvm93q5) -> Select a file (selected into a clippy before hand) -> Run it -> Pipeline failing with 404 (pipeline_instances/qr1hi-d1hrv-udkzowfb2qr65nb)

stderr arvados.errors.ApiError: <HttpError 404 when requesting https://qr1hi.arvadosapi.com/arvados/v1/collections/qr1hi-4zz18-pudu13a9iuwvgwg%2Ffile1_in_main.txt?alt=json returned "Path not found”

#2 Updated by Tim Pierce about 6 years ago

  • Description updated (diff)

The URL makes this look like the problem could be excessive URL-encoding. Is this a problem specific to Clippy? If so I'm inclined to call this a nice-to-have until we either retire Clippy or decided that we're not going to.

#3 Updated by Ward Vandewege about 6 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints

#4 Updated by Ward Vandewege about 6 years ago

  • Story points set to 0.5

#5 Updated by Tom Clegg about 6 years ago

  • Target version changed from Arvados Future Sprints to 2014-12-10 sprint

#6 Updated by Radhika Chippada about 6 years ago

  • Assigned To set to Radhika Chippada

#7 Updated by Radhika Chippada about 6 years ago

  • Assigned To deleted (Radhika Chippada)

#8 Updated by Brett Smith about 6 years ago

  • Assigned To set to Brett Smith

#9 Updated by Brett Smith almost 6 years ago

  • Status changed from New to In Progress

#10 Updated by Brett Smith almost 6 years ago

  • Status changed from In Progress to Closed

This is not an Arvados bug. This error is happening because of a mistake in the pipeline template: it says that the input parameter to the do_hash script is a File, but do_hash is actually expecting a Collection—it calls one_task_per_input_file to work iteratively over each file in the Collection. Ultimately do_hash crashes trying to treat the File spec as a Collection.

There's lots of discussion to be had about ways we might prevent this or make it clearer to the user—CollectionReader could support path syntax, or Arvados could have more information about how scripts expect their inputs to be formatted—but those need further discussion and speccing out as stories. Since I expect those to be handled separately, I'm closing this.

#11 Updated by Brett Smith almost 6 years ago

I just double-checked our documentation and confirmed that it doesn't suggest using do_hash with a File dataclass.

Also available in: Atom PDF