Project

General

Profile

Actions

Bug #4524

closed

[Workbench] Error running pipeline with File input data type.

Added by Radhika Chippada over 9 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Category:
Workbench
Target version:
Story points:
0.5

Description

Run pipeline with File input (pipeline_templates/qr1hi-p5p6p-xt9iw6rmhvm93q5) -> Select a file (selected into a clippy before hand) -> Run it -> Pipeline failing with 404 (pipeline_instances/qr1hi-d1hrv-udkzowfb2qr65nb)

stderr arvados.errors.ApiError: <HttpError 404 when requesting https://qr1hi.arvadosapi.com/arvados/v1/collections/qr1hi-4zz18-pudu13a9iuwvgwg%2Ffile1_in_main.txt?alt=json returned "Path not found”

Actions #1

Updated by Radhika Chippada over 9 years ago

  • Category set to Workbench

Run pipeline with File input (pipeline_templates/qr1hi-p5p6p-xt9iw6rmhvm93q5) -> Select a file (selected into a clippy before hand) -> Run it -> Pipeline failing with 404 (pipeline_instances/qr1hi-d1hrv-udkzowfb2qr65nb)

stderr arvados.errors.ApiError: <HttpError 404 when requesting https://qr1hi.arvadosapi.com/arvados/v1/collections/qr1hi-4zz18-pudu13a9iuwvgwg%2Ffile1_in_main.txt?alt=json returned "Path not found”

Actions #2

Updated by Tim Pierce over 9 years ago

  • Description updated (diff)

The URL makes this look like the problem could be excessive URL-encoding. Is this a problem specific to Clippy? If so I'm inclined to call this a nice-to-have until we either retire Clippy or decided that we're not going to.

Actions #3

Updated by Ward Vandewege over 9 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints
Actions #4

Updated by Ward Vandewege over 9 years ago

  • Story points set to 0.5
Actions #5

Updated by Tom Clegg over 9 years ago

  • Target version changed from Arvados Future Sprints to 2014-12-10 sprint
Actions #6

Updated by Radhika Chippada over 9 years ago

  • Assigned To set to Radhika Chippada
Actions #7

Updated by Radhika Chippada over 9 years ago

  • Assigned To deleted (Radhika Chippada)
Actions #8

Updated by Brett Smith over 9 years ago

  • Assigned To set to Brett Smith
Actions #9

Updated by Brett Smith about 9 years ago

  • Status changed from New to In Progress
Actions #10

Updated by Brett Smith about 9 years ago

  • Status changed from In Progress to Closed

This is not an Arvados bug. This error is happening because of a mistake in the pipeline template: it says that the input parameter to the do_hash script is a File, but do_hash is actually expecting a Collection—it calls one_task_per_input_file to work iteratively over each file in the Collection. Ultimately do_hash crashes trying to treat the File spec as a Collection.

There's lots of discussion to be had about ways we might prevent this or make it clearer to the user—CollectionReader could support path syntax, or Arvados could have more information about how scripts expect their inputs to be formatted—but those need further discussion and speccing out as stories. Since I expect those to be handled separately, I'm closing this.

Actions #11

Updated by Brett Smith about 9 years ago

I just double-checked our documentation and confirmed that it doesn't suggest using do_hash with a File dataclass.

Actions

Also available in: Atom PDF