Feature #10498

[CWL] Single --register subcommand selects workflow or pipeline

Added by Peter Amstutz almost 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
11/17/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

Currently there are separate --create-template, --create-workflow and --update-workflow subcommands. The user needs to know which API is in use (jobs or containers) to use the correct one.

Merge them into a --register-workflow and --update-workflow and select the API to use (pipeline template or workflow) based on existing logic that detects API from the discovery document.

Update user guide.


Subtasks

Task #10504: Review 10498-register-with-chosen-apiResolvedRadhika Chippada


Related issues

Has duplicate Arvados - Bug #10520: [CrunchV2] arvados-cwl-runner unhandled error when using --create-templateResolved

Associated revisions

Revision 322d7845
Added by Tom Clegg almost 5 years ago

Merge branch '10498-register-with-chosen-api' closes #10498

Revision 7588bb6a
Added by Peter Amstutz almost 5 years ago

Merge branch '10498-arvdocker-project-uuid' refs #10498

History

#1 Updated by Peter Amstutz almost 5 years ago

  • Story points set to 0.5

#2 Updated by Peter Amstutz almost 5 years ago

  • Description updated (diff)

#3 Updated by Tom Clegg almost 5 years ago

  • Assigned To set to Tom Clegg

#4 Updated by Tom Clegg almost 5 years ago

10498-register-with-chosen-api

test a4e81f561cce966074a72ff6219826a89152451a

from the commit message...

    10498: --create-workflow and --create-template do the same thing, depending on --api choice.

    Also:

    * --update-workflow works for pipeline templates, too.

    * --update-workflow establishes --api=jobs or --api=containers
      depending on the UUID. (If this conflicts with an explicit --api
      argument, that is an error.)

    * If no --project-uuid is given, the user's UUID is not passed
      explicitly with create/update requests. The API will default to the
      user's UUID or (when updating an existing object) leaving the
      existing object's owner unchanged.

#5 Updated by Tom Clegg almost 5 years ago

  • Status changed from New to In Progress

#6 Updated by Radhika Chippada almost 5 years ago

cwl-runner.html

  • "Registering a workflow to use in Workbench" => I think this should be "Registering a workflow to use in Arvados" because as the first line in description says it can be used by other areas of Arvados, not just workbench?

init.py

  • I think we should call "update_uuid" as "workflow_uuid"? I think calling it that would make it clear and no need to wonder what type of uuid it is. Similar to "project_uuid", "task_uuid" etc. Now that we are overloading "workflow" to also mean "template", it seems appropriate to call it that now.
  • "Create an Arvados workflow or pipeline template (depending on selected API, see --api)" => Can we spell it out instead? Something like "Create an Arvados workflow when using containers api or pipeline template when using jobs api (see --api for more information)" ?

tests

  • Please consider adding a test_incompatible_api with "containers api and template uuid for update_workflow" as well

#7 Updated by Tom Clegg almost 5 years ago

Radhika Chippada wrote:

cwl-runner.html

  • "Registering a workflow to use in Workbench" => I think this should be "Registering a workflow to use in Arvados" because as the first line in description says it can be used by other areas of Arvados, not just workbench?

I thought of that too because technically that's true, but OTOH saying "in Arvados" seems confusing here -- aren't we already doing everything in Arvados at this point? E.g., an earlier section is "Submitting a workflow to an Arvados cluster" (aside: should that be "executing" instead? "Submitting" sounds a bit like what "registering" actually does...)

  • I think we should call "update_uuid" as "workflow_uuid"? I think calling it that would make it clear and no need to wonder what type of uuid it is. Similar to "project_uuid", "task_uuid" etc. Now that we are overloading "workflow" to also mean "template", it seems appropriate to call it that now.

At that point it is either a workflow UUID or a pipeline_template UUID... but I've renamed it to "existing_uuid" which (at least to me) seems more evocative.

  • "Create an Arvados workflow or pipeline template (depending on selected API, see --api)" => Can we spell it out instead? Something like "Create an Arvados workflow when using containers api or pipeline template when using jobs api (see --api for more information)" ?

Updated:

Create an Arvados workflow (if using the 'containers' API) or pipeline template (if using the 'jobs' API). See --api.
  • Please consider adding a test_incompatible_api with "containers api and template uuid for update_workflow" as well

Considered... but seems like a dubious use of testing volume/complexity. The "a != b" code being tested is pretty simple, and even if the sanity check doesn't work, the result is just a different error later on when the update() call fails 404...

I did update this test to capture the logs and ensure this error is actually reported, rather than just exiting 1 for some other reason.

test 6f2ac70b70d4b64b728b815d5be429d5a165e2f0

#8 Updated by Radhika Chippada almost 5 years ago

Tom said:

"Registering a workflow to use in Arvados" => I thought of that too because technically that's true, but OTOH saying "in Arvados" seems confusing here

My concern is that it is not only usable by Workbench, by other areas of Arvados as well. Can we use ""Registering a workflow" instead?

Agree with the other comments.

LGTM (with or without that doc title update). Thanks.

#9 Updated by Tom Clegg almost 5 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:322d784513c37abc8fa1d1c05dd5f41019868735.

Also available in: Atom PDF