Project

General

Profile

Actions

Idea #2992

closed

[Workbench] Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench.

Added by Tom Clegg almost 10 years ago. Updated about 5 years ago.

Status:
Rejected
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
0.5

Description

background

Pipeline components are stored in a hash. In Ruby, and in the YAML representation of components that is stored in the database, the ordering of hash keys is consistent. However, when the components are handled by other languages such as Python (for example, by arv-copy) the order of components is likely to get permuted because key order is generally not predictable in hash/dictionary implementations.

Pipeline template authors should be able to control the display order (without relying on ordered dictionary implementations) by supplying a display_order key (with a numeric value) in each component.

solution / implementation

Workbench should sort components by display_order when rendering pipeline templates and instances.
  • If no display_order is given, don't sort: assume the hash implementation provides the correct order (as it does now).
API server, before saving a pipeline template, should:
  • Renumber the given display_order figures so they start at 1 and increase by 1.
  • Set max = number of components that had a display_order provided.
  • Iterate over all components (in whatever order is provided by the hash implementation, i.e., the order they appeared in the JSON document on the wire). Assign ++max to display_order for any component where it is missing.
  • Leave the components hash alone if it doesn't have the expected structure (e.g., it looks like {"foo":[1,2,3]} and Workbench wouldn't be able to render it anyway).

Older templates (if they are never updated) will still have undefined order if they end up getting copied/consumed by a Python program like arv-copy, etc. -- this is OK.

Update fixtures and examples in docs to include display_order.


Subtasks 2 (0 open2 closed)

Task #5855: Add Workbench supportClosed03/13/2019Actions
Task #5854: Add default display_order on server sideClosedTom Clegg03/13/2019Actions

Related issues

Related to Arvados - Idea #3490: [Workbench] Create and edit pipeline templatesRejectedActions
Related to Arvados - Bug #6095: [SDKs] Make arv-copy preserve pipeline template component orderResolvedBrett Smith05/20/2015Actions
Has duplicate Arvados - Feature #4051: [Workbench] Pipeline component display order should be controlled by sort_order attributes if present.RejectedActions
Has duplicate Arvados - Bug #4432: [SDKs] When copying pipeline instances using arv-copy, the component jobs get re-arrangedClosed11/05/2014Actions
Actions #1

Updated by Peter Amstutz over 9 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 9 years ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz over 9 years ago

  • Target version set to Arvados Future Sprints
Actions #4

Updated by Peter Amstutz over 9 years ago

  • Description updated (diff)
Actions #5

Updated by Brett Smith over 9 years ago

I'm not sure lexical ordering is what our users want. Pipeline templates often have their components ordered by the logical order that the jobs run in. Arbitrarily reordering them by default might be surprising. That's why the current issue with arv-copy is so surprising: the copied pipeline, when viewed in Workbench, "looks" wrong.

What if the proposed display_order field was, by default, set to match the ordering in the components as submitted? That would go farther to preserve the current behavior, and still provide the flexibility proposed in this ticket. I think it should be possible to refresh it correctly on update by checking if components.changed? and not display_order.changed?.

Actions #6

Updated by Peter Amstutz over 9 years ago

  • Target version changed from Arvados Future Sprints to 2015-01-07 sprint
Actions #7

Updated by Peter Amstutz over 9 years ago

  • Target version changed from 2015-01-07 sprint to Arvados Future Sprints
Actions #8

Updated by Tom Clegg almost 9 years ago

  • Target version changed from Arvados Future Sprints to 2015-05-20 sprint
Actions #9

Updated by Tom Clegg almost 9 years ago

  • Subject changed from Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench. to [Workbench] Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench.
  • Description updated (diff)
  • Category set to Workbench
Actions #10

Updated by Tom Clegg almost 9 years ago

  • Assigned To set to Tom Clegg
Actions #11

Updated by Brett Smith almost 9 years ago

  • Target version changed from 2015-05-20 sprint to Arvados Future Sprints
Actions #12

Updated by Tom Morris about 5 years ago

  • Assigned To deleted (Tom Clegg)
  • Target version deleted (Arvados Future Sprints)
Actions #13

Updated by Tom Morris about 5 years ago

  • Category deleted (Workbench)
Actions #14

Updated by Tom Morris about 5 years ago

  • Status changed from New to Rejected

Pipeline template are gone

Actions

Also available in: Atom PDF