Story #2992

[Workbench] Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench.

Added by Tom Clegg almost 5 years ago. Updated 2 months ago.

Status:
Rejected
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
03/13/2019
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

background

Pipeline components are stored in a hash. In Ruby, and in the YAML representation of components that is stored in the database, the ordering of hash keys is consistent. However, when the components are handled by other languages such as Python (for example, by arv-copy) the order of components is likely to get permuted because key order is generally not predictable in hash/dictionary implementations.

Pipeline template authors should be able to control the display order (without relying on ordered dictionary implementations) by supplying a display_order key (with a numeric value) in each component.

solution / implementation

Workbench should sort components by display_order when rendering pipeline templates and instances.
  • If no display_order is given, don't sort: assume the hash implementation provides the correct order (as it does now).
API server, before saving a pipeline template, should:
  • Renumber the given display_order figures so they start at 1 and increase by 1.
  • Set max = number of components that had a display_order provided.
  • Iterate over all components (in whatever order is provided by the hash implementation, i.e., the order they appeared in the JSON document on the wire). Assign ++max to display_order for any component where it is missing.
  • Leave the components hash alone if it doesn't have the expected structure (e.g., it looks like {"foo":[1,2,3]} and Workbench wouldn't be able to render it anyway).

Older templates (if they are never updated) will still have undefined order if they end up getting copied/consumed by a Python program like arv-copy, etc. -- this is OK.

Update fixtures and examples in docs to include display_order.


Subtasks

Task #5855: Add Workbench supportClosed

Task #5854: Add default display_order on server sideClosedTom Clegg


Related issues

Related to Arvados - Story #3490: [Workbench] Create and edit pipeline templatesRejected

Related to Arvados - Bug #6095: [SDKs] Make arv-copy preserve pipeline template component orderResolved05/20/2015

Has duplicate Arvados - Feature #4051: [Workbench] Pipeline component display order should be controlled by sort_order attributes if present.Rejected

Has duplicate Arvados - Bug #4432: [SDKs] When copying pipeline instances using arv-copy, the component jobs get re-arrangedClosed11/05/2014

History

#1 Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)

#2 Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)

#3 Updated by Peter Amstutz over 4 years ago

  • Target version set to Arvados Future Sprints

#4 Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)

#5 Updated by Brett Smith over 4 years ago

I'm not sure lexical ordering is what our users want. Pipeline templates often have their components ordered by the logical order that the jobs run in. Arbitrarily reordering them by default might be surprising. That's why the current issue with arv-copy is so surprising: the copied pipeline, when viewed in Workbench, "looks" wrong.

What if the proposed display_order field was, by default, set to match the ordering in the components as submitted? That would go farther to preserve the current behavior, and still provide the flexibility proposed in this ticket. I think it should be possible to refresh it correctly on update by checking if components.changed? and not display_order.changed?.

#6 Updated by Peter Amstutz over 4 years ago

  • Target version changed from Arvados Future Sprints to 2015-01-07 sprint

#7 Updated by Peter Amstutz over 4 years ago

  • Target version changed from 2015-01-07 sprint to Arvados Future Sprints

#8 Updated by Tom Clegg about 4 years ago

  • Target version changed from Arvados Future Sprints to 2015-05-20 sprint

#9 Updated by Tom Clegg about 4 years ago

  • Subject changed from Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench. to [Workbench] Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench.
  • Description updated (diff)
  • Category set to Workbench

#10 Updated by Tom Clegg about 4 years ago

  • Assigned To set to Tom Clegg

#11 Updated by Brett Smith about 4 years ago

  • Target version changed from 2015-05-20 sprint to Arvados Future Sprints

#12 Updated by Tom Morris 2 months ago

  • Assigned To deleted (Tom Clegg)
  • Target version deleted (Arvados Future Sprints)

#13 Updated by Tom Morris 2 months ago

  • Category deleted (Workbench)

#14 Updated by Tom Morris 2 months ago

  • Status changed from New to Rejected

Pipeline template are gone

Also available in: Atom PDF