Arvados

background¶

Pipeline components are stored in a hash. In Ruby, and in the YAML representation of components that is stored in the database, the ordering of hash keys is consistent. However, when the components are handled by other languages such as Python (for example, by arv-copy) the order of components is likely to get permuted because key order is generally not predictable in hash/dictionary implementations.

Pipeline template authors should be able to control the display order (without relying on ordered dictionary implementations) by supplying a display_order key (with a numeric value) in each component.

solution / implementation¶

Workbench should sort components by display_order when rendering pipeline templates and instances.

• If no display_order is given, don't sort: assume the hash implementation provides the correct order (as it does now).

API server, before saving a pipeline template, should:

• Renumber the given display_order figures so they start at 1 and increase by 1.
• Set max = number of components that had a display_order provided.
• Iterate over all components (in whatever order is provided by the hash implementation, i.e., the order they appeared in the JSON document on the wire). Assign ++max to display_order for any component where it is missing.
• Leave the components hash alone if it doesn't have the expected structure (e.g., it looks like {"foo":[1,2,3]} and Workbench wouldn't be able to render it anyway).

Older templates (if they are never updated) will still have undefined order if they end up getting copied/consumed by a Python program like arv-copy, etc. -- this is OK.

Update fixtures and examples in docs to include display_order.