Project

General

Profile

Actions

Idea #3187

closed

[Workbench] Better information and features on "show pipeline instance" page

Added by Tom Clegg over 10 years ago. Updated about 10 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench
Target version:
Start date:
08/05/2014
Due date:
Story points:
3.0

Description

Summary:
  • current state
  • when last changed state
  • total elapsed time of all finished/running jobs
  • some sort of error message from failed job (if any)
Details (for each component):
  • component name
  • job description: script_name, repository, script_version, docker image
  • if a job is submitted/assigned for the component: id (link), script_version, docker image, create timestamp, start/finish timestamp if applicable
  • was the job submitted in the course of running this pipeline, or was an existing job reused?[1]
  • status label (queued, running, finished, cancelled, failed)
  • progress bar (if running)
  • elapsed time [so far] if applicable
  • output and log links if applicable
  • "cancel job" button if applicable

1 This might not yet be recorded anywhere. Sometimes, but not always, job-creation-time < pipeline-start-time for existing jobs that were reused.

See Workbench UI images for layout advice.

Imported from #3188:

Structural:
  • Showing a job should look (nearly) the same as showing a pipeline instance with a single component. Some fields aren't applicable, like component name. Future work may change the CLI tools so jobs never even get submitted except as part of pipelines.
  • On the pipeline instance page, instead of linking to "job details", make all the details available on the pipeline page.
Job summary:
  • script name
  • current state (queued, running, finished, failed)
  • when last changed state
  • who submitted
  • progress bar (if running)
  • live log messages (if running)
  • number of tasks (total = done + running + todo)
  • number of task failures (if any)
  • link to output collection
Job details:
  • repository
  • script version
  • parameters (with preview of collections?)
  • runtime parameters
  • docker image
  • timestamps (submit, start, finish, cancel)
  • elapsed time between submit and cancel/start (or now)
  • elapsed time between start and cancel/finish (or now)

Subtasks 6 (0 open6 closed)

Task #3850: Add started_at and finished_at to tasks and pipelinesResolvedPeter Amstutz08/05/2014Actions
Task #3862: Review 3187-pipeline-instance-pageResolvedPeter Amstutz08/05/2014Actions
Task #3888: Fix testsResolvedPeter Amstutz08/05/2014Actions
Task #3766: Send screen capture to Tom O'Keefe to get design/layout adviceResolvedPeter Amstutz08/05/2014Actions
Task #3487: Show all component and job details, and overall timing statsResolvedPeter Amstutz08/05/2014Actions
Task #3849: Review 3187-start-finish-timestamps-tasks-pipelinesResolvedWard Vandewege08/05/2014Actions

Related issues

Related to Arvados - Idea #3188: [Workbench] Merge "show job" functionality into "show pipeline instance" pageClosedActions
Actions #1

Updated by Tom Clegg over 10 years ago

  • Target version set to 2014-08-06 Sprint
Actions #2

Updated by Radhika Chippada over 10 years ago

  • Subject changed from Better information and features on "show pipeline instance" page to [Workbench] Better information and features on "show pipeline instance" page
  • Category set to Workbench
Actions #3

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #4

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #5

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #6

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #7

Updated by Tom Clegg over 10 years ago

  • Assigned To set to Phil Hodgson
Actions #8

Updated by Tom Clegg over 10 years ago

  • Target version changed from 2014-08-06 Sprint to Arvados Future Sprints
Actions #9

Updated by Tom Clegg over 10 years ago

  • Target version changed from Arvados Future Sprints to 2014-08-27 Sprint
Actions #10

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #11

Updated by Phil Hodgson over 10 years ago

  • Status changed from New to In Progress
Actions #12

Updated by Tom Clegg over 10 years ago

  • Description updated (diff)
Actions #13

Updated by Ward Vandewege about 10 years ago

  • Target version changed from 2014-08-27 Sprint to 2014-09-17 sprint
Actions #14

Updated by Tom Clegg about 10 years ago

  • Description updated (diff)
  • Story points changed from 2.0 to 3.0
Actions #15

Updated by Peter Amstutz about 10 years ago

  • Assigned To changed from Phil Hodgson to Peter Amstutz
Actions #16

Updated by Peter Amstutz about 10 years ago

Plan:

  • Use collapsible panels for each component. Show summary at the top, user can click to expand and get details.
  • Use advanced log viewer for live log view. Add feature to filter by job.
  • Use single set of erb templates for job/template, render single job by creating stand-in pipeline instance?
  • Do something useful with crunchstat.
Additional desirable features:
  • Adjust timestamps using JS to use local timezone instead of displaying UTC.
Actions #17

Updated by Peter Amstutz about 10 years ago

Also show queue position of queued jobs.

Actions #18

Updated by Radhika Chippada about 10 years ago

Peter,
I may not have done a great job of reviewing any potential issues with the updates. However, I have a few comments based on my reading through the code and testing:

  • apps/workbench/app/assets/javascripts/dates.js
    • Forgotten console.log statement?
    • if you want to keep it, what are you trying to log? Can you please make it more descriptive?
    • Way too many “woble!” lines when I run the tests :)
    • “var re” can you call it “regexp” or “date_regexp”? I would also prefer we call “var v” as “var matches” or something like that.
  • pipeline_instances_helper.rb:
    • merge_range method - what is “j” referenced in comments?
    • determine_wallclock_runtime method
      • What is the input for this method? Is it “job” or range? In the PipelineInstancesControllerTest, you call the input to this method as “r”, which makes me wonder if it is “range”. Also, can you please use a more explanatory name in the test, such as “range” or “time_range” instead of “r”?
      • Forgotten “puts” statement in this method?
    • runtime method
      • It appears that this is not returning the pipeline’s runtime. Can you use a more descriptive name for this method? May be something like “component_runtime”?
      • It appears that the method parameter “long” indicates that the user wants a long description of the runtime duration? How about a name such as “long_description”? “long” on its own feels so confusing with number-like keyword
  • apps/workbench/app/views/pipeline_instances/_show_components_running.html.erb
    • Forgotten puts statement at lines 56 and 85?
    • indentation off at line 49
  • Question about nil check
    • Can “if not job[:cancelled_at].nil?” be simplified as “if !job[:cancelled_at]” ? There were a few such cases and just want to see if we can improve readability.
  • Pipeline UI: The “Components” tab has text “Has not started. . .”. This pipeline summary needs to be a bit more descriptive. Right now, I could not understand what it is telling me.
  • Advanced tab: Can we rename “curl example” as “Curl example” so that it looks similar to the other accordion titles? I looked at curl “man” page and it does use “Curl” some places and hence I think it does not seem too offensive to use the uppercase letter.
  • Two failing integration tests:
    • PipelineInstancesTest#test_Create_and_run_a_pipeline:
      test/integration/pipeline_instances_test.rb:78:in `block in <class:PipelineInstancesTest>'
    • PipelineInstancesTest#test_Create_pipeline_inside_a_project_and_run:
      test/integration/pipeline_instances_test.rb:141:in `block in <class:PipelineInstancesTest>'
Actions #19

Updated by Peter Amstutz about 10 years ago

  • Target version changed from 2014-09-17 sprint to 2014-10-08 sprint
Actions #20

Updated by Peter Amstutz about 10 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF