Story #3187

[Workbench] Better information and features on "show pipeline instance" page

Added by Tom Clegg over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench
Target version:
Start date:
08/05/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
3.0

Description

Summary:
  • current state
  • when last changed state
  • total elapsed time of all finished/running jobs
  • some sort of error message from failed job (if any)
Details (for each component):
  • component name
  • job description: script_name, repository, script_version, docker image
  • if a job is submitted/assigned for the component: id (link), script_version, docker image, create timestamp, start/finish timestamp if applicable
  • was the job submitted in the course of running this pipeline, or was an existing job reused?[1]
  • status label (queued, running, finished, cancelled, failed)
  • progress bar (if running)
  • elapsed time [so far] if applicable
  • output and log links if applicable
  • "cancel job" button if applicable

1 This might not yet be recorded anywhere. Sometimes, but not always, job-creation-time < pipeline-start-time for existing jobs that were reused.

See Workbench UI images for layout advice.

Imported from #3188:

Structural:
  • Showing a job should look (nearly) the same as showing a pipeline instance with a single component. Some fields aren't applicable, like component name. Future work may change the CLI tools so jobs never even get submitted except as part of pipelines.
  • On the pipeline instance page, instead of linking to "job details", make all the details available on the pipeline page.
Job summary:
  • script name
  • current state (queued, running, finished, failed)
  • when last changed state
  • who submitted
  • progress bar (if running)
  • live log messages (if running)
  • number of tasks (total = done + running + todo)
  • number of task failures (if any)
  • link to output collection
Job details:
  • repository
  • script version
  • parameters (with preview of collections?)
  • runtime parameters
  • docker image
  • timestamps (submit, start, finish, cancel)
  • elapsed time between submit and cancel/start (or now)
  • elapsed time between start and cancel/finish (or now)

Subtasks

Task #3850: Add started_at and finished_at to tasks and pipelinesResolvedPeter Amstutz

Task #3862: Review 3187-pipeline-instance-pageResolvedPeter Amstutz

Task #3888: Fix testsResolvedPeter Amstutz

Task #3766: Send screen capture to Tom O'Keefe to get design/layout adviceResolvedPeter Amstutz

Task #3487: Show all component and job details, and overall timing statsResolvedPeter Amstutz

Task #3849: Review 3187-start-finish-timestamps-tasks-pipelinesResolvedWard Vandewege


Related issues

Related to Arvados - Story #3188: [Workbench] Merge "show job" functionality into "show pipeline instance" pageClosed

Associated revisions

Revision ac594f16
Added by Peter Amstutz about 5 years ago

Merge branch '3187-start-finish-timestamps-tasks-pipelines' refs #3187

Revision 9a88a95c (diff)
Added by Peter Amstutz about 5 years ago

Updated Gemfile to pull in changes from ac594f1 refs #3187

Revision 1d5d09e8 (diff)
Added by Peter Amstutz about 5 years ago

Bugfix extra space in display in render_runtime and queued time for pipeline.
refs #3187

Revision 92211c8b (diff)
Added by Peter Amstutz about 5 years ago

Bugfix time display, also bugfix for job state "Completed" -> "Complete". refs #3187

History

#1 Updated by Tom Clegg over 5 years ago

  • Target version set to 2014-08-06 Sprint

#2 Updated by Radhika Chippada over 5 years ago

  • Subject changed from Better information and features on "show pipeline instance" page to [Workbench] Better information and features on "show pipeline instance" page
  • Category set to Workbench

#3 Updated by Tom Clegg over 5 years ago

  • Description updated (diff)

#4 Updated by Tom Clegg over 5 years ago

  • Description updated (diff)

#5 Updated by Tom Clegg over 5 years ago

  • Description updated (diff)

#6 Updated by Tom Clegg over 5 years ago

  • Description updated (diff)

#7 Updated by Tom Clegg over 5 years ago

  • Assigned To set to Phil Hodgson

#8 Updated by Tom Clegg over 5 years ago

  • Target version changed from 2014-08-06 Sprint to Arvados Future Sprints

#9 Updated by Tom Clegg about 5 years ago

  • Target version changed from Arvados Future Sprints to 2014-08-27 Sprint

#10 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#11 Updated by Phil Hodgson about 5 years ago

  • Status changed from New to In Progress

#12 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#13 Updated by Ward Vandewege about 5 years ago

  • Target version changed from 2014-08-27 Sprint to 2014-09-17 sprint

#14 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)
  • Story points changed from 2.0 to 3.0

#15 Updated by Peter Amstutz about 5 years ago

  • Assigned To changed from Phil Hodgson to Peter Amstutz

#16 Updated by Peter Amstutz about 5 years ago

Plan:

  • Use collapsible panels for each component. Show summary at the top, user can click to expand and get details.
  • Use advanced log viewer for live log view. Add feature to filter by job.
  • Use single set of erb templates for job/template, render single job by creating stand-in pipeline instance?
  • Do something useful with crunchstat.
Additional desirable features:
  • Adjust timestamps using JS to use local timezone instead of displaying UTC.

#17 Updated by Peter Amstutz about 5 years ago

Also show queue position of queued jobs.

#18 Updated by Radhika Chippada about 5 years ago

Peter,
I may not have done a great job of reviewing any potential issues with the updates. However, I have a few comments based on my reading through the code and testing:

  • apps/workbench/app/assets/javascripts/dates.js
    • Forgotten console.log statement?
    • if you want to keep it, what are you trying to log? Can you please make it more descriptive?
    • Way too many “woble!” lines when I run the tests :)
    • “var re” can you call it “regexp” or “date_regexp”? I would also prefer we call “var v” as “var matches” or something like that.
  • pipeline_instances_helper.rb:
    • merge_range method - what is “j” referenced in comments?
    • determine_wallclock_runtime method
      • What is the input for this method? Is it “job” or range? In the PipelineInstancesControllerTest, you call the input to this method as “r”, which makes me wonder if it is “range”. Also, can you please use a more explanatory name in the test, such as “range” or “time_range” instead of “r”?
      • Forgotten “puts” statement in this method?
    • runtime method
      • It appears that this is not returning the pipeline’s runtime. Can you use a more descriptive name for this method? May be something like “component_runtime”?
      • It appears that the method parameter “long” indicates that the user wants a long description of the runtime duration? How about a name such as “long_description”? “long” on its own feels so confusing with number-like keyword
  • apps/workbench/app/views/pipeline_instances/_show_components_running.html.erb
    • Forgotten puts statement at lines 56 and 85?
    • indentation off at line 49
  • Question about nil check
    • Can “if not job[:cancelled_at].nil?” be simplified as “if !job[:cancelled_at]” ? There were a few such cases and just want to see if we can improve readability.
  • Pipeline UI: The “Components” tab has text “Has not started. . .”. This pipeline summary needs to be a bit more descriptive. Right now, I could not understand what it is telling me.
  • Advanced tab: Can we rename “curl example” as “Curl example” so that it looks similar to the other accordion titles? I looked at curl “man” page and it does use “Curl” some places and hence I think it does not seem too offensive to use the uppercase letter.
  • Two failing integration tests:
    • PipelineInstancesTest#test_Create_and_run_a_pipeline:
      test/integration/pipeline_instances_test.rb:78:in `block in <class:PipelineInstancesTest>'
    • PipelineInstancesTest#test_Create_pipeline_inside_a_project_and_run:
      test/integration/pipeline_instances_test.rb:141:in `block in <class:PipelineInstancesTest>'

#19 Updated by Peter Amstutz about 5 years ago

  • Target version changed from 2014-09-17 sprint to 2014-10-08 sprint

#20 Updated by Peter Amstutz about 5 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF