Feature #16672

Live container logs panel

Added by Peter Amstutz almost 2 years ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench2
Target version:
Start date:
03/22/2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Incorporate live container logs into a sub panel of process view

  • Should support resizable height. Can be partial height or maximized.
  • Select which log stream to show, or all of them combined (as it already works now) (combined view needs to be sorted by timestamp)
  • Fix the issue where text sometimes overlaps with itself (wraparound or something else)
  • Stretch goals for log view
    • Make URLs, UUIDs and PDH clickable in the log view
    • Rewrite timestamp on each log line from UTC to user's local time zone

Subtasks

Task #18829: Review 16672-live-container-logsResolvedLucas Di Pentima


Related issues

Related to Arvados Epics - Story #16945: WB2 Workflows / containers feature parityIn Progress08/01/202105/31/2022

Related to Arvados Workbench 2 - Bug #16246: [Process view] Logs are difficult to reachResolved

Associated revisions

Revision 342f4ed1
Added by Lucas Di Pentima about 2 months ago

Merge branch '16672-live-container-logs'. Closes #16672

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <>

History

#1 Updated by Peter Amstutz over 1 year ago

  • Related to Story #16945: WB2 Workflows / containers feature parity added

#2 Updated by Peter Amstutz 3 months ago

  • Target version set to 2022-03-16 sprint
  • Assigned To set to Lucas Di Pentima
  • Description updated (diff)

#3 Updated by Peter Amstutz 3 months ago

  • Release changed from 31 to 47
  • Category set to Workbench2
  • Subject changed from Live container logs not updating to Live container logs panel
  • Tracker changed from Bug to Feature

#4 Updated by Peter Amstutz 3 months ago

  • Description updated (diff)

#5 Updated by Lucas Di Pentima 2 months ago

  • Status changed from New to In Progress

#6 Updated by Lucas Di Pentima 2 months ago

  • Target version changed from 2022-03-16 sprint to 2022-03-30 Sprint

#7 Updated by Lucas Di Pentima 2 months ago

  • Related to Bug #16246: [Process view] Logs are difficult to reach added

#8 Updated by Lucas Di Pentima 2 months ago

Peter Amstutz wrote:

  • Select which log stream to show, or all of them combined (as it already works now) (combined view needs to be sorted by timestamp)

I've been looking at this, and there's an issue related to the fact that logs are periodically flushed to the logs table and their "event_at" data seem to save the "flush time" (or rather, log record creation time?) so a combined log view may include incorrectly sorted lines. Workbench1 currently suffers from this issue.

For a perfect "all logs" merging, the code should split the lines and then parse them by their real timestamps... but there're other logs (node-info, container) that don't have embedded timestamps.

#9 Updated by Peter Amstutz 2 months ago

Lucas Di Pentima wrote:

Peter Amstutz wrote:

  • Select which log stream to show, or all of them combined (as it already works now) (combined view needs to be sorted by timestamp)

I've been looking at this, and there's an issue related to the fact that logs are periodically flushed to the logs table and their "event_at" data seem to save the "flush time" (or rather, log record creation time?) so a combined log view may include incorrectly sorted lines. Workbench1 currently suffers from this issue.

For a perfect "all logs" merging, the code should split the lines and then parse them by their real timestamps... but there're other logs (node-info, container) that don't have embedded timestamps.

I think best effort is perfectly fine here.

#10 Updated by Lucas Di Pentima 2 months ago

Updates at arvados-workbench2|29df452f - branch 16672-live-container-logs - developer-tests-workbench2: #621

New features / fixes

  • Improves the Multi Panel View task bar panel switching feature by waiting some milliseconds before switching to any panel. This avoids most accidental panel switches when quickly passing the mouse pointer over the task bar.
  • Removes the "Log" menu item from the process' "More" (3-dotted) button.
  • Adds MPVContent panel with the Log viewer, visible by default (for now? should we leave it hidden unless the process is in "Running" state?)
  • The Log panel can be maximized, and will take up to 50% of the viewport when not.
  • Adds font size control buttons (with 5 steps total available)
  • Moves the log selector & "Go to collection" link to the panel's top to save vertical space. The "Go to collection" link is now a button.
  • Adds word-wrapping control capabilities. By default, the log viewer's word-wrap mode is enabled.
  • Adds "Copy to clipboard" feature for easy copy&pasting logs.
  • Adds log "auto-follow" mode: in a running process scenario, if the user scrolls the log viewer all the way down, the viewer will auto-scroll to the bottom when new logs appear.
  • Adds UUID & PDH link rending on logs.

Pending / missing / left for a future round?

  • URL link rendering on logs (this wasn't added because its behavior should be different as internal UUID & PDH links, IMO).
  • Log timestamp auto-conversion (should this be toggable?, if not: what should the "copy to clipboard" feature copy?)
  • Perfect log ordering by timestamp: This is currently somewhat difficult to do because log records usually contain multiple log lines, as it's flushed periodically to the database.
  • Tests: we should have some way to run a container or set up a fixture so that we can test the UI.

#11 Updated by Lucas Di Pentima 2 months ago

Stephen: Re: tests -- I'll be writing the needed commands on Cypress so that we can set up different containers / container_requests objects to be able to have proper tests. Will do that on this current branch but I think you've enough to be able to start reviewing.

#12 Updated by Stephen Smith 2 months ago

Only thing I noticed is that after clicking a linked UUID (in my case `186304b01836978500373c7fadf127cc+156`), I have to quickly double press back to return to the run. Other than that it looks good to me!

#13 Updated by Lucas Di Pentima 2 months ago

Thanks! The PDH issue isn't a log viewer thing but the way wb2 works when "navigating to" a PDH that resolves to just 1 UUID: instead of showing a listing of all UUIDs that match, it just redirects to the collection panel for that single UUID.

#14 Updated by Lucas Di Pentima about 2 months ago

Rebased and added Cypress tests at arvados-workbench2|d1fa1c8 - developer-tests-workbench2: #622

  • Added some new custom commands to be able to create container requests for future further testing of the process UI.
  • Added a couple of process log tests, including the log filter.

#15 Updated by Lucas Di Pentima about 2 months ago

Updates at arvados-workbench2|563b376 - developer-tests-workbench2: #628

  • Fixed layout issues that made some Cypress tests fail.

#16 Updated by Stephen Smith about 2 months ago

This lgtm!

#17 Updated by Lucas Di Pentima about 2 months ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

Also available in: Atom PDF