Story #2638

Workbench displays garbage collection histogram

Added by Misha Zatsman over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Misha Zatsman
Category:
-
Start date:
04/16/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.50 h)
Story points:
1.0

Description

Display the garbage collection (block age vs free space) histogram output by the datamanager.

cache age vs disk histogram.png (39.4 KB) cache age vs disk histogram.png Screenshot of graph Misha Zatsman, 05/07/2014 12:34 PM

Subtasks

Task #2717: Figure out how to make histogram graph in flot.ResolvedMisha Zatsman

Task #2718: Add histogram to workbenchResolvedMisha Zatsman

Task #2748: Review 2638-add-cache-age-disk-usage-histogram-to-workbench ResolvedTim Pierce

Associated revisions

Revision 1ca9c477
Added by Misha Zatsman over 7 years ago

Merge branch '2638-add-cache-age-disk-usage-histogram-to-workbench' closes #2638

History

#1 Updated by Misha Zatsman over 7 years ago

  • Story points set to 1.0

#2 Updated by Misha Zatsman over 7 years ago

  • Subject changed from Workbench displays grabage collection histogram to Workbench displays garbage collection histogram

#3 Updated by Tom Clegg over 7 years ago

  • Status changed from New to In Progress

#4 Updated by Tim Pierce over 7 years ago

Review @ 7fb83a33:

  • apps/workbench/app/controllers/keep_disks_controller.rb
    • Tricky to understand what's going on, because it looks like the code for generating block-age-free-space-histogram log events is not there yet!
    • Add comments explaining the structure of the @cache_age_histogram array, what it's for and how it's produced: i.e. an array of objects suitable for passing to Morris.Area when renderHistogram is called, and briefly describe the elements of the source histogram array and the objects that it returns.
    • Rename @histogram_date to @histogram_date_ms for clarity about units

#5 Updated by Misha Zatsman over 7 years ago

Tim Pierce wrote:

Review @ 7fb83a33:

  • apps/workbench/app/controllers/keep_disks_controller.rb
    • Tricky to understand what's going on, because it looks like the code for generating block-age-free-space-histogram log events is not there yet!

Sorry, you're right, it's in datamanager on a branch of mine. You can see the running code at:
https://shell.arvados:3001/keep_disks

  • Add comments explaining the structure of the @cache_age_histogram array, what it's for and how it's produced: i.e. an array of objects suitable for passing to Morris.Area when renderHistogram is called, and briefly describe the elements of the source histogram array and the objects that it returns.

Added tons of comments. Sorry, I didn't really stop to think about how confusing this would be for another person to read. Part of the problem is the histogram we had in mind changed between when the underlying data was produced and written to the logs and when we came up with something that seems more usable. Eventually I'll go back and clean up the generating code to output what we want, but I wanted to get some eyeballs on the chart first.

  • Rename @histogram_date to @histogram_date_ms for clarity about units

done, thanks

#6 Updated by Misha Zatsman over 7 years ago

Also I added a screenshot to make things clearer if you have trouble getting to the server.

#7 Updated by Tim Pierce over 7 years ago

Thanks for the comments! This helps a lot.

It took me a bit to feel confident that I understand how this code is using @cache_age_histogram, but I think it makes sense to me now, and if we expect to go back and make the code to generate histogram data a little more sensible then I'm not fussed about it. LGTM.

#8 Updated by Anonymous over 7 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:1ca9c477ffa476305a920a0df7ae1e8d5b69c8b2.

Also available in: Atom PDF