Project

General

Profile

Actions

Idea #1968

closed

Monitor disk usage (per user and per site; split by transient/persistent; optionally weighted by #users who want persistent).

Added by Tom Clegg over 10 years ago. Updated almost 10 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Misha Zatsman
Category:
-
Start date:
04/09/2014
Due date:
Story points:
2.0
Release relationship:
Auto

Description

Focus on
  • Showing usage in Workbench
  • Preliminary garbage collection results
Targets
  • Set up a data manager service that generates reports for qr1hi periodically.
  • Deliver reports as JSON in Log objects. (At least while our metadata/event/log systems are being sorted out.)
  • Show on Workbench
    • Workbench "user list" / "group list" pages will show latest usage numbers for each user/group
    • Workbench "show user" / "show group" pages will show last few usage reports for that user/group
  • Garbage collection & replication-checking results -- some attainable subset of...
    • Report statistics (blobs with less/more than required replication)
    • Report hashes of blobs with less than required replication
    • Next candidates for deletion (oldest not mentioned in any wanted collection)
    • Report cache window vs. % free space: "we can get 30% free space by deleting everything >54 days old" (this graph might conceivably show interesting usage patterns)
    • Candidates appear in which unwanted collections (if any)? Which of those collections are currently intact (i.e., will get their first "holes" in next GC run)
    • (?) Report of affected collections for each user (but we'd need "replication=0" instead of "don't want" in order to know this, right?)

Subtasks 7 (0 open7 closed)

Task #2572: Report weighted disk usage based on users' desired replication.ResolvedMisha Zatsman04/09/2014Actions
Task #2571: Read (and store) replication preferences from "want" link properties.ResolvedMisha Zatsman04/09/2014Actions
Task #2590: Report deletion candidates sorted by age.ResolvedMisha Zatsman04/09/2014Actions
Task #2574: Report blocks that are replicated less than the max desired by all persisters.ResolvedMisha Zatsman04/09/2014Actions
Task #2579: Starting logging to the API server,ResolvedMisha Zatsman04/09/2014Actions
Task #2573: Modify reader disk usage numbers to not consider disk replication.ResolvedMisha Zatsman04/09/2014Actions
Task #1757: User can see in Workbench total amount storage space being usedResolved04/16/2014Actions
Actions #1

Updated by Peter Amstutz about 10 years ago

  • Story points set to 2.0
Actions #2

Updated by Ward Vandewege about 10 years ago

  • Target version set to 2014-04-16 Dev tools and data/resource management
Actions #3

Updated by Tom Clegg about 10 years ago

  • Assigned To set to Misha Zatsman
Actions #4

Updated by Tom Clegg about 10 years ago

  • Description updated (diff)
Actions #5

Updated by Tom Clegg about 10 years ago

  • Description updated (diff)
Actions #6

Updated by Tom Clegg about 10 years ago

  • Description updated (diff)
Actions #7

Updated by Tom Clegg about 10 years ago

  • Status changed from New to In Progress
Actions #8

Updated by Tom Clegg about 10 years ago

  • Status changed from In Progress to Closed
Actions

Also available in: Atom PDF