Project

General

Profile

Actions

Feature #17993

open

[deduplication-report] supports storage classes

Added by Ward Vandewege over 2 years ago. Updated 27 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

Output could look something like this:

wardv@shell:~$ arv collection list --order 'file_size_total desc' --limit 10 | jq -r '.items[] | [.portable_data_hash,.uuid] |@csv' |sed -e 's/"//g'|tr '\n' ' ' |xargs arvados-client deduplication-report

Collection _____-_____-_______________: pdh ________________________________+167305537; nominal size 22808792764220 (21 TiB); file count 4642651; storage classes fast, big
Collection _____-_____-_______________: pdh ________________________________+167305537; nominal size 22808792764220 (21 TiB); file count 4642651; storage classes big
Collection _____-_____-_______________: pdh ________________________________+137315151; nominal size 22011136341600 (20 TiB); file count 4076624; storage classes fast, big
Collection _____-_____-_______________: pdh ________________________________+137315151; nominal size 22011136341600 (20 TiB); file count 4076624; storage classes big
Collection _____-_____-_______________: pdh ________________________________+23118809; nominal size 17113218001658 (16 TiB); file count 135751; storage classes fast
Collection _____-_____-_______________: pdh ________________________________+16823121; nominal size 17109593452137 (16 TiB); file count 67876; storage classes fast
Collection _____-_____-_______________: pdh ________________________________+16812571; nominal size 17093959566454 (16 TiB); file count 67876; storage classes fast, big
Collection _____-_____-_______________: pdh ________________________________+9616302; nominal size 15334419799731 (14 TiB); file count 220; storage classes fast, big
Collection _____-_____-_______________: pdh ________________________________+125109120; nominal size 13268828856746 (12 TiB); file count 3995974; storage classes fast, big
Collection _____-_____-_______________: pdh ________________________________+125109120; nominal size 13268828800485 (12 TiB); file count 3995974; storage classes fast, big

Collections:                              10

Storage class: fast

Nominal size of stored data: 182828706688851 bytes (166 TiB)
Actual size of stored data:   93084713928994 bytes (85 TiB)
Saved by Keep deduplication:  89743992759857 bytes (82 TiB)

Storage class: big

Nominal size of stored data: 182828706688851 bytes (166 TiB)
Actual size of stored data:   93084713928994 bytes (85 TiB)
Saved by Keep deduplication:  89743992759857 bytes (82 TiB)

Related issues

Related to Arvados - Idea #17697: Design for reporting tools to determine what data is on multiple storage classes.ResolvedWard VandewegeActions
Related to Arvados - Feature #17994: [api] storage class fields should be supported in filtersResolvedTom Clegg08/27/2021Actions
Actions #1

Updated by Ward Vandewege over 2 years ago

  • Description updated (diff)
Actions #2

Updated by Ward Vandewege over 2 years ago

  • Related to Idea #17697: Design for reporting tools to determine what data is on multiple storage classes. added
Actions #3

Updated by Ward Vandewege over 2 years ago

  • Related to Feature #17994: [api] storage class fields should be supported in filters added
Actions #4

Updated by Ward Vandewege over 2 years ago

Actions #5

Updated by Peter Amstutz over 2 years ago

  • Target version set to 2021-11-24 sprint
Actions #6

Updated by Peter Amstutz over 2 years ago

Actions #7

Updated by Peter Amstutz over 2 years ago

  • Target version deleted (2021-11-24 sprint)
Actions #8

Updated by Peter Amstutz over 2 years ago

  • Target version set to 2021-12-08 sprint
Actions #9

Updated by Peter Amstutz over 2 years ago

  • Target version deleted (2021-12-08 sprint)
Actions #10

Updated by Peter Amstutz about 1 year ago

  • Release set to 60
Actions #11

Updated by Peter Amstutz 27 days ago

  • Target version set to Future
Actions

Also available in: Atom PDF