Feature #11184
Updated by Tom Morris over 6 years ago
As an Arvados system administrator, I'd want the ability take advantage of the cool/cold storage classes offered by cloud vendors. This would potentially require the ability to designate storage tier in some way (collection, all collections in a project, etc) as well as a way to migrate between storage classes/locations, [[Keep storage groups]] (previous [[Keep storage tiers]]) tiers]] Keep volumes have storage attributes that they offer. Collections have storage attributes that they desire. Blocks get written to same Keep store server as today. Keep balance moves data to volumes that have the desired attributes and updates the collection to reflect the actual storage class (ie there are separate requested storage class and current storage class). Keep balance needs to track blocks by volume rather than by server as it currently does. If multiple collections have the same block with different storage classes, the block gets stored multiple times, once in each. For the initial iteration, there is no client side support. Microsoft Azure (pricing at 50-500 TB level) * LRS-COOL $0.01/GB/mo, $0.01/10Kops + $0.01GB * LRS-HOT $0.0177/GB/mo, $0.05/10Kops * GRS-COOL $0.02/GB/mo, $0.20/10Kops + $0.01/GB * GRS-HOT $0.0354/GB/mo, $0.10/10Kops * RAGRS-COOL $0.025/GB/mo, $0.20/10Kops + $0.01/GB * RAGRS-HOST $0.0442/GB/mo, $0.10/10Kops Amazon S3 (pricing at 50-500TB level) * Standard - $0.022/GB, $0.004/10Kops (get) * Infrequent Access - $0.0125/GB, $0.01/10Kops (get) * Glacier - $0.004/GB + variable retrieval charge depending on speed Google * Multi-Regional Storage $0.026/GB/mo * Regional Storage $0.02/GB/mo * Nearline Storage $0.01/GB/mo, $0.01/GB retrieval charge * Coldline Storage $0.007, $0.05/GB retrieval charge * Optional bucket versioning