Feature #17967
closed
Prioritize reads from different storage classes
Description
Add a list of storage classes
StorageClasses:
default: {Priority: 100, DefaultWrite: true}
foo: {Priority: 50, DefaultWrite: false}
bar: {Priority: 20, DefaultWrite: false}
Higher priority has higher precedence.
Validate the storage classes of each volume list against the main list.
Also includes adding an exported set of possibly multiple "default" storage classes.
Use priority to determine order to check volumes for blocks.
Updated by Peter Amstutz over 3 years ago
- Related to Idea #16107: Storage classes added
Updated by Peter Amstutz over 3 years ago
- Blocks Feature #17696: Exported config has default storage class(es), SDKs use the configured default storage class if not overridden added
Updated by Tom Clegg over 3 years ago
It seemed reasonable to add the whole StorageClasses config section (with Default for #17696 as well as Priority for this issue). Then I forgot #17696 was already assigned to someone else and implemented the Default behavior as well.
17967-storage-classes-config @ 13b280ea288fd08e8c77300b087037635d739eaa -- developer-run-tests: #2621
- add config section with default behavior as described in #17696
- try higher-priority volumes (i.e., volumes with higher-priority storage classes) first when reading
- change keepstore behavior to write to the configured default storage classes when the client didn't specify any (previously was "disregard storage classes" in that case) -- I figured this should make the SDK side much easier
- change arv-put behavior to not specify storage classes for newly written collections/blocks if none were specified by the caller (previously was ["default"])
- TODO: update admin/storage-classes.html
Updated by Tom Clegg over 3 years ago
- update docs
Updated by Peter Amstutz over 3 years ago
# If you use multiple storage classes, specify them here, using
# the storage class name as the key (in place of "SAMPLE" in
# this sample entry).
This comment doesn't explain what a storage class is, it should either be extended a bit ("Storage classes enable you to control which volumes should be used to store the blocks of a given collection") and/or link to the documentation.
Need to mention the config changes in doc/admin/upgrading.html.textile.liquid
There's some changes to the API server behavior, perhaps that should get a test?
Updated by Tom Clegg over 3 years ago
- add doc link to config file
- add upgrade note about adding StorageClasses section
- add/update railsapi tests for default storage classes
Updated by Peter Amstutz over 3 years ago
Tom Clegg wrote:
17967-storage-classes-config @ 7860f93d77e8690ba90e839b52e2886a08466773 -- developer-run-tests: #2625
- add doc link to config file
- add upgrade note about adding StorageClasses section
- add/update railsapi tests for default storage classes
This LGTM
Updated by Tom Clegg over 3 years ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|818e9974a935703de29239f97b225ba7505537f8.