A PUT request can specify a storage class:

X-Keep-Storage-Classes: foo, bar

The keepstore should respond with how many replicas it was able to write for each storage class.

Keepstore should go through all volumes it has access to and attempt to record the block enough times to satisfy all requested storage classes. Once a storage class is satisfied it is removed from the list. When all storage classes are satisfied or all volumes have been checked, a response is returned: success if at least one block was written (or already exists) with any of the desired storage classes, and failure if not.

Proposed format:

X-Keep-Storage-Classes-Confirmed: foo=1, bar=1

The client (#17392) is responsible for keeping track of how many replicas have been written for each storage class. If one storage class is satisfied but not another, the satisfied one can be removed from the X-Keep-Storage-Classes list. For the client, the overall write is a success if all storage classes were satisfied, and a failure if any of the storage classes could not be satisfied.

If the keepstore server can satisfy more than one requested storage class with multiple volumes, the block should be written to all of them. If a volume has more than one storage class, it can satisfy multiple storage classes at once.

The X-Keep-Storage-Classes-Confirmed header should return the replicas per storage class. If two storage classes were satisfied then it would return "foo=1, bar=1" even if only a single replica to a single volume was actually written.

"X-Keep-Replicas-Stored" will be the sum of the actual replicas stored (for foo=1,bar=1 this could be 1 or 2 depending on whether foo and bar are the same or separate volumes).