Arvados

Functional requirements¶

• The Go SDK Keep client will talk to all accessible Keep services, regardless of their service type.
• The client will send the X-Keep-Desired-Replication header in all PUT requests, and respect the X-Keep-Replicas-Stored header in the response, regardless of service type
• The client only has one writer at a time writing to non-disk services, to avoid sending redundant requests (e.g., if it wants to 2 replicas of a block, it doesn't send the block to two gateways that will each replicate the block 2+ times).
• Use "disk" timeouts for all services that don't have type "proxy". (Choosing timeouts based on type is rather wrong anyway -- but until we have a better approach, "api told us to use a proxy" is still the best readily available indication that the network between us and the keep service is slow.)

Implementation¶

We don't get the "max replicas service X is able to store" information from the API server, but we'll move the code in that direction.

• KeepClient gets a new attribute, replicasPerService int.
• DiscoverKeepServers sets replicasPerService to 1 if all non-ReadOnly services have SvcType=="disk", otherwise 0. (Note the current code forgets to reset Using_proxy to false if the API server stops giving out proxies between one call and the next. We should at least avoid making that mistake again -- and possibly fix that Using_proxy bug, since it's trivial.)
• DiscoverKeepServers adds all services to localRoots, not just disk and proxy. The only reason to look at SvcType is to choose which this.setClientSettings* function to use.

  -        switch service.SvcType {
  -        case "disk":
  -            localRoots[service.Uuid] = url
  -        case "proxy":
  -            localRoots[service.Uuid] = url
  -            this.Using_proxy = true
  -        }
  +        localRoots[service.Uuid] = url
  +        if service.SvcType == "proxy" {
  +            this.Using_proxy = true
  +        }
  

• putReplicas reduces the number of concurrent writer threads according to replicasPerService. The easiest way to do this is probably something like:

  -       for active < remaining_replicas {
  +       replicasPerThread := this.replicasPerService
  +       if replicasPerThread < 1 {
  +               // unlimited or unknown
  +               replicasPerThread = remaining_replicas
  +       }
  +
  +       for active * replicasPerThread < remaining_replicas {
  

• uploadToKeepServer should ignore this.Using_proxy here:

  -    if this.Using_proxy {
           req.Header.Add(X_Keep_Desired_Replicas, fmt.Sprint(this.Want_replicas))
  -    }