Project

General

Profile

Actions

Bug #7161

closed

[SDKs] PySDK supports any Keep service type, using proxy replication logic for non-disk types

Added by Brett Smith over 8 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
SDKs
Target version:
Story points:
1.0

Description

Functional requirements:

  • The Python SDK Keep client will talk to all accessible Keep services, regardless of service type.
  • The client will send the X-Keep-Desired-Replication header in all PUT requests, and respect the X-Keep-Replicas-Stored header in the response, regardless of service type (currently sends the request header only to proxies).
  • The client only has one thread at a time writing to non-disk services: If there are any non-"disk" services in the "accessible" list, use ThreadLimiter(1).

Future work: Anticipate API server telling us max replication for each service, meanwhile say 1 for disks and 3 for others. Let a thread start writing only when sum(max replication) < copies_todo, i.e., we will need to write more copies even if all currently writing threads succeed with max replication.

TBD: How should KeepClient choose a timeout for non-disk, non-proxy services?


Subtasks 1 (0 open1 closed)

Task #7377: Review branch 7161-support-all-service-typesResolvedRadhika Chippada09/23/2015Actions

Related issues

Related to Arvados - Feature #7159: [Keep] Implement an Azure blob storage volume in keepstoreResolvedTom Clegg08/28/2015Actions
Related to Arvados - Bug #7180: [Keep] [SDKs] Logic about which services to try should not reference service_typeClosed09/01/2015Actions
Actions

Also available in: Atom PDF