Keep » History » Version 1
Tom Clegg, 04/10/2013 04:23 PM
| 1 | 1 | Tom Clegg | h1. Keep |
|---|---|---|---|
| 2 | 1 | Tom Clegg | |
| 3 | 1 | Tom Clegg | The Keep storage system consists of data block read/write services, SDKs, and management agents. |
| 4 | 1 | Tom Clegg | |
| 5 | 1 | Tom Clegg | The responsibilities of the Keep service are: |
| 6 | 1 | Tom Clegg | |
| 7 | 1 | Tom Clegg | * Write data blocks |
| 8 | 1 | Tom Clegg | * When writing: ensure data integrity by comparing client-supplied MD5 digest to client-supplied data |
| 9 | 1 | Tom Clegg | * Read data blocks (subject to permission, which is determined by the system/metadata DB) |
| 10 | 1 | Tom Clegg | * Send read/write/error event logs to management agents |
| 11 | 1 | Tom Clegg | |
| 12 | 1 | Tom Clegg | The responsibilities of the SDK are: |
| 13 | 1 | Tom Clegg | |
| 14 | 1 | Tom Clegg | * When writing: split data into ≤64 MiB chunks |
| 15 | 1 | Tom Clegg | * When writing: encode directory trees as manifests |
| 16 | 1 | Tom Clegg | * When writing: write data to the desired number of nodes to achieve storage redundancy |
| 17 | 1 | Tom Clegg | * After writing: register a collection with Arvados |
| 18 | 1 | Tom Clegg | * When reading: parse manifests |
| 19 | 1 | Tom Clegg | * When reading: verify data integrity by comparing locator to MD5 digest of retrieved data |
