Project

General

Profile

Actions

Idea #16516

closed

Run Keepstore on local compute nodes

Added by Peter Amstutz almost 4 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Target version:
-
Story points:
-
Release:
Release relationship:
Auto

Description

  • new API endpoint that returns the keep volumes section of config + credentials
    • only available with a special token that combines container uuid + system root token
  • should run in separate process so that problems like OOM kill can be detected
  • specify number of memory buffers to use
    • need to be able to configure the static / dynamic keepstore instances separately
  • need to include memory that will be used by keepstore in node size request

Related issues

Related to Arvados - Idea #10477: [keepstore] switch s3 driver from goamz to a more actively maintained client libraryResolvedWard Vandewege11/08/2016Actions
Related to Arvados - Feature #16745: [keep-web] Improve performance of S3 APIs using server-side cacheResolvedTom Clegg03/02/2021Actions
Related to Arvados - Feature #16347: crunch-run runs local keepstoreResolvedTom Clegg10/08/2021Actions
Related to Arvados - Feature #17749: [Keep] avoid AWS S3 request limits -- add option to use more prefixes on S3ResolvedTom Clegg09/21/2021Actions
Related to Arvados - Feature #18216: Refactor keepstore to be part of arvados-serverResolvedTom Clegg09/30/2021Actions
Blocked by Arvados - Feature #16513: Get reference Keep performance numbers for Keep-on-S3ResolvedWard Vandewege06/15/2020Actions
Actions

Also available in: Atom PDF