Arvados

h1. Proposals and design documents

_Note: these are work in process documents used to facilitate discussion among the engineering team.  They may be of historical interest to learn the thinking when a feature was original developed.  Do not assume any of these are comprehensive, reflect the eventual actual implementation, describe a feature that actually exists, or is any guarantee that a feature will be implemented in the future.  Visit the "documentation site":https://doc.arvados.org for official documentation of fully implemented and supported features._

h2. Current/pending proposals 

* [[Stress tests and benchmarks]]

* [[Multi-pass mode to reduce keep-balance memory footprint]]

* [[Efficient block packing for small WebDAV uploads]]

* [[Workbench responsiveness]]

* [[Credential storage]]

* [[Objects as pseudo-blocks in Keep]]

* [[FUSE v2]]

* [[Concurrent writes to a single collection]]

* [[Admin CLI for managing dispatcher / cloud VMs]]

* [[Glacier]]

* [[Auto-discovery]]

* [[Changing Keep hash algorithm]]

* [[Service containers]]

h2. Design documents

* [[Groups, Projects, Ownership and Permissions Specification]]

* [[Workbench authentication process]] implementation notes

* [[Keep Design Doc]]

* [[Authentication]]

* [[Containers API]]

* [[Container dispatch]]

* [[Audit logs]] and other events/logs

* [[Crunch2 run]]

* [[Crunch runner]]

* [[Keep-web flow]]

* [[Websocket server]]

h2. Archived

* [[WebDAV performance metrics]]

* [[Optimal runtime constraints API]]

* [[Scaling things]]

* [[Efficient live access to container logs]]

* [[Diagnostics command]]

* [[Accessing Keep using S3 protocol]]

* [[All-in-one package]]

* [[Accepting OpenID access tokens]]

* [[API Historical/Forcasting data for CR]]

* [[Multi-cluster user database]]

* [[Native login implementation]]

* [[Index of files in collections]]

* [[Browsing shared projects]]

* [[Controller architecture transition]]

* [[Controller architecture]]

* [[Cluster-wide favorites]]

* [[Golang API server]]

* [[Dispatching containers to cloud VMs]] ([[Fixing cloud scheduling]])

* [[Build docker images as part of a workflow]]

* [[Use a docker image from a remote registry]]

* [[Container status / outcome reporting]]

* [[API token format]]

* [[Source tree layout]]

* [[Installing controller service]]

* [[Collection version history]]

* [[Declarative test framework]]

* [[Container secret mounts]]

* [[Cluster configuration]]

* [[Vault]]

* [[Message queue]]

* [[Keep storage groups]]

* [[Keep storage tiers]]

* [[Keep service hints]]

* [[Federation]]

** [[Federation use cases]]

** [[Federated identity]]

** [[Distributed workflows]]

** [[Routing multi cluster requests]]

* [[Websocket v0 shim]]

* [[Websocket server]]

* [[Events API]]

* [[S3 bucket volume implementation]]

* [[Crunch2 installation]]

* [[Jobs API]], [[Crunch1-in-Crunch2]]

* [[Content addressed records]]

* [[File splits]]

* [[Separating files from collections]]

* [[Collection API - Performance enhancements]]

* [[Performance profiling]]

* [[Collection update API]]

* [[Git strategy for pipeline development]]

* [[CLI client]]

* [[Node manager policy matrix]]

* [[Expiring collections]]

* [[Docker security]]

* [[Metadata]]

* [[Package versioning]]

* [[Consul]]

* [[Crunch v2 cloud scheduling]]

* [[Keep cache policy]]

* [[Privileged containers]]

* [[Keep Proxy Specification]]

* [[Collections as regular Arvados objects]]

* [[Reusable tasks]]

* [[Everything is a task]]

* [[Tasks as jobs]]

* [[Pipelines as jobs]]

* [[Better git integration]]

* [[Writable FUSE mount]]

* [[Data Manager Design Doc]]