To Be Groomed

Due in about 14 months (01/01/2021) (Sprint start date 12/31/2019)

A fake sprint to hold stories to be groomed


214 issues   (0 closed — 214 open)

To Be Groomed

Arvados Contributor Wiki

This wiki has information related to the open source project. It's a work in progress, so come back often for updates and feel free to become a contributor.

Platform Overview

Release Notes

Please see our blog

Key Components

Using Arvados

Contributing and hacking



Issues by










Related issues
# Subject Story Points
Bug #15836 Escape / convert forward slashes in collection names accessed via WebDAV
Story #15835 [docs] Document Prometheus metrics available from Arvados
Story #15834 [docs] Document admin user management features of workbench2
Story #15823 [crunch-dispatch-cloud] [docs] Add crunch-dispatch-cloud management APIs to doc site
Arvados Workbench 2 - Feature #15768 [Workbench 2] Support multi-select operations
Arvados Workbench 2 - Feature #15012 [WB2] Add /all_processes page
Bug #15819 [packages] workbench dev package not being built
Story #15795 [API] Accept configured SystemRootToken without doing a database lookup
Arvados Workbench 2 - Story #15780 WB2: starting process should also show in which project the process would be started
Bug #15773 [docs] Properly document user/group management using command line
Arvados Workbench 2 - Bug #15764 [Workbench 2] Right clicking on a node in the navigation tree should select it
Arvados Workbench 2 - Bug #15763 [Workbench 2] Owner field in Details pane should contain name and should link to correct location
Feature #15758 [keep-balance] [keepstore] Report blocks/bytes in trash
Story #15581 [Workbench2] Improve error handling and reporting
Story #14151 Extend vocabulary support for properties to support strong identifiers and multiple labels
Feature #15681 wb2 workflows run in "Workflow runs" project by default
Story #15697 [doc] explain lifecycle of Keep blocks, and how it affects storage backend usage/cost
Bug #15692 [CWL] test_with_arvbox integration tests use tests inside image
Bug #15579 Staging a large number of files with "loadListing: no_listing" still takes more than 30 mins
Story #15292 Compute and store per-file checksums
Story #15335 [Workbench2] Integration testing
Story #15336 [Workbench2] Logging
Story #15580 [CWL] Register workflow and run from git repo
Story #15540 Paginate crunchstat-summary HTML output
Story #15406 [cwl] Implement (or just document?) safer bash defaults for CommandLineTools
Story #15436 can run offline
Story #15441 mounts->tmp->device_type is unimplemented, should not be documented as if it were
Bug #15428 [doc] our documentation still refers to outdated config.yml parameters
Story #15382 Synthetic database generator tool
Feature #15370 [arvados-dispatch-cloud] loopback driver
Story #15381 can populate/use local package cache directories
Story #15339 ExpressionTool can be explicitly run under a CR
Story #15364 Better CWL onboarding
Story #15333 [Epic] Workbench2 has reached feature parity with Workbench, so that that latter can be deprecated
Bug #15321 [workbench2] Test run, package building & deploy integration
Feature #15320 [API] add integration test for new user signup
Arvados Workbench 2 - Feature #15353 [Data operations] Chooser supports browing/picking projects/collections on federated clusters
Story #14964 Port arvados-pam to Python 3
Bug #15183 [Workbench] No errors reported when file is missing a block
Bug #15177 Internal Server Error:
Feature #15204 Federation diagnostic tools/APIs
Feature #15279 [cwl] expressions in keep_cache
Feature #15250 [cwl] a-c-r set TTL on final output collection
Feature #15249 [cwl] a-c-r supports --js-console
Bug #15118 [keepstore] Return 5xx (not 4xx) if block is not found due to transient backend device failure
Feature #15135 [Crunch2] Post-run hooks
Story #15259 Invalidate tokens federation-wide on when invalidated on home cluster
Arvados Workbench 2 - Story #15217 [Workbench 2] Add support for setting trashIntermediate value on workflows
Story #15206 Dockerize unit tests
Feature #15125 [keep-balance] [keepstore] Procedure to halt/reverse/investigate a suspected data loss incident
Story #9046 [Crunch2] Can issue container request to pull and import Docker images
Bug #14967 [SSO] fails to start with Ruby 2.4 (future of the SSO server)
Story #14989 [SSO] Upgrade to Rails 5
Feature #15082 [controller] [all services] Publish entire config including secrets
Story #15070 Update search API to support OR queries across text and vocabulary IDs
Story #15068 Migrate existing vocabulary property data to new ID scheme
Story #15015 replace epydoc
Bug #15046 [documentation] config.yml needs to be documented on
Bug #15045 [arvados-cloud-dispatch] commit 115cbd6482632c47fdcbbbe4abc9543e7e8e30ec breaks API host loading
Bug #15043 [SDKs] [R] Missing stylesheet in R SDK docs
Bug #15018 [Workbench2] Show the version number in workbench2
Story #13648 [Epic] Use one cluster configuration file for all components
Story #15502 [API] API call or script to show why jobs didn't reuse
Feature #14974 [Workbench2] Linking to collections in input object display, recording uuids of input collections
Bug #14872 [Epic] Kubernetes support
Bug #14983 [API] searching for "test" on c97qk takes a long time
Story #14923 [CWL] Reduce memory footprint by using webdav to get file/dir listings in collections (instead of Python SDK)
Feature #15025 [arvados-dispatch-cloud] GCE driver (Google Compute Engine)
Support #14906 [SDK] Maintain Arvados SDKs in Bioconda
Feature #14706 [Crunch2] Retain references + permissions to earlier containers when retrying a container request
Story #14284 Send real time container logs to a suitable log distribution system (instead of adding rows to the postgres logs table)
Story #14824 postgresql 10.x compatibility
Story #13921 Log stats for the entire node lifecycle
Feature #13557 Copying a container request should link to the original container request
Feature #13643 [CWL] Requirement for storage classes of data
Story #14997 [crunchstat-summary] Support multiple data series per graph (e.g. TX/RX, user/sys CPU, etc)
Bug #14650 [arvados-cwl-runner] Inaccessible collection causes long traceback
Bug #14527 Workbench/API Authentication issue
Bug #14488 [keepstore] Do not try to trash blocks that were already trashed in previous sweep
Feature #14457 [api] send e-mail notification to user when their arvados account is activated
Bug #14428 [api] after deleting a container object, it is impossible to fix/delete the corresponding container request
Story #14803 CWL v1.1 specifies directory listings are ordered
Bug #14414 [keepstore] ManagementToken is not documented in the --help output
Bug #14405 [API] Avoid long-running transactions when sweeping trashed objects
Feature #14257 [controller] refresh manifest API
Feature #14200 [API] Reduce privilege exposure via API tokens in multi-cluster workflows
Feature #11442 [CWL] Resolve symbolic names to collections
Feature #14168 [Running a workflow] Input presets
Feature #14030 [CWL] Use streamable flag as a hint to use keep output
Feature #14021 [crunch-dispatch-slurm] option to set job priority directly instead of using nice values
Feature #14018 [CWL] arvados-cwl-runner should support tagging container requests using properties
Feature #14016 [API] Container request can provide existing collection UUID that will accept CR output
Story #14004 [Controller] Keep database in tune
Story #14001 [Spike] [Controller] Port "update workflow" API to controller
Bug #13998 [keepstore] Cached block fetch for federation
Story #13997 [Controller] Port "collection update" API from Rails
Story #13995 [Controller] Port "collection update" functional tests from Rails API to controller test suite
Bug #13991 crunch-dispatch-slurm does not warn when slurm MaxJobCount reached
Bug #13982 crunch-dispatch-slurm / Go SDK Dispatcher not able to "let go" of deleted containers
Bug #13981 Make keep-web documentation available on
Bug #13972 Listing collections by PDH and name can be very slow
Bug #13960 Workbench takes new users to inexistant tutorial
Story #13954 [API server] Implement request throttling
Story #13925 Default keep cache scales with machine size
Bug #13916 Discovery document inconsistencies
Story #13792 Document strategies on how to use existing resources to save user preferences
Bug #13813 [CLI] arv traceback on simple timeout
Bug #13804 [Node manager] shutting down nodes when there is a wishlist for same size nodes
Story #13794 Build Docker images without requiring Docker on client
Bug #13763 [API] groups#contents response should obey caller-provided order (not sort on type first)
Bug #13956 [keep-web] (webdav) Support COPY and MOVE between collections
Story #13384 [Workbench] Search users by email address and shell account
Story #13760 Provide more information to SLURM to make scheduling decisions on HPC
Support #13716 [CWL] Developer docs for arvados-cwl-runner
Feature #13675 [Composer/Workbench] Import of public git repo to a new Arvados repository
Bug #13644 [STORAGE-CLASES] Report the storage classes used by a pipeline
Bug #13636 crunch-run takes a very long time for CWL steps with large numbers of File inputs - could use a new kind of mounts entry to address this
Feature #6865 [Documentation] Higher Level Python SDK Reference Page
Bug #13418 Python SDK must have docstrings for all public interfaces
Bug #12684 [Python SDK] Retry on HTTP 5xx errors
Bug #13607 [arv-mount] avoid excessive memory consumption of idle arv-mounts
Bug #13565 [cwl] a-c-r should log useful information on failure even with --quiet
Bug #13542 No timeout is specified in A.C.R or as a default in the python sdk
Story #13494 [Workbench2] View/copy/expunge previous versions of a collection
Story #13390 Document API in API discovery document
Story #12703 [Workbench] Self serve account merge
Feature #13389 [Epic] Easy cluster install
Feature #13388 [Epic] Zero-downtime upgrade
Bug #13368 [API] Add "authorizations" table
Story #12666 --ignore-docker-for-reuse not passed to crunch v2 jobs started via --submit --no-wait
Story #13326 As a data provider, I would like the ability to provide metered access to my data on Arvados, including start/end dates, bandwidth caps, and % of file accessed
Story #13325 As a CWL learner, I would like to be able to run CWL workflows easily on
Story #12961 Support Bitbucket git repo access from Composer
Story #13110 [WebDAV] WebDAV performance enhancements
Story #9653 [keepstore] Limit entire process to configured amount or % of system memory (replace -max-buffers=X)
Bug #13162 [SDKs] Remove arvadosclient and keepclient libraries
Bug #13144 [SDKs] Wean all components off old arvadosclient and keepclient libraries
Bug #13102 containers are not reused unless runtime constraints (including RAM) match exactly
Bug #13094 Fix misleading crunch file upload messages
Story #13074 Notify Composer user when file changed
Feature #13062 [SDK] Reduce collection class memory footprint
Bug #13005 [Crunch2] All stdout gets the same timestamp and other logging problems
Bug #11625 [Workbench] Ensure that running processes are visible on the dashboard
Feature #12517 [Workbench] page with a view of all the top level container requests
Feature #13126 [keep] Investigate using signed URLs to delegate access to cloud buckets
Bug #12854 [crunchstat-summary] broken option for skip child jobs?
Bug #12794 [Node Manager] Behave smarter in environments where scratch space can be arbitrarily sized, like GCP and AWS
Bug #12791 [API] fix race between arrival of trash time and next sweep
Support #12696 install docs for keepproxy are out of date
Feature #12692 [keep-web] Accept /t=token/c=collection/path (currently only /c=collection/t=token/path)
Story #12704 [API] Associate multiple identity_url from SSO with same user
Feature #12090 Collections/data access API
Story #12571 arv-mount by_tag directory support for properties
Story #12527 [Composer] Refactor to separate vendor-specific pieces and all them to be replaced
Story #12288 Status endpoint aggregator
Bug #12178 Activating an account (setting is_active from false to true) does not email the new user
Story #14888 [CWL] running expression tool on arvados doesn't return proper output
Story #11770 [Python SDK] Implement support for universal newline mode in Collections API
Story #11748 Standard convention for Nagios-compatible health check scripts kept alongside source for services
Story #11724 Pull Docker images without requiring Docker on client
Feature #11183 [Deployment] Use consul to report health of services/components
Feature #10767 [API] [arvados-ws] remove all websockets and puma related code from the api server codebase
Story #10525 Integrate CI::Reporter to provide Ruby test output in JUnit XML format
Bug #10458 Document crunchstat-summary
Bug #10390 crunchstat-summary should explain why "(no report generated)" when possible
Bug #10298 [Crunch2] [API] Accept collection UUID or PDH as container_image in a container request
Story #9425 [API?/Workbench] Search for a link tag value returns linked objects
Story #8573 crunchstat-summary should calculate averages of all metrics
Bug #7479 [Node Manager] Aggressively throttles node creation requests when the cloud region is full
Feature #9136 [Python SDK] Add a Collection method to efficiently concatenate files
Bug #12198 Copying a project with a docker image doesn't copy the docker metadata
Feature #5214 [SDKs] Use sphinx to render for Python SDK docs
Story #14467 [SDK] Merge Go ArvadosClient and arvados.Client and migrate to arvados.Client