Arvados 1.2

This is the Q3 2018 quarterly release of the Arvados open source project.


Release Backlog

# Tracker Subject Story Points

Closed Sprints

2018-04-25 Sprint


# Tracker Subject Story Points
13385 Bug login-sync test broken
12518 Idea Roche 2017Q4 deliverables
13108 Bug arvados-cwl-runner dispatches container requests very slowly
11876 Idea [R SDK] Create a Bioconductor/R SDK
12320 Feature Access Arvados projects under the "by_id" directory in keep mount 0.5
13076 Idea R SDK autogenerate API to access all Arvados object types
11583 Bug [crunch-run] Fix excessive memory use

2018-04-11 Sprint


# Tracker Subject Story Points
13324 Bug Python packaging broken
13301 Bug cwl ramMin expression doesn't work anymore
13201 Bug arvados-cwl-runner workflow checks are expensive and repeated twice when run with --submit
12085 Idea Add monitoring/alarm for failed/slow job dispatch & excess idle nodes 1.0
13168 Bug [API] state/priority-change triggers should not change container request modified_by_user_uuid to root
13113 Bug arvados-cwl-runner error while getting output object
13100 Bug [crunch-run] Replace custom manifest-writing code with collectionFS 2.0
12414 Idea [API] Contents of trashed projects are actually deleted once delete_at is past 1.0
13105 Bug Missing federation config makes the multi-site search's session manager fail to log in

2018-05-09 Sprint


# Tracker Subject Story Points
13051 Idea Spike - Investigate/prototype AWS spot instance support in libcloud 2.0
13111 Idea [WebDAV] Support browsing of project hierarchies 1.0
12626 Feature [API] Merge user accounts (redirect=true case) 3.0
13165 Bug arvados-cwl-runner on a large workflow results in long (45s+) api server database queries
13435 Bug a-c-r and python SDK ruamel.yaml versioning
13383 Feature [keepstore] configurable number of trash workers 0.5
13377 Bug keep 401s
13365 Bug default secondaryFiles not being found
13338 Bug [workbench] when running a workflow from workbench, it should create its containers in that project
12167 Feature [SDKs] [API] [keepstore] [keepproxy] Facilitate request tracing for all services 1.0
13282 Idea Create sysadmin doc section and move all relevant wiki pages
13120 Bug arvados-cwl-runner does not pass --debug flag to RunnerContainer (when running with --submit and --no-wait)

2018-05-23 Sprint


# Tracker Subject Story Points
12995 Idea [Workbench] Allow user to add a new Google account to their Arvados account 3.0
13453 Bug Fix R SDK incorrect REST call
13430 Idea [arv-put] [Python] Allow caller to specify storage classes when writing data to Keep 2.0
13446 Idea [keepstore] secure keepstore traffic with TLS 1.0
9918 Bug keep-balance fails with "Malformed index line" error
12737 Bug Update ruby Gemfile/Gemfile.lock files to address stale dependencies & known vulnerabilities
12708 Idea [keep-balance] Move blocks to satisfy storage_classes_desired 0.5
13455 Bug [keepproxy] Test suite sometimes hangs in go 1.10.1
13473 Bug [c-d-s] jobs are stuck in queue
12256 Bug [CWL] default local directories do not work

2018-06-06 Sprint


# Tracker Subject Story Points
11162 Idea [GA4GH] Support workflow execution API
13500 Bug crunch-dispatch-slurm PG::TRDeadlockDetected: ERROR: deadlock detected
13517 Bug keepstore memory usage appears to not be related to MaxBuffers
12061 Bug [Node manager] Fix flaky integration tests
13516 Bug run-arvados-cwl-conformance-tests failing
13431 Feature [keepproxy] [GoSDK] Propagate desired storage classes in PUT request headers 1.0
13559 Bug ciso8601 2.0 API change breaks tests
13513 Bug [keep-balance] hang on ComputeChangeSets

2018-06-20 Sprint


# Tracker Subject Story Points
13579 Idea Document storage classes for sysadmin 1.0
13638 Bug Fix incorrect python-arvados-cwl-runner package version
13580 Idea Document storage classes for users/bioinformaticians
13164 Bug [API] dispatch sometimes tries to run cancelled containers at the expense of pending containers
13581 Bug Report node not satisfiable users to the user
13429 Idea [API] [arvados-cwl-runner] Save workflow outputs to desired storage classes 1.0
13617 Bug [a-n-m] labels in instances in AWS are confusing
13407 Bug [keep-balance] Take keepstore-reported volume replication level into account 1.0
11907 Bug [CWL] a-c-r uploader out of order
7478 Idea [Node Manager] Creates compute nodes using AWS spot instances 3.0
13594 Bug PG::TRDeadlockDetected when running cwl tests in parallel
12960 Idea [API] Migrate remaining "properties" fields to jsonb 1.0
13562 Feature [API] Add JSONB properties field on groups 0.5
13491 Bug arvbox deadlocks on parallel usage
13480 Bug [a-n-m] if there is 1 VM with no tags, a-n-m doesn't spin up any node.
13427 Bug [keep-balance] Handle volumes that are mounted simultaneously by multiple servers 1.0

2018-07-03 Sprint


Sprint ends early due to USA July 4 holiday

# Tracker Subject Story Points
13613 Bug [R SDK] document minimum version of R required (and check at runtime) 1.0
13596 Bug R SDK documentation uses incorrect function names
13547 Bug [python sdk] does not respect ARVADOS_API_HOST_INSECURE when speaking to keep(-proxy) via SSL
13219 Idea Allow user to specify time limit for submitted jobs 3.0
11594 Bug [API] [Workbench] set a default config.log_level
13706 Bug [API] [psql] c97qk dik full because a statement in postgres has some type issues
13569 Idea Add units tera / giga / mega / kilo for configurations 1.0
13674 Bug [crunch-dispatch-slurm] Use hash instead of array for InstanceTypes config
13649 Bug c-d-s doesn't request a preemptible instance when it should
13650 Feature [DOC] document arvados-kubernetes
13543 Bug [Workbench][API Server] Version reporting broken
13255 Idea Provide account activation configuration options for federated logins 2.0
13627 Idea Update a-c-r to newest cwltool 0.5

2018-07-18 Sprint


# Tracker Subject Story Points
13799 Idea Organize manual install docs into sections
13803 Bug Big manifest produces NoMemoryError on API server
13780 Bug Update sprockets to ~> 2.12.5 for security vulnerability fix
13767 Bug ReserveExtraRam missing from c-d-s install docs
13330 Bug Clean up container input collection naming and properties
13692 Bug [arvados-cwl-runner] Improve error message when environment variables missing
13681 Bug [CWL] Cannot set submit-runner-ram when running from Workbench 1.0
13668 Bug [VERSION] add a human readable version as Default 1.0
12983 Idea crunch-run does not notice when containerd dies 1.0
13501 Idea [arvados-cwl-runner] [arv-keepdocker] Use subprocess32 instead of subprocess 0.5

2018-08-01 Sprint


# Tracker Subject Story Points
13497 Idea [API] Initial "arvados-controller" server that proxies API endpoints to Rails server 0.5
13910 Bug [GEM] gem install arvados arvados-cli pulls in two versions of the arvados gem, one of which is old
13920 Bug arvados-cli fails with ruby versions >= 2.3.7
13570 Bug [Documentation] add how much extra overhead is added for RAM requirement when spinning up a compute node
13690 Bug [Composer] Document install & use of Composer 1.0
13892 Bug [arvados-cwl-runner] Include a deprecation warning if using jobs API 0.5
12444 Feature Compute nodes monitor the tmpdir space over time 1.0
13791 Bug Document existing metrics endpoints for Keep store server and Node manager 1.0
13879 Bug arvbox support for arvados-server
13868 Bug [Node manager] Gets into trouble if nodes don't have arvados_node_size tag
13851 Bug Workbench multi-site search bug
13546 Bug [Crunch1] Sanity Check must have a timeout
13788 Bug crunch-dispatch-slurm fatal error: concurrent map writes

2018-08-15 Sprint


# Tracker Subject Story Points
14028 Bug [Workbench] [security] should not render user-provided script tags in descriptions
13931 Bug [CWL] size field not accessible on arvados
13399 Bug [crunch-dispatch-slurm] error: Invalid nice value, must be between -10000 and 10000 0.5
13976 Bug arvados-cwl-runner version cannot find default keep references
13849 Bug [CWL] secondaryFiles checks failed in single container
13609 Bug [R SDK] Copying files using the R SDK results in 0 byte files 0.5
12762 Bug [API] fix SafeYAML warning
13939 Bug [Workbench] Pipelines & processes doesn't show all top level container requests
13934 Bug add code comment to avoid reversion of 13788

2018-09-05 Sprint


# Tracker Subject Story Points
14009 Bug container request creation very slow when there are many potentially reusable containers
14008 Bug new index on containers locked_by_uuid
14153 Bug [gem] make google-api-client compatible with ruby 2.3.7
13959 Bug crunch-dispatch-slurm / Go SDK Dispatcher can block indefinitely on d.Arv.List("containers", params, &list)
13198 Idea [Keep-web] Add metrics endpoint 0.5
13340 Bug links for pdhs on e51c5 on container_requests is not working correctly
13146 Idea [API] Endpoint to get projects shared with me 3.0
14020 Idea [keep-web] Instruct browsers to allow webdav headers in cross-origin requests 0.5
13933 Bug crunch-dispatch-slurm / Go SDK Dispatcher performs poorly in the presence of a large backlog
13924 Bug Timeout when getting collection by portable data hash

2018-09-19 Sprint


# Tracker Subject Story Points
14017 Bug a-c-r logs "submitted container [uuid]" but the uuid is actually a container_request
13558 Feature include req_id in rails SQL debug logging
14075 Bug extremely long running database query