2017-08-02 sprint

Due in 6 days (08/02/2017)

28%

95 issues   (22 closed — 73 open)

Related issues

	Feature #11906: Basic authenticated http health check ("ping") for each system service
	Task #8979: Review branch: 8937-head-request-in-python-keep-client
	Task #9041: Review branch 8937-arv-put-cache-check
	Task #10379: Review 10359-crunchstat-summary-serial
	Task #10381: Review
	Task #11191: Review 11158-fuse-projects
	Task #11392: Review
	Task #11571: Review
	Task #11757: Review
	Task #11759: Review 7475-nodemgr-unsatisfiable-job-comms
	Task #11827: Review
	Task #11879: Review
	Task #11890: Review 10584-fuse-stop-threads
	Task #11894: Review
	Task #11937: Review 11167-wb-remove-arvget
	Task #11963: Review 11948-cwl-conformance-fix
	Task #11978: Review branch 11906-keepstore-ping
	Task #11981: Review branch 11906-api-ping
	Task #11987: Review branch 11906-wb-ping
	Task #11996: Refactor health check handlers into an SDK library
	Task #11998: Review branch 11995-collecion-filetypes
	Task #12000: Review 11906-health-check-lib
	Task #12001: Review
	Task #12003: Review
	Task #12005: Review
	Task #12006: Review
	Task #12021: Review branch 11906-keepproxy-ping
	Task #12040: Review 12037-cwl-v1.0.1
	Task #12041: Review branch 11906-keep-web-ping
	Task #12043: Review branch 11906-arv-git-httpd-ping
	Story #7475: [Node manager] Better communication when job is unsatisfiable
	Story #8937: [SDKs] Write integration test for when arv-put resumes from a cache with expired access tokens
	Story #10717: Deprecate arv-run
	Story #11167: [Workbench] Remove arv-get file download fallback
	Story #11285: Update Pipeline Optimization wiki with CWL/Crunchv2
	Story #11309: [Crunch2] crunchstat-summary --container UUID should summarize container logs
	Story #11545: Create a CWL stress test for node manager
	Story #11995: [Workbench] Support more filetypes for in-browser display
	Story #11997: [Spike] Investigate CI packaging improvements
	Story #12037: [CWL] Pass conformance for v1.0.1

2017-08-19 sprint

Due in 20 days (08/16/2017)

0%

3 issues   (0 closed — 3 open)

Related issues

Story #12032: Allow projects to be deleted (ie placed in the trash can)

Arvados Future Sprints

Due in over 2 years (01/02/2020)

12%

752 issues   (78 closed — 674 open)

Related issues

	Feature #2897: [OPS] Implement new deploy procedure
	Feature #3101: [Tests] Websockets should work out of the box in test/development modes, even when using an ssh tunnel and a self-signed certificate.
	Feature #3125: [Workbench] Add a "docker images" page to Workbench that lists available docker images
	Feature #3223: [Documentation] Tutorial for adding a crunch script wrapper for an existing tool that is already hosted on github.
	Feature #3226: [Workbench] Project chooser dialog should have a "create new project" button.
	Feature #3284: [Workbench] Project show page should have buttons to add objects besides Collections
	Feature #3311: [SDKs] Improve metadata created by arv-put
	Feature #3336: [Docker] use local docker install for additional testing
	Feature #3445: [Workbench] Improve chooser preview for collections, and add previews for objects that don't have any yet (pipeline instances, templates, jobs, projects)
	Feature #3734: [Keep] Keepstore and keepproxy support HTTP Range requests
	Feature #3818: [API] Discovery document schema should include whether field on resource is writable or read-only.
	Feature #3829: [Crunch] Email notification when pipeline finishes, depending on user prefs flag
	Feature #3892: [SDKs] Install man pages for CLI tools (auto generate from the --help content)
	Feature #3928: [Workbench] Reveal log collection ID in pipeline/job summary and job Log tab
	Feature #3964: [SDKs] Command-line diagnostic suite reveals problems on a shell VM (e.g., arv-mount, SDK version skew)
	Feature #3980: [SDKs] CLI tool to view real time logs from running jobs/pipelines, much like the Log tab of the Workbench PipelineInstance#show page
	Feature #4193: [SDK] Make arv keep docker smarter about storing docker layers, thus making uploads of changed images (much) faster
	Feature #4223: [SDKs] arv keep docker --download to get image from Arvados
	Feature #4254: [Tests] Add crunch-job functional tests to the CI suite
	Feature #4302: [Crunch] Pipelines should not fail immediately after one job failure, but continue running as much as possible
	Feature #4343: [SDKs] arv create needs --project-uuid and --name parameters for appropriate classes
	Feature #4349: [SDKs] When editing a template in arv-edit and faced with a parse error, when trying again, it should automatically take you to the line/column of the error
	Feature #4393: [Workbench] Display portable data hash, parent project, file count in Collection tables
	Feature #4404: [API] [SDKs] Include provenance information for uploaded Collections
	Feature #4446: [Workbench] Provide feedback on dashboard to indicate that NodeManager is booting a node.
	Feature #4456: [Workbench] Provide more feedback about when a queued job is likely to start.
	Feature #4502: [SDKs] "arv recent [objectType]" lists recently modified objects
	Feature #4503: [SDKs] Option to watch logs (instead of exiting) when queuing a job or pipeline instance using arv job create, arv pipeline run, etc.
	Feature #4508: [Documentation] Best practices for getting the most out of job re-use
	Feature #4516: [Workbench] Don't show empty projects/collections when using the chooser dialog to select pipelines, files, etc.
	Feature #4530: [Workbench] Show more (earlier) log entries for a running job
	Feature #4531: [Documentation] Set of introductory tutorials each focused on accomplishing a simple task
	Feature #4539: [Workbench] Job detail page should show the job status when the job is not running or has completed running (button that shows 'queued', 'not ready', 'failed', 'complete', etc)
	Feature #4561: [SDKs] Refactor run-command so it can be used as an SDK by scripts in a git tree
	Feature #4579: [Documentation] Run-command docs should remind user how & why to exit non-zero on failure.
	Feature #4642: [Workbench] "Download" / "Save as" button for provenance graph
	Feature #4647: [Workbench] Projects dropdown should have a free text filter box
	Feature #4650: [API] API method and CLI shortcut for refreshing the signatures on some block locators (without creating a collection)
	Feature #4808: [Workbench] "Add data from another project" modal supports regex filtering
	Feature #4823: [SDKs] Good Collection API for Python SDK
	Feature #4835: [Workbench] Log Graph - Feature requests
	Feature #4906: [Documentation/FUSE] Document '.arvados#collection' in shell VM keep mount
	Feature #4935: [Crunch] Choose worker nodes where the desired docker image is already available.
	Feature #5032: [Workbench] User can trash and un-trash items
	Feature #5033: [SDKs] `arv keep docker` should pipe the data directly to Keep instead of store-and-forward, and provide some progress feedback up front in any case.
	Feature #5035: [Documentation] Document the minimum docker image / Dockerfile that can be used to run a job.
	Feature #5036: [arv-mount] Change default mount type from "fuse" to "fuse.arvados"
	Feature #5064: [Crunch] Automatically restart jobs after internal/intermittent errors
	Feature #5097: [SDKs] CLI tools support project paths in addition to uuids
	Feature #5118: [OPS] Use packer to generate docker images for our various services
	Feature #5119: [OPS] Write a wrapper script that calls packer to generate customized docker images for a new installation (with IP ranges, hostnames, etc preseeded)
	Feature #5133: [Crunch] Use uncommitted code in a git work tree to run a dockerized crunch job in a VM (using crunch-job or arv-run-pipeline-instance --run-jobs-here)
	Feature #5138: [SDKs] arv-run-pipeline-instance should accept project and title parameters
	Feature #5139: [SDKs] arv-get should support getting multiple files by shell glob
	Feature #5193: [Workbench] Public project page should encourage visitors to try using Arvados themselves
	Feature #5201: [SDKs] arv-put --description X --properties P should set respective attrs on new collection.
	Feature #5214: [SDKs] Use sphinx to render for Python SDK docs
	Feature #5237: [Workbench] Pipeline re-run dialog should offer docker image choices (latest/same), similar to script version choices
	Feature #5245: [Documentation] Show footer with date last updated
	Feature #5254: [Testing] Make run-tests.sh able to run only unit tests or integration tests (or both)
	Feature #5259: [Tests] Reduce dependency on integration tests
	Feature #5260: [Tests] Implement long-running keepstore services in run-tests, similar to what api got in #3021
	Feature #5284: [Docker] Auto setup as much as possible for new users
	Feature #5296: [Workbench] Extend Sharing tab to collections
	Feature #5297: [Workbench] Enable users to share with any other user
	Feature #5316: [DRAFT] [Workbench] Browse change history for records
	Feature #5375: [workbench] Enable in-browser preview for text files in collections
	Feature #5382: [Documentation] arvados VM simulator, demonstrating how it looks to use Arvados from CLI
	Feature #5384: [DRAFT] [Workbench] Improve display of pipelines and jobs
	Feature #5387: [DRAFT] [Workbench] Improve provenance/used-by graphs so they are legible even for large pipelines / long provenance trails
	Feature #5391: [SDK] Go KeepClient offers a "read block" API that recovers from transmission errors by attempting alternate servers.
	Feature #5413: [DRAFT] [API/SSO] Use externally-defined (e.g., LDAP) groups as permission groups
	Feature #5414: [Keep] Use data stored remotely (not in local keepstore)
	Feature #5415: [Workbench] Improve presentation of anonymously-viewed collections (via both "sharing link" and regular copy-and-pasted Location)
	Feature #5439: [Tests] Provide a docker image, based on puppet/packer process, that can run the entire test suite quickly.
	Feature #5459: [Crunch] Job logs include node-level resource usage statistics (cpu, ram, swap, etc)
	Feature #5494: [Documentation] Run an uncommitted script straight from a working directory using arv-run-pipeline-instance --run-here
	Feature #5495: [Workbench] Change "recent pipelines" to curated selection of "public pipelines"
	Feature #5510: [DRAFT] [Crunch] User can configure task retries
	Feature #5511: [Workbench] Workbench should support viewing VCF files
	Feature #5540: [Crunch] Contain all job/task containers in a memory-limiting cgroup
	Feature #5582: [API] [Workbench] Add read-only ("locked") flag to projects
	Feature #5646: [Workbench] Display duration on pipeline instance listings
	Feature #5668: [Workbench] Favorite projects, pipelines, collections
	Feature #5676: [SDK] arv-copy automatically creates new repository(ies) on the destination cluster
	Feature #5740: [OPS] Add diagnostics script for shell nodes
	Feature #5767: [API] Websockets subscribe API should accept a start timestamp, so clients don't have to miss any events logged between the given start time and websocket connection success.
	Feature #5769: [OPS] Routinely check for stale Gemfile.lock
	Feature #5778: [FUSE] Support efficient copy at command line
	Feature #5782: [Workbench] Support entering boolean script_parameters when editing a pipeline instance
	Feature #5828: [Workbench] Create new collection with selected files copy streams as well as files
	Feature #5840: [API] Relax repository name constraints to permit [-_.] -- ideally allow anything github allows
	Feature #5846: [Workbench] [Fiddlesticks] Live job log should show recent history
	Feature #5900: [SDK] Implement arvados.gettaskparam(*args) to match arvados.getjobparam(*args)
	Feature #5928: Syntax highlighting of viewing source in workbench
	Feature #5962: [Keep] Use bufferpool in keepproxy to limit memory use when there are many concurrent PUTs
	Feature #6015: [SSOs] Support authentication against multiple backend LDAP servers
	Feature #6310: [FUSE] Support scaling the internal block cache based on number of open files
	Feature #6348: [Deployment] [Documentation] Minimize system-wide dependencies for compute node setup
	Feature #6442: [Workbench] [API] Support "world-readable, but not world-searchable" permission on projects: "anyone with the link can view"
	Feature #6517: Port arvados-libpam to centos
	Feature #6574: [Workbench] [Webshell] Rough edges in web shell
	Feature #6782: [API] [Workbench] Generate debconf template from config yaml
	Feature #7024: [Keep] keepstore status.json should report runtime environment
	Feature #7127: [API] Job log cleaner should not clean logs when the job's log is the empty collection
	Feature #7328: [Crunch] [UX] Standard excepthook to help debug job problems
	Feature #7411: [Workbench] Pipeline template listings should include links to the scripts
	Feature #7595: [keep-web] Cache keep_services list between requests
	Feature #7694: [API] [Workbench] Check for nonexistent config params during rake config:check
	Feature #7818: [SDKs] Document that arv-copy accepts collection PDHs (ie content addresses) as arguments
	Feature #7845: [Crunch] Crunchstat and arv-mount print final stats before exiting
	Feature #7847: [SDKs] Update run-command to use arv-mount --mount-tmp instead of staging directory
	Feature #8163: [FUSE] arv-mount should detect and log any files/dirs that are still open after unmounting
	Feature #8212: [SDKs] arv CLI program accepts "-" and "/path/to/file" as values for parameters that expect JSON (most notably the "body" parameter)
	Feature #8213: [SDKs] arv get (and edit) should support a flag that suppresses display of non-writable attributes like "status" and "writable_by"
	Feature #8221: [Workbench] Add an "invert selection" button to checklists alongside "un/select all"
	Feature #8228: [SDKs] [FUSE] Python SDK and arv-mount use Range requests when a caller requests part of a block that has been ejected from the cache
	Feature #8311: [Crunch2] git tree mount points
	Feature #8318: [git] Should be able to clone git repositories that are shared with "anonymouspublic" without authentication
	Feature #8333: [SDKs] `arv keep docker` supports `repo:tag` image name scheme
	Feature #8344: [login-sync] Sets up settings.conf
	Feature #8363: [FUSE] arv-mount exits non-zero if any I/O errors were returned
	Feature #8379: [SDKs] Recursive arv-copy should report progress of the entire copy, in addition to individual objects
	Feature #8415: [SDKs] arv-put's default output includes both collection UUID and PDH
	Feature #8486: [Workbench] Change project "Jobs and pipelines" tab to just "Pipelines"
	Feature #8498: [SDKs] GoSDK arvadosclient parameters are more transparent
	Feature #8501: [SDKs] Add client-side join switches to `arv foo list`
	Feature #8502: [SDKs] Fetch all results with the CLI `list` command
	Feature #8515: [Node Manager] Detect and shut down nodes that fail provisioning in Azure
	Feature #8551: [Node Manager] Minimum idle time before node is eligible for shutdown
	Feature #8641: add key/value support for tags
	Feature #9017: [Tests] Make slow tests optional (e.g., run-tests.sh --short)
	Feature #9136: [Python SDK] Add a Collection method to efficiently concatenate files
	Feature #9245: [FUSE] Make .arvados#collection available in collection subdirectories
	Feature #9364: [keep-balance] "Expedited delete" tool: perform garbage collection on some specific (recently deleted) collections, bypassing usual GC race protections
	Feature #9747: [CWL] Secondary files entering into Arvados automatically
	Feature #9802: [Workbench UI] Pipeline instances should have a link to view the CWL file used.
	Feature #10123: [Workbench] [UNC] Copying collections also copies metadata
	Feature #10183: [Node Manager] Support ephemeral additional storage for Azure node types that do not have sufficient instance storage
	Feature #10191: Arvados should be able to cat/zcat files in collections easily
	Feature #10594: [API] [Workbench] Error out when unrecognized keys appear in config files
	Feature #10830: [Workbench] Support array type input parameters to Workflows
	Feature #10998: [keep-web] configurable block cache size
	Feature #11146: [Crunch2] [Workbench] Show slurm queue position of containers submitted to slurm but not yet running
	Feature #11184: [Keep] Support multiple storage tiers
	Feature #11745: Support schema.org/Google structured markup for datasets
	Feature #12017: Windows file system driver for Keep (arv-mount equiv)
	Feature #12018: Synchronize group membership with external data source
	Task #2822: [SDKs] Use Ruby SDK instead of google-api-client in arv-run-pipeline-instance
	Task #2995: Stub crunch-job with something that spits many logs, in order to test the limiting feature
	Task #3028: Implement testing framework for crunch-dispatch.rb
	Task #3190: [Blocker] Clarify design/requirements
	Task #3247: add some basic tests to the docker step in the build pipeline on jenkins
	Task #3272: Cache user authentication tokens
	Task #3358: make xxxx_production branch for each installation
	Task #3359: make staging branch
	Task #3475: Consult API server for user's admin status and scopes.
	Task #3591: Make chooser html partial DRYer
	Task #3592: Handle display of any user's home page with projects#show
	Task #3737: Include workbench URI in discovery document
	Task #3738: arv-run-pipeline-instance: display workbench URI instead of the bare uuid
	Task #3920: Use .utc-date data-utc-date javascript
	Task #4144: Use num_retries on regular collection writer instead of robust_put
	Task #4169: Write test harness for local mode crunch-job
	Task #4211: a-r-p-i sets pipeline instance to 'Paused' in a catchall exception
	Task #4212: [Workbench] Pipeline instance viewer should not omit paused pipelines
	Task #4815: Make it highlight the differences once again
	Task #4816: Review 4358
	Task #4837: [SDKs] Define API and in-memory data structure for collections in Python SDK
	Task #4874: Test that comparisons display correctly
	Task #4893: Review 4823-python-sdk-writable-collection-api
	Task #4987: Refactor Python tests to use fewer servers as able
	Task #4993: Implement test server service, and use it across test suites
	Task #5103: Implement live sync, merge
	Task #5167: Figure out appropriate scope for first implementation
	Task #5249: Write up implementation summary on wiki
	Task #5376: [workbench] wget link or download-all link in public collections
	Task #5454: Prototype keep-to-s3 gateway
	Task #5458: Update keep clients to heed proxy-uuid hints
	Task #5497: Review branch: 5375-preview-collection-text-files
	Task #5603: Review 5414-keep-service-hints
	Task #5649: [Workbench] We need better job graphs, less spaghetti
	Task #5651: Fix reported scaling factor for failed jobs (see notes)
	Task #5686: Review 5614-workbench-optimize-combine-collections-wip
	Task #5689: Review 4223-arv-keepdocker-download
	Task #5791: [Workbench] Job log memory and swap lines should use the same scale
	Task #5854: Add default display_order on server side
	Task #5855: Add Workbench support
	Task #5860: Provide detail about expected information/presentation
	Task #5918: Review 5523-stats-error
	Task #5938: Handle normal container startup and shutdown races without logging an error/notice or missing the first interval
	Task #5995: Review 5736-node-manager-easy-slot-cleanup-wip
	Task #6109: Review
	Task #6124: Add "goal mode" infrastructure
	Task #6125: Enter goal mode when hitting Run on a pipeline template
	Task #6126: Decide URL and advice for appropriate next step when in "pipeline goal" mode
	Task #6127: Preserve template goal through login->UA->profile->back-to-work process, and then convert template to instance goal
	Task #7339: Update the "Using arv-copy" user guide to include pipeline instances as one of the object types that can be copied over.
	Task #7749: Draw UI Implementation
	Task #8126: Review
	Task #8210: review 8189-handle-large-collections-better
	Task #8359: Improve node manager logging
	Task #8360: Review 7667-node-manager-logging
	Task #8367: Review
	Task #8368: Update docs
	Task #8369: Test process
	Task #8462: Review 8388-events-terminate
	Task #8719: Review PR #40
	Task #8976: Review 8912-missing-blocks-report
	Task #9052: Review 9017-skip-slow-tests
	Task #9101: Communicate --short flag to Rails test suites
	Task #9102: Skip collection performance tests in API suite
	Task #9103: Skip collection performance tests in Python SDK suite
	Task #9158: Skip expensive setup/teardown, too
	Task #9293: Review docs/spec [[Expiring collections]]
	Task #9294: Document desired behavior and interpretation
	Task #9296: Update API to desired behavior
	Task #9297: Update Workbench to desired behavior
	Task #9298: Update arv-mount to desired behavior
	Task #9302: Review 9278-expiring-collections
	Task #9778: Upgrade API Server to Rails 4.2
	Task #10046: rails systemd unit or nginx config
	Task #10066: Review
	Task #10073: Transition plan
	Task #10740: Review 10705-state-transition-error
	Task #11115: Review
	Task #11198: Review
	Task #11390: Review
	Task #11391: Review
	Task #11479: Review
	Task #11526: Review 11470-update-task-fields
	Task #11845: Review 11836-nodemanager-status-json
	Task #11932: Review 11843-arpi-transient-error
	Task #11933: Review 11908-properties-column-json
	Task #11940: Review 11917-dont-clear-cache
	Story #2992: [Workbench] Support "display_order" (integer) key for pipeline template components. Obey when rendering pipeline summary/details in Workbench.
	Story #3022: Improvements to sortable log viewer
	Story #3040: Replace the ApiClient is_trusted attribute with a per-{user,client} flag which is set by the user when logging in with a client for the first time.
	Story #3163: [Tests] Non-destructive test script suitable for running on a production VM/server in order to verify correct operation of basic features.
	Story #3179: "Dump core" option when running a job (or every job in a pipeline): save the state of a failed task's container before tearing it down.
	Story #3199: [SDK] Keep clients (e.g., arv-put and crunch-job) ask API to refresh blob signatures if expiry times are sooner than ETA of collection.create
	Story #3279: [Workbench] Link to OFTC web chat on version/bugreport modal
	Story #3298: [Workbench] Rearrange collection chooser to show a preview (file list) inline instead of in a separate preview pane. See [[Workbench UI images]]
	Story #3355: [SDK] Show the API server URI for the new pipeline instance when running a pipeline from the command line.
	Story #3365: [SDK] Show the Workbench URI for the job when running a job or pipeline from the command line
	Story #3490: [Workbench] Create and edit pipeline templates
	Story #3532: [Keep] Keepstore queries API server to verify tokens
	Story #3640: [SDKs] Add runtime option to SDKs (esp Python and arv-mount) to use a filesystem directory block cache as an alternative to RAM cache.
	Story #3711: [Docs] Update Arvados permissions model in docs
	Story #3815: [SDK] arv-put supports --ignore-symlinks
	Story #3897: [SDKs] Clients can specify which auth config file to load.
	Story #4019: [API] Support query of "properties" field on objects
	Story #4030: [SDK] Command line tool to manipulate collection manifests
	Story #4128: [Workbench] [refactor] JavaScript refactoring
	Story #4137: [SDKs] arvados.collection.Manifest classes for manipulating collection manifests
	Story #4237: [Documentation] Update and expand user/tutorials/tutorial-keep-mount.html to show practical ways of finding data via FUSE mount.
	Story #4255: [SDKs] Implement "arv keep less" and "arv keep check" natively instead of hoping whless and whcheck are installed.
	Story #4350: [Workbench] Order by Collections "Name" column puts "folders" (collections containing files) at the top
	Story #4442: [API] [Workbench] Site-wide regex searching
	Story #4566: [Documentation] Introduce compute workflow / pipeline concepts in user guide
	Story #4744: [Workbench] Merge "user agreement" and "required profile fields" into a single page
	Story #4833: [Crunch] Quick turnaround time when updating a docker image in order to re-run a job
	Story #4919: [API] Arvados clients can use standard OAuth2 protocol instead of custom token handling mechanism
	Story #5007: [Workbench] Pilot material design theme
	Story #5094: [Workbench] Make pipeline rerun UI more like jobs'
	Story #5098: [Keep] Design: using Arvados to browse, and run jobs on, input data that is stored remotely (not on local Keep disks)
	Story #5142: [DRAFT] [API] Enrich supported dataclass values for pipeline templates
	Story #5428: [Tests] Enhance keep testing to support signed locators.
	Story #5468: [SDKs] Refactor arv-get/put/copy into the Python "arv" wrapper using common exception-handling and argument parsing
	Story #5621: [Documentation] Improve SDK documentation
	Story #5624: [API] [Workbench] [Performance] Use object caching to improvement performance (such as memcache).
	Story #5781: [API] [DRAFT] Provide API methods for manipulating and combining collections
	Story #5788: [SDKs] [refactor] Dry up KeepService get() and put() in arvados/keep.py
	Story #5793: [Workbench] Add search filter to "/links" page
	Story #5865: [Crunch] Run a local job using a docker image directly from the shell VM (without writing the docker image to Keep)
	Story #5914: [DRAFT] Provide one clear way for users to get data from an external source into Arvados
	Story #6058: [Workbench] Provide a guided workflow to new users who want to upload data and run a public pipeline
	Story #6307: [Workbench] Public project listing should exclude subprojects
	Story #6311: [Maybe] [SDKs] Support caching Keep blocks in memcached
	Story #6313: [Node Manager] Booting nodes shouldn't satisfy min_nodes
	Story #6387: [SDKs] arv-put progress meter scales based on file size, rather than being fixed to MiB
	Story #6405: [Workbench] Allow additional curation of the public projects listing
	Story #6570: [DRAFT] [Workbench] More encouragement for users to send feedback
	Story #6642: [API] Include a mechanism to automatically trust some clients
	Story #6783: [Performance] Compare Collection performance benchmarks across components
	Story #6784: [Keep] Write performance tests
	Story #6840: [SDKs] Teach arv-copy to copy whole projects
	Story #7394: [SDKs] arv-copy copies all metadata associated with an object
	Story #7439: [SDKs] CLI tools support Mac
	Story #7468: [Workbench] Displays content at the top of the dashboard, set by the sysadmin
	Story #7469: [Workbench] Provide an interface for the sysadmin to set Dashboard banner content
	Story #7470: [Workbench] Provide an interface to delete files from collections
	Story #7471: [SDKs] Provide a CLI tool to add files to an existing collection
	Story #7472: [SDKs] Provide a CLI tool to remove files from a collection
	Story #7473: [FUSE] [SDKs] Ensure FUSE and other SDK clients work with a default replication level != 2
	Story #7474: [FUSE] Add a --replication switch
	Story #7476: [SDKs] Ensure `arv keep put` replicates properly
	Story #7477: [SDKs] Go Keep client enforces a minimum transfer rate for proxy connections
	Story #7542: [Documentation] Write a Crunch wiki page
	Story #7543: [Documentation] Write a Crunch2 wiki page
	Story #7621: [API] Job model validates that the script exists in the repository at script_version
	Story #7830: [Workbench] User admin interface should show which VM(s) the user can login to
	Story #7831: [FUSE] Add --umask, --fmask, --dmask options to control file permissions
	Story #7973: [FUSE] Reset mount-tmp to the state of another collection
	Story #7980: [Node Manager] Broken nodes always get shut down, exempt from all node-counting considerations
	Story #7990: [FUSE] With --exec, expose the mount point in an environment variable
	Story #8000: [Node Manager] Shut down nodes in SLURM 'down' state
	Story #8022: [SDKs] CLI tools install tab completion aids
	Story #8058: [Documentation] Add Gitolite sanity check to install guide
	Story #8091: [Workbench] Show all known runtime_constraints in job status pane
	Story #8179: [keep-balance] Refuse to act when planning to delete data above an admin-specified threshold
	Story #8180: [Keepstore] Don't trash/delete blocks until they appear on multiple trash lists
	Story #8181: [Keepstore] Support a portable data challenge API
	Story #8313: [SDK] arv-copy better Git repository error reporting
	Story #8314: [SDKs] arv-copy continue on error
	Story #8327: [Deployment] Update Rails package dependencies
	Story #8328: [Deployment] .yml.example files should be consistent
	Story #8329: [Docs] SSO install guide documents default_link_*
	Story #8331: [Documentation] Clarify PostgreSQL setup commands for Software Collections users
	Story #8332: [Docs] Fix flow of Rails servers' install guides
	Story #8552: [Crunch] Log throttling shouldn't throttle logs from crunchstat, crunch-job
	Story #8560: [Workbench] Ability to create a new project when creating a new pipeline instance
	Story #8562: [Workbench] Project listings act like file manager directory trees, with expanding+collapsing trees
	Story #8565: [API] Go code to connect to the API server database and make basic sense of it
	Story #8566: [API] Go code to implement the Arvados API permissions model
	Story #8773: [GATK Queue] Support a mechanism to redirect stderr for children
	Story #8774: [Crunch] Limit maximum wait time between retries
	Story #8775: [Docs/Workbench] Improve SSH agent documentation
	Story #8778: [Docs] Document Git 1.7.10+ dependency for shell node and HTTPS Git cloning
	Story #8780: [SDKs] Git credentials helper in CLI tools
	Story #8781: [Docs/Workbench] Document how to clone repositories over HTTPS
	Story #8838: [SDKs] PySDK Collection class lets you iterate files and subcollections separately
	Story #8885: [PySDK] API client logs a warning when you create an object without an owner_uuid
	Story #8894: [Node Manager] Less logging of tracebacks for routine exceptions
	Story #8912: [Data manager] Verbose reporting
	Story #8999: [node manager] Upgrade to libcloud 2.0
	Story #9045: [SDKs] `arv keep docker` can upload an image already saved as .tar
	Story #9046: [Crunch2] Can issue container request to pull and import Docker images
	Story #9147: [SDKs] `arv keep docker` can make links for an image already in a collection
	Story #9148: [API] Finalize and document the collections/provenance and collections/used_by API calls
	Story #9150: [API] Synthetic collections include the "kind" field
	Story #9278: [Crunch2] Document/fix handling of collections with non-nil expires_at field
	Story #9428: [SDKs] Port arv-keepdocker to use the docker-py library
	Story #9441: [Workbench] Improve user page group memberships management [to be groomed]
	Story #9584: [FUSE] Don't display expiring collections in regular collection listings
	Story #9590: [FUSE] Trash directory to list, inspect, and un-trash trashed collections
	Story #9591: [FUSE] Undelete collections by moving them out of the TrashDirectory
	Story #9592: [FUSE] rmdir on CollectionDirectory sets expires_at
	Story #9839: [Crunch2][Workbench] List both pipeline_templates and workflows on both pages
	Story #9960: Automate starting/enabling services during Workbench and API package installation
	Story #10354: Add varchar_pattern_ops to all relevant PostgreSQL UUID indexes
	Story #10477: Switch from goamz to aws-sdk-go
	Story #10525: Integrate CI::Reporter to provide test output in JUnit XML format
	Story #10679: [Workbench] Update job stats to be more useful to end user
	Story #10996: [SDK] Writable file-like interface for collections in Go SDK
	Story #11452: Support rendering of CWL description fields with embedded markup
	Story #11657: [SDKs] publish python3-arvados-python-client packages
	Story #11705: [API] Remove async permissions code
	Story #11740: Support display & reseting of expiration dates for collections
	Story #11741: [Crunch2] Create workflow object for CWL runs which are initiated from the command line
	Story #11742: [API Server] Support for pulling Docker images and uploading them to Keep
	Story #11770: [Python SDK] Implement support for universal newline mode in Collections API
	Story #11836: [Nodemanager] Improve status.json for monitoring
	Story #11875: User accounts optionally have read-only access
	Story #11876: [R SDK] Create a Bioconductor/R SDK
	Story #11884: Convert Collection properties column type to JSONB
	Story #11908: Migrate Collections.properties to JSONB
	Story #11942: [CWL] arvados-cwl-runner should support tagging output collection using properties
	Story #11983: On the collections page indicate expiring collections

OPS - Ops Future Sprints

Due in over 2 years (01/03/2020)

21%

88 issues   (18 closed — 70 open)

Related issues

	Feature #7331: [Deployment] Add systemd unit definition to the keepproxy package
	Feature #7407: [Deployment] Add sysvinit script to the keepstore package
	Task #7408: Review

Curoverse Support - Pipeline Future Sprints

Due in over 2 years (01/05/2020)

8%

38 issues   (2 closed — 36 open)

Related issues

	Task #8974: Review
	Story #8563: [Docs] Pipeline author guide gives a basic demonstration of including a third-party tool
	Story #8676: [Documentation] Promote pipeline porting pages
	Story #8770: [Documentation] Integrate Sally's "Porting a pipeline" training presentation with Arvados documentation

Deferred

Due in over 3 years (12/31/2020)

44%

52 issues   (24 closed — 28 open)

Related issues

	Feature #2580: [Tests] Keep load test
	Feature #2656: [Keep] Keep performance testing
	Feature #2706: [SDKs] Python SDK Keep client can enable "HEAD before PUT" mode when writing. Expose this as an arv-put option.
	Feature #3146: Write network and disk IO performance stats to Keep/KeepProxy logs
	Feature #3162: [API] Client can request signed locators for specific blobs from API server
	Feature #3231: design document for websockets/event bus
	Feature #3402: If a pipeline/job is in a project, it would be nice if the breadcrumbs reflected this for easy maneuvering back and forth
	Feature #3498: Counting the number your pipelines were used by unique users
	Feature #3528: Machine-readable API schema that indicates which fields of each object type permit updating by regular (non-admin) users who have can_write or can_manage permission.
	Feature #4011: [Crunch] port crunch-job to python
	Feature #4217: [Metadata] As an adminstrator, I want to specify a required ontology for a given data type for a defined class of users
	Feature #5768: [Node Manager] Add max_node_per_user config
	Task #2455: Fill in missing story description - goals / examples
	Task #2565: Check out 2028 notes and sync up with Tom
	Task #2625: Compose tests
	Task #2626: Presentation for stress test results
	Task #2633: Implement TLS support internally
	Task #2636: Test SSL support
	Task #2643: "Ready to ingest" endpoint (in /actions controller?)
	Task #2647: Put files from staging directory into Keep
	Task #2648: Write tests
	Task #2649: Assign collection to project
	Task #3189: [Blocker] Clarify design/requirements
	Task #3459: Add documentation on how to specify a UserAgreement
	Task #3460: Add a link and make it possible to see signed user agreements
	Task #3461: Review 3257
	Task #3523: Use fancy editor component for Textile textareas
	Task #5503: Deploy "proxy_buffer off;" on /collections/download
	Story #1558: [Lightning] Lightning Memory-Mapped Database
	Story #2028: [SDKs] Python SDK supports Django model/manager style usage
	Story #2293: [Keep] Keep server supports SSL
	Story #2523: [Sample clients] Ingestor utility responds to "ready to ingest" events by copying the specified files from staging directory to Keep and storing the collection in the specified group.
	Story #2679: [API] If enabled by user via "preferences" checkbox in Workbench, notification service sends email to owner when a pipeline instance finishes or fails.
	Story #3093: [Crunch] Pipeline can use other pipelines templates as components
	Story #3180: [Crunch] When a job fails in a pipeline instance, should the other queued/running jobs get cancelled automatically?
	Story #3299: [Workbench] Render textile markup in description fields
	Story #3304: [Workbench] Add checkboxes to multi-type chooser modal to filter by object type
	Story #3473: Ability to mark one's own projects as read-only to avoid changing it by accident.
	Story #3491: [Keep] Support transparent compression of blocks in Keep
	Story #3519: Move permission links to their own table in API server.
	Story #5471: [Workbench] Remove ActiveRecord dependency