Development 2023-09-27 sprint open Due in 1 day (09/27/2023) 59% 98 issues (58 closed — 40 open) Related issues Feature #15768: Support multi-select operations Actions Feature #19982: Ability to know when a container died because of spot instance reclamation and option to resubmit Actions Feature #20225: Let users choose collection subdirectories for workflow Directory inputs Actions Feature #20609: Progress bar showing state of subprocesses Actions Feature #20732: Playground welcome banner Actions Feature #20825: Extension to run subworkflow in its own runner Actions Feature #20850: Notification that workbench 1 is deprecated Actions Feature #20880: Can link to wb2 URL to prompt user to launch workflow, specify project uuid in query Actions Feature #20937: http-to-keep feature of arv-copy Actions Task #17875: Review Actions Task #20329: Review Process document process Actions Task #20330: Review 20225-directory-input-subfolder-selection Actions Task #20441: Review 15768-multi-select-operations Actions Task #20442: Review Actions Task #20503: Review 20497-updating-wgs-tutorial Actions Task #20510: Review Actions Task #20741: Review customer support procedure Actions Task #20742: Review operational systems review process Actions Task #20743: Review cybersecurity assessment process Actions Task #20744: Review incident disclosure and response process Actions Task #20745: Review personal data inventory process Actions Task #20746: Review Curii cybersecurity program Actions Task #20747: Review Curii data protection policy Actions Task #20759: Review Actions Task #20761: Review Actions Task #20770: 1. Prepare X.Y-staging branch Actions Task #20771: 2. Update the "Upgrading Arvados and Release notes" doc page (main and release branch), update provision.sh, formula and arvbox to point to release Actions Task #20772: 3. Ensure that the entire automated testing pipeline is passing on Jenkins Actions Task #20773: 4. Review release branch Actions Task #20774: 5. Draft release notes and publish them to www-dev Actions Task #20775: 6. Review release notes Actions Task #20776: 7. Create next redmine release Actions Task #20777: 8. Build RC packages Actions Task #20778: 9. Build RC arvados/jobs image Actions Task #20779: 10. Ensure installer tests are passing Actions Task #20780: 11. Build compute image & deploy RC packages to playground Actions Task #20781: 12. Run bam-to-vcf demo pipeline Actions Task #20782: 13. Manual testing Actions Task #20783: 14. Approve RC for release Actions Task #20784: 15. Build final release packages Actions Task #20785: 16. Publish stable arvados/jobs Docker image Actions Task #20786: 17. Push packages to stable Actions Task #20787: 18. Publish Python and Ruby packages Actions Task #20788: 19. Publish Java package Actions Task #20789: 20. Publish R package Actions Task #20790: 21. Publish arvados/arvbox-demo image Actions Task #20791: 22. Tag commits, fast-forward X.Y-release branch to match X.Y-staging Actions Task #20792: 23. Ensure doc.arvados.org is up to date Actions Task #20793: 24. Merge release notes (step 6) from "develop" to "main" on arvados.org Actions Task #20794: 25. Send out release announcements Actions Task #20795: 26. Major releases only: Copy "run-tests" jobs in jenkins Actions Task #20796: 27. Add the release to zenodo.org Actions Task #20812: Review Actions Task #20864: Review https://github.com/arvados/google-api-ruby-client/pull/1 Actions Task #20866: Review Actions Task #20867: Review 20850-wb1-deprecated Actions Task #20902: Review 20862-google-api-client Actions Task #20911: Review Actions Task #20940: Review 19359-optional-enum Actions Task #20941: Review 20933-arv-copy-cwl Actions Task #20942: Review 20937-arv-copy-http Actions Task #20944: Review 1.2.1 release notes Actions Task #20946: Review Actions Task #20948: Review 20930-websocket Actions Task #20952: Review 20926-installer-pg-client-version Actions Task #20957: Review Actions Task #20958: Review Actions Task #20959: Review 20932-wb2-docs Actions Task #20960: Review 20955-log-update-doc Actions Task #20963: Review Actions Task #20964: Review 20885-pdoc-style Actions Task #20966: Review Actions Task #20967: Review 20947-update-doc-script Actions Task #20968: Review Actions Task #20992: Review 20990-name-btree Actions Story #20300: RailsAPI upgrade from 5.2.x to 6.1.x Actions Story #20885: Revise Python docstring style for Markdown extensions Actions Story #20932: Workbench 1 loose ends in the documentation to update for Workbench 2 Actions Story #20951: Document deprecated api_client_authorization fields Actions Story #20955: Crunch log transition documentation issues Actions
Development 2023-10-11 sprint open Due in 15 days (10/11/2023) 0% 10 issues (0 closed — 10 open) Related issues Feature #20981: Visual design for workflow search and picking Actions Feature #20983: Tool to automatically populate Arvados InstanceTypes Actions Task #20871: Review Actions Task #20962: Review Actions Story #20846: Support Ubuntu 22.04 LTS Actions
Development 2023-10-25 sprint open Due in 29 days (10/25/2023) 0% 7 issues (0 closed — 7 open) Related issues Feature #18961: Go FileSystem / FUSE mount supports block prefetch Actions Feature #20978: Support multiple candidate instance types to assign containers Actions Feature #20994: Feature to incorporate custom panels into collection and project view Actions Feature #20995: Prefetch small files when scanning a collection directory Actions Task #20437: Review Actions Story #20690: Remove workbench 1 from main branch !!!! Actions
Development 2023-11-08 sprint open Due in 43 days (11/08/2023) 0% 5 issues (0 closed — 5 open) Related issues Feature #2960: Keep can stream GET and PUT requests Actions Feature #15317: [Keep] expose backend latency stats in the Prometheus metrics Actions Feature #19675: Panel that lists configured instance types Actions Task #20332: Review Actions Arvados Workbench 2 - Feature #19302: Left side panel changes Actions
Development 2023-11-22 sprint open Due in 57 days (11/22/2023) 0% 1 issue (0 closed — 1 open) Related issues Feature #20318: Go SDK supports local filesystem-backed data cache Actions
Development 2023-12-06 sprint open Due in about 2 months (12/06/2023) 0% 1 issue (0 closed — 1 open) Related issues Feature #20693: Server side coordination of multiple writers to a collection Actions
Development 2023-12-20 sprint open Due in about 3 months (12/20/2023) 0% 1 issue (0 closed — 1 open) Related issues Feature #20996: Efficient packing of small files into blocks in keep-web Actions
To be scheduled open Due in over 13 years (01/01/2037) 2% 67 issues (1 closed — 66 open) Related issues Feature #14922: Run multiple containers concurrently on a single cloud VM Actions Feature #19378: Can select folder to recursively upload all the files to Collection Actions Feature #19860: Support "pull image" container request Actions Feature #19970: a-c-r should refuse to run if git has locally modified files Actions Feature #20192: Move AuditLogs.tidy_in_background from Rails to controller Actions Feature #20756: Support crunchstat tracking and memory limits with singularity Actions Task #13355: Review Actions Task #19398: Review Actions Task #19641: Review: Personas Actions Task #19646: Review Actions Task #19805: Review Actions Task #19946: Review Actions Task #19949: group review Actions Task #19951: Review Actions Task #20191: Review Actions Task #20333: Review Actions Task #20436: Review Actions Task #20444: Review Actions Task #20671: Review Actions Task #20870: Review Actions Story #12917: Support ?include=container_uuid for container request lists and group contents Actions Story #17477: document arv-bash Actions Story #19463: Redesign process info panel Actions Story #19578: Update wb2 Use Cases and Personas Actions Story #19636: Access to editing inputs on "draft" tasks Actions Story #19818: Write docstrings for arvados.api Actions Story #19819: Write docstrings for arvados.arvfile Actions Story #19820: Write docstrings for arvados.cache Actions Story #19821: Write docstrings for arvados.collection Actions Story #19822: Write docstrings for arvados.config Actions Story #19823: Write docstrings for arvados.crunch Actions Story #19824: Write docstrings for arvados.errors Actions Story #19825: Write docstrings for arvados.events Actions Story #19826: Write docstrings for arvados module Actions Story #19827: Write docstrings for arvados.keep Actions Story #19828: Write docstrings for arvados.stream Actions Story #19829: Write docstrings for arvados.timer Actions Story #19830: Write docstrings for arvados.util Actions Story #19831: Write docstrings for arvados.vocabulary Actions Story #19832: Re-work picking dialog into 3 panel view Actions Story #19833: Untag arvados/jobs:latest from docker hub Actions Story #19843: arv-federation-migrate supports LoginCluster type migrations Actions Story #19846: Use collection properties instead of links to tag docker images Actions Story #19977: Update tests to run with a strict vocabulary Actions Story #20224: Workbench 2 accepts modern SSH key types (elliptic curve/ecdsa/ed25519) Actions Story #20435: CWL user guide release Actions Story #20720: Every section header in the documentation should have an id for easy linking Actions Story #20724: Silence httplib2/ssl deprecation warning Actions Arvados Workbench 2 - Feature #19885: Collapse middle breadcrumbs with ellipse when window too narrow Actions
To be groomed open Due in over 14 years (01/01/2038) 1% 227 issues (2 closed — 225 open) Related issues Feature #9964: arvados-cwl-runner limits output data to keep using output_glob Actions Feature #12430: Crunch2 limit output collection to glob patterns Actions Feature #15250: a-c-r set TTL on final output collection Actions Feature #16057: Combine selected collections Actions Feature #16241: Object's description display & editing is inconsistent & lacks textile support Actions Feature #16316: a-c-r handles resource range requests (especially CPU) and adjusts requests based on what is in InstanceTypes list Actions Feature #16369: Should be possible to promote a workflow run via arvados-cwl-runner to a published workflow with one click Actions Feature #16385: add prebuilt container images for Arvados releases Actions Feature #16593: Way to determine owner to be able to request access Actions Feature #17261: Determine ownership of objects within wb2 Actions Feature #17695: [costanalyzer] make an accurate report for spot instances on AWS Actions Feature #18563: Simplify/streamline InternalURLs/ExternalURL situation Actions Feature #18573: packer compute image builder script should take an explicit arvados package version to install instead of latest in selected package repository Actions Feature #18970: Add support for browsing as anonymous user Actions Feature #19080: Offer to un-trash items, instead of "not found" dialog Actions Feature #19086: Can provide input JSON/YAML when launching a Workflow as an alternative to the input form Actions Feature #19181: [tracking] Workbench 2 file operations Actions Feature #19194: Return can_manage and can_write for all object types (not just users/groups/projects) Actions Feature #19197: Optimize permission checks for can_write/can_manage fields Actions Feature #19369: Update browser title to reflect current main panel view Actions Feature #19637: Recursive delete of workflow & associated intermediates Actions Feature #19871: Rendering user names in project columns Actions Feature #19887: Include modified_by_user_uuid in project details Actions Feature #19919: Recursive copy through workbench Actions Feature #19968: Extension to control max retries of workflow steps Actions Feature #19974: Option to re-submit preemptible jobs to reserved nodes when previous attempt was interrupted Actions Feature #19979: Open sidebar to get file details, make it easy to copy keep: and keep-web links Actions Feature #19987: email notifications Actions Feature #20040: Getting additional cost info from AWS Actions Feature #20058: Easier presentation of search results Actions Feature #20081: Easy copy project name to clipboard Actions Feature #20094: Render workflow with cwl-svg Actions Feature #20211: Revamp Sharing dialog to show access via ownership Actions Feature #20220: Dispatcher uses live logs endpoint on crunch-run to fetch logs and store a backup locally Actions Feature #20285: Link to monitoring (grafana) in admin menu Actions Feature #20293: Configure links that appear under User/Admin/Help menus Actions Feature #20302: Banner has a "don't show this to me again" checkbox instead of always defaulting to hiding banner after being shown. Actions Feature #20304: Get billing cost from cloud and include it in metrics Actions Feature #20310: Add server-sent events stream at .../containers/{uuid}/log_events Actions Feature #20375: Multi-select operations on the process list page Actions Feature #20376: Select individual process status Actions Feature #20383: Monitoring that gives list of compute containers that don't seem to be making progress Actions Feature #20385: Dispatcher aggregates local keepstore usage on compute nodes Actions Feature #20426: Installer sets up health check aggregator and monitoring/alerting based on health checks Actions Feature #20471: Admin CLI for managing dispatcher / cloud VMs Actions Feature #20532: Grouping steps by sample Actions Feature #20542: Report metrics for expensive response fields Actions Feature #20546: Displaying AWS RDS metrics on grafana Actions Feature #20592: cwl-runner should use arv:WorkflowRunnerResources.acrContainerImage Actions Feature #20604: crunch-run retry timeout should increase for long-running containers Actions Feature #20622: arv-keepdocker uses new "pull image" API Actions Feature #20632: Complex recursive copy/delete operations Actions Feature #20640: API for admin to query materialized permissions Actions Feature #20641: extensions for options currently only available on command line Actions Feature #20659: Tool to update a-c-r version in workflow records Actions Feature #20708: Control permission for individual users to run containers or use container shell feature Actions Feature #20733: installer supports automated rolling upgrade of load-balanced nodes Actions Feature #20736: expand & collect keepstore and keepweb performance metrics Actions Feature #20751: Searching or linking a container uuid should bring bring up list of container requests Actions Feature #20805: Create empty folder and move within collection operations Actions Feature #20806: CLI operations to manipulate files in collections Actions Feature #20851: Ability to define which federated clusters are searched by default in workbench2 Actions Feature #20972: Bulk update API Actions Feature #20979: Research spot instance retry strategies Actions Feature #20980: Visual design for description panel Actions Feature #20988: Diagnostics should test that websockets is functional Actions Task #16096: Review Actions Task #16206: Review 16168-process-container-request-properties Actions Task #16354: Review Actions Task #17459: Review Actions Task #18041: Review Actions Task #18709: Review Actions Task #19117: Review Actions Task #19121: Review Actions Task #19151: Review 19143-project-list-workflows Actions Task #19160: Review Actions Task #19162: Add tests Actions Task #19174: Review tests branch Actions Task #19242: Engineering discussion about desired solution Actions Task #19397: Review Actions Task #19679: Review "data-mgtmt" branch in arvados-www Actions Task #20061: Process page -- update timing information Actions Task #20195: Review Actions Task #20202: Review 19981-reuse-flex-keep-cache Actions Task #20280: Review Actions Task #20555: Review Actions Task #20879: Review Actions Story #13327: More responsive page loads with API ?include=container_uuid Actions Story #15397: Declutter the API Actions Story #15624: [WB2] Provide capability to use CSV file to populate inputs of workflow Actions Story #15880: Remove hosted git service Actions Story #18846: Rendering rich descriptions Actions Story #19612: Frozen project UI follow-up improvements Actions Story #19633: UX redesign ideas Actions Story #19676: Turn data organization patterns deck into documentation page and/or blog post Actions Story #19935: Document arv-copy usage and format Actions Story #19939: Fully document, illustrate how to use arv-mount --exec Actions Story #19985: crunch-run sets OOMScoreAdjust for the container it runs and other child processes Actions Story #20059: Redesign Register Workflow Panel Actions Story #20060: Table Grouping for Project Page Actions Story #20062: WB2 Landing Page Actions Story #20063: Subprocesses lag when loading process page Actions Story #20097: Improve workflow picking/selection Actions Story #20107: Research retry strategies when SDK API calls return 5xx errors Actions Story #20213: Design scale tests for Arvados ML Actions Story #20230: Ensure that large container/request field loads are size-limited Actions Story #20231: container/request list calls do not return large fields by default Actions Story #20232: Don't de/serialize large container/request columns for simple list requests Actions Story #20234: Streamline workflow finding/launching process/Workflow versioning Actions Story #20237: fair scheduling in cloud dispatcher Actions Story #20239: Dispatcher calculates final container cost after runner/instance disappears Actions Story #20248: Support simple math expressions in Containers.LSF.BsubArgumentsList Actions Story #20250: Simple read-only S3 endpoint Actions Story #20286: WB2 labels and buttons look too similar Actions Story #20288: Need better icon for registered workflows in WB2 Actions Story #20289: Button in LHS panel (more info) are not consistent Actions Story #20290: Consider making all buttons blue in wb2 Actions Story #20311: Update Python packages to build with PEP 517/518 Actions Story #20390: Let users view Prometheus stats through an Arvados API endpoint with permissions check Actions Story #20391: crunchstat puts all the stats it records in Prometheus Actions Story #20392: Workbench 2 displays container performance metrics from Prometheus statistics Actions Story #20458: Document arvados-client logs command Actions Story #20517: Impedance mismatch using `systemctl is-system-running` as default BootProbeCommand Actions Story #20518: Remove calls to datetime utcnow/utcfromtimestamp Actions Story #20519: Support path-like objects throughout the SDK Actions Story #20521: Python tools emit logs from googleapiclient Actions Story #20650: a-c-r natively supports S3 inputs just like HTTP/S Actions Story #20651: a-c-r lets users specify workflow by name (+version?) Actions Story #20662: API to monitor container/requests Actions Story #20692: keep-web can pack multiple small files into a single block Actions Story #20723: Stop running setup.py in our build+test infrastructure Actions Story #20727: Publish standalone binaries for arvados-client, other Go client tools Actions Story #20728: Remove build instructions for PySDK C dependencies, if possible Actions Story #20752: Update aws-sdk-go-v2 and support HostnameImmutable flag Actions Story #20800: When runtime_constraints.ram exactly matches instance_type.RAM, try to use that instance type Actions Story #20802: shellinabox not packaged for RH8-based distro Actions Story #20837: systemd controls memory usage on Arvados services Actions Story #20852: Convert cookbook recipes into SDK methods Actions Story #20917: Test PySDK with different versions of pycurl Actions Story #20923: Way to mark a project as favorite from the project page Actions Story #20924: Search supports workflows Actions Story #20925: WGS tutorial papercuts Actions Story #20927: When validating a remote token, translate transient network errors to a 5xx status Actions Story #20938: Fix apt lock failure in test-provision-debian10 Actions Story #20945: Using + button to save new sharing links is not obvious Actions Story #20954: Playground uses tooltips to introduce Workbench UI Actions Story #20971: Priority of 3rd level containers Actions Story #20977: a-c-r crashes with "Secret store only accepts strings" if you try to register a workflow with secrets Actions Story #20991: Let container requests specify whether they'd rather run fast or run cheap Actions Arvados Workbench 2 - Feature #14847: [Workbench] Version/ Debugging info Actions Arvados Workbench 2 - Feature #15353: [Data operations] Chooser supports browing/picking projects/collections on federated clusters Actions Arvados Workbench 2 - Feature #16168: edit 'properties' of container requests Actions Arvados Workbench 2 - Feature #19480: remove "choose" flow from the registered workflow run page Actions Arvados Workbench 2 - Task #19834: Notification with email Actions Arvados Workbench 2 - Story #16071: [Collection view] Provenance, used by for collections Actions Arvados Workbench 2 - Story #19154: Sharing dialog improvements followup Actions Arvados Workbench 2 - Story #19584: Can we use better terms for a group_filter and projects Actions