Arvados

Story

Resolved

Sprint Impediments

3550 3550 Tom Clegg Tom Clegg 3 SDKs 54 SDKs [SDKs] arv-run-pipeline-instance supports running jobs locally using arv-crunch-job Resolved Resolved 3 1.0 1.0 1.0 3550 36 3 74 6 1.0	3772 3772 Review 3550-local-pipeline Tom Clegg 3 0 3550 3 36 -c-a 5 3771 3771 Add --run-jobs-here flag Tom Clegg 3 0 3550 3 36 -c-a 5
3823 3823 Tim Pierce Tim Pierce 44 [Support] Fix bugs and write tests (the second half of the sprint) Resolved Resolved 3 2.5 2.5 2.5 3823 36 44 74 3 2.5	3732 3732 Fix bugs the second half of the sprint Tim Pierce 44 0 3823 3 36 -c-a 5
2761 2761 Radhika Chippada Radhika Chippada 72 [Workbench] diagnostic suite runs against a production instance Resolved Resolved 3 3.0 3.0 3.0 2761 36 72 74 6 3.0	3870 3870 Review branch: 2761-diagnostic-suite Radhika Chippada 72 0 2761 3 36 -c-a 5
3411 3411 Tim Pierce Tim Pierce 44 [API] Implement Trash behavior using collection expiration Resolved Resolved 3 1.0 1.0 1.0 3411 36 44 74 6 1.0	3739 3739 Add default_trash_lifetime Add default_trash_lifetime config setting to workbench config/application.default.yml See [[Hacking Workbench]] Tim Pierce 44 0 3411 3 36 -c-a 5 3741 3741 API server filters for trashed objects API server implicitly selects delete_at is null or delete_at > now when querying collections table (rails "default scope" can do this) Tim Pierce 44 0 3411 3 36 -c-a 5 3883 3883 Review 3411-expire-collections Tom Clegg 3 0 3411 3 36 -c-a 5
3705 3705 Tim Pierce Tim Pierce 44 Keep 53 Keep [Keep] Generalize PullList module to BlockWorkList Resolved Resolved 3 2.0 2.0 2.0 3705 36 44 74 6 2.0	3858 3858 Review 3705-keep-blockworklist Summary: The pull list was previously implemented as a "PullList" type. Tom asked me to reimplement it as a generic "BlockWorkList" so that the pull list and trash list could share the underlying implementation. This ticket adds BlockWorkList and removes the pull_list package. When Keepstore receives a properly authenticated "PUT /pull" request, it delivers the list of pull requests to the "pullmgr" BlockWorkList. Currently, nothing actually reads and processes pull requests from pullmgr; that will be addressed in #3761. For now, pullmgr just holds on to the pull list and replaces it when given a new one. Tim Pierce 44 0 3705 3 36 -c-a 5 3652 3652 Rewrite PullList as a BlockManager implementation Keepstore should be able to fulfill a pull list asynchronously: once a pull list has been received, send it to a goroutine for fulfillment. The goroutine satisfies each request by trying each server in the order given, and stores the block locally exactly as if a user had delivered it with a PUT request. If all servers for a given request fail, that request is abandoned. Once all requests have been processed, the pull list may be abandoned. There is no need to retry failed requests. The goroutine should check its input channel after handling each pull request. If new data is available (indicating that a new pull list has been sent by Data Manager) it should abandon the current queue and read the new pull list. Tim Pierce 44 0 3705 3 36 -c-a 5 3752 3752 Define BlockManager interfaces and types Tim Pierce 44 0 3705 3 36 -c-a 5
3877 3877 Tom Clegg Tom Clegg 3 Crunch 52 Crunch [Crunch] crunch-dispatch gets large and slow after a verbose job exceeds max log threshold Resolved Resolved 3 0.5 0.5 0.5 3877 36 3 74 1 0.5	3879 3879 Review 3877-log-memory-leak Ward Vandewege 1 0 3877 3 36 -c-a 5
2875 2875 Radhika Chippada Radhika Chippada 72 Workbench 56 Workbench [Workbench] Editable description field for pipeline instances and templates Resolved Resolved 3 1.0 1.0 1.0 2875 36 72 74 2 1.0	3039 3039 Add description attribute to PipelineInstance model See [[Hacking API server]] Radhika Chippada 72 0 2875 3 36 -c-a 5 3191 3191 Add editable title/description fields to pipeline template and pipeline instance pages, just like project page Radhika Chippada 72 0 2875 3 36 -c-a 5 3765 3765 [Workbench] Pipeline instance #show page should not show the _current_ template name/description in a misleading way (the template might have been edited beyond recognition since the instance was created) Radhika Chippada 72 0 2875 3 36 -c-a 5 3867 3867 Add description option to arv-run-pipeline-instance script Radhika Chippada 72 0 2875 3 36 -c-a 5 3868 3868 Add / update integration tests to verify description for pipeline templates, instances, and jobs. Radhika Chippada 72 0 2875 3 36 -c-a 5 3861 3861 Review branch: 2875-pipeline-description Peter Amstutz 47 0 2875 3 36 -c-a 5
3702 3702 Brett Smith Brett Smith 79 Tests 59 Tests [Support] Fix bugs and write tests (the first half of the sprint) Resolved Resolved 3 2.5 2.5 2.5 3702 36 79 74 3 2.5	3730 3730 Fix bugs the first half of the sprint Brett Smith 79 0 3702 3 36 -c-a 5
3654 3654 Radhika Chippada Radhika Chippada 72 [Workbench] Add "combine selected collections" and "combine selected files" actions to projects#show and collections#show pages. Resolved Resolved 3 2.0 2.0 2.0 3654 36 72 74 6 2.0	3796 3796 Add "Combine selected collections and files into a new collection" action to selection dropdown for project show page Radhika Chippada 72 0 3654 3 36 -c-a 5 3797 3797 Add selection dropdown with "Combine selected collections and files into a new collection" action to collections page Radhika Chippada 72 0 3654 3 36 -c-a 5 3798 3798 Add selection dropdown with "Combine selected collections and files into a new collection" action to collection show page Radhika Chippada 72 0 3654 3 36 -c-a 5 3799 3799 Add integration tests to test "Combine selected collections and files into a new collection" action for project show page Radhika Chippada 72 0 3654 3 36 -c-a 5 3800 3800 Add integration tests to test "Combine selected collections and files into a new collection" action for collection show page Radhika Chippada 72 0 3654 3 36 -c-a 5 3839 3839 Display the view icon for files in collections only when the file suffix is one of txt, png, jpg etc. Radhika Chippada 72 0 3654 3 36 -c-a 5 3832 3832 Review branch: 3654-combine-selections Peter Amstutz 47 0 3654 3 36 -c-a 5 3280 3280 [Workbench] Collection#show preview/download buttons use wrong icons and encourage previewing binary data in the browser window In a collection file view, there are two buttons. One tries to view the file in the browser; the other forces a download prompt. * The "in-browser view" button uses the fa-search icon, which is counterintuitive and confusing. * Considering the data that typically goes into Arvados, much of it is not typically viewable in-browser. This makes it seem like both buttons are a download button, which increases confusion. Consider ways to make this clearer. (Maybe only offer inline view for files that reasonably look viewable, like not-too-large TXT, PDF, and image files? Just one idea.) Radhika Chippada 72 0 3654 3 36 +c-a 5
3453 3453 Peter Amstutz Peter Amstutz 47 SDKs 54 SDKs [SDK] "arv keep docker" should support listing images in Keep and placing images in specific projects Resolved Resolved 3 1.0 1.0 1.0 3453 36 47 74 2 1.0	3788 3788 Review 3453-arv-list-docker-images Peter Amstutz 47 0 3453 3 36 -c-a 5
3663 3663 Tim Pierce Tim Pierce 44 SDKs 54 SDKs [SDKs] Python CollectionReader should return at least one byte to caller per block read from Keep. Resolved Resolved 3 0.5 0.5 0.5 3663 36 44 74 1 0.5	3816 3816 Review 3663-collection-reader-performance Tim Pierce 44 0 3663 3 36 -c-a 5 3745 3745 Print output immediately after read Tim Pierce 44 0 3663 3 36 -c-a 5
3586 3586 Peter Amstutz Peter Amstutz 47 API 57 API [API] Job "priority" field defaults to "null", should default to "0" Resolved Resolved 3 0.5 0.5 0.5 3586 36 47 74 1 0.5	3749 3749 Review 3586-job-priority Peter Amstutz 47 0 3586 3 36 -c-a 5 3748 3748 Add migration Peter Amstutz 47 0 3586 3 36 -c-a 5
3644 3644 Peter Amstutz Peter Amstutz 47 [SDK] Fix arv-mount --tags and --groups modes to handle "home" project and #3036 collections changes. Resolved Resolved 3 1.0 1.0 1.0 3644 36 47 74 1 1.0	3804 3804 Review 3644-arv-mount-projects Peter Amstutz 47 0 3644 3 36 -c-a 5 3767 3767 Get feedback from Abram and Sasha on proposed directory hierarchy Peter Amstutz 47 0 3644 3 36 -c-a 5 3728 3728 Handle "home" and name fields correctly. Peter Amstutz 47 0 3644 3 36 -c-a 5
3720 3720 Brett Smith Brett Smith 79 API 57 API [API] Fetching collection record is very slow when manifest is not small. Resolved Resolved 3 0.5 0.5 0.5 3720 36 79 74 1 0.5	3806 3806 Review 3720-big-manifest-performance-wip See #3720 for background. Brett Smith 79 0 3720 3 36 -c-a 5
3704 3704 Brett Smith Brett Smith 79 Workbench 56 Workbench [Workbench] Server-side search fails choosing pipeline inputs with search terms Resolved Resolved 3 0.5 0.5 0.5 3704 36 79 74 1 0.5	3773 3773 Reproduce and diagnose Tom Clegg 3 0 3704 3 36 -c-a 5 3778 3778 Add a test case 0 3704 3 36 -c-a 5 3777 3777 review and merge 3704-broken-filters-param Brett Smith 79 0 3704 3 36 -c-a 5
3637 3637 Radhika Chippada Radhika Chippada 72 [Workbench] Add "copy" operation to selection dropdown on projects#show tabs. Remove copy/move from clippy menu. Resolved Resolved 3 1.0 1.0 1.0 3637 36 72 74 1 1.0	3790 3790 Review branch: 3637-copy-selections Tim Pierce 44 0 3637 3 36 -c-a 5 3776 3776 Pass selected items list through chooser to its action Tom Clegg 3 0 3637 3 36 -c-a 5 3779 3779 Review 3637-selection-through-chooser Radhika Chippada 72 0 3637 3 36 -c-a 5
3763 3763 Peter Amstutz Peter Amstutz 47 Workbench 56 Workbench [Workbench] "Move selected" on collections produces error. Resolved Resolved 3 0.5 0.5 0.5 3763 36 47 74 1 0.5	3764 3764 Review 3763-move-collections-error Tom Clegg 3 0 3763 3 36 -c-a 5
3714 3714 Radhika Chippada Radhika Chippada 72 [Workbench] small bugs in "report bug on the page" feature Resolved Resolved 3 0.5 0.5 0.5 3714 36 72 74 1 0.5	3716 3716 submitting without contents in the "describe the problem" is possible, should not submit a bug and tell the user to put something in the box 0 3714 3 36 -c-a 5 3715 3715 workbench and api server version always render as 6333000 This should be a git hash. 0 3714 3 36 -c-a 5 3746 3746 Review branch: 3714-report-issue-issues Small update to the report issue page to disable the submit button until problem description is entered. Radhika Chippada 72 0 3714 3 36 -c-a 5
3710 3710 Radhika Chippada Radhika Chippada 72 [Workbench] Clipboard "Combine selected collections and files into a new collection" outputs an empty collection Resolved Resolved 3 1.0 1.0 1.0 3710 36 72 74 1 1.0	3814 3814 Review 3710-combine-selections-bug Peter Amstutz 47 0 3710 3 36 -c-a 5
3719 3719 Peter Amstutz Peter Amstutz 47 SDKs 54 SDKs [Crunch] run-command crashes: TypeError: coercing to Unicode ... NoneType found Resolved Resolved 3 0.5 0.5 0.5 3719 36 47 74 1 0.5	3721 3721 Improve error reporting Peter Amstutz 47 0 3719 3 36 -c-a 5 3747 3747 Review 3719-improve-subst-error-reporting Brett Smith 79 0 3719 3 36 -c-a 5
3709 3709 Peter Amstutz Peter Amstutz 47 SDKs 54 SDKs [SDKs] arv-put should return Arvados UUID of collection by default instead of portable data hash. Resolved Resolved 3 1.0 1.0 1.0 3709 36 47 74 6 1.0	3754 3754 Review 3709-arv-put-print-uuid Peter Amstutz 47 0 3709 3 36 -c-a 5 3726 3726 Change arv-put Peter Amstutz 47 0 3709 3 36 -c-a 5 3727 3727 Fix crunch-job Peter Amstutz 47 0 3709 3 36 -c-a 5
3751 3751 Tom Clegg Tom Clegg 3 Crunch 52 Crunch [Crunch] crunchstat does not pass through its stdin to its child process Resolved Resolved 3 0.5 0.5 0.5 3751 36 3 74 1 0.5
3822 3822 Peter Amstutz Peter Amstutz 47 [Crunch] 'arv pipeline run ...' generates 'ERROR: duplicate key value violates unique constraint...' when submitting identical pipeline instance Resolved Resolved 3 1.0 1.0 1.0 3822 36 47 74 1 1.0