Arvados

Story

Resolved

Closed

Sprint Impediments

3410

[API] Rename collection "desired replication" attributes, add model constraints

Tom Clegg

impediments

-c-a

5072 5072 Brett Smith Brett Smith 79 [Documentation] (recurring) Refresh user documentation Resolved Resolved 3 1.0 1.0 1.0 5072 36 79 85 2 1.0		5164 5164 Review branch: 4926-fix-imagesizes The varying image sizes were causing the image carousel to bounce around in size. I re-cropped all the images to a consistent size. Nancy Ouyang 235 0 5072 5 36 -c-a 5
2659 2659 Radhika Chippada Radhika Chippada 72 [Workbench] Anonymous user can access publicly shared data using Workbench and curl Resolved Resolved 3 2.0 2.0 2.0 2659 36 72 85 6 2.0	5125 5125 Review branch: 2659-anonymous-share-projects Tom Clegg 3 0 2659 3 36 -c-a 5
5071 5071 Radhika Chippada Radhika Chippada 72 [Support] Fix bugs and write tests (second half) Resolved Resolved 3 5.0 5.0 5.0 5071 36 72 85 3 5.0
4138 4138 Brett Smith Brett Smith 79 Node Manager 66 Node Manager [Node Manager] Support Google Cloud Platform in node manager Resolved Resolved 3 1.0 1.0 1.0 4138 36 79 85 2 1.0	4880 4880 get credentials for testing Ward Vandewege 1 0 4138 3 36 -c-a 5 4203 4203 Implement arvnodeman.computenode.googlecloud Implement the arvnodeman.computenode.googlecloud module, providing a ComputeNodeDriver that can be used to interact with Google Cloud services. Tim Pierce 44 0 4138 3 36 -c-a 5 5222 5222 Review 4138-node-manager-gce-wip Tom Clegg 3 0 4138 3 36 -c-a 5	4580 4580 Review 4138-node-manager-google-cloud Ready to review at commit:ab8acfe Some implementation notes: * This code has not been tested against a live GCE account. We can test when we get GCE credentials that allow us to spin nodes up and down. * GCE does not permit filtering instances on tags the way AWS does, so filtering has not been implemented. Clients will need to list all instances and do their own filtering. * GCE does implement a 'tag' concept, but it is a flat keyword list rather than a key/value structure. When a node is synced, the Arvados hostname is updated on the node by setting a tag 'hostname-arvados.fqdn.name'. ** The underlying GCE library supports a @setMetadata@ call that should allow us to use metadata to track this in a cleaner way, but the libcloud API doesn't support this call yet. * GCE node access is controlled with "service accounts" rather than security groups. These can be defined in gce.example.cfg by setting the @service_account@ setting to a JSON string with the desired service account list. For reviewing: the GCE drivers and tests were developed by ripping out everything in the EC2 drivers that doesn't look like GCE. Accordingly, I strongly recommend: <pre> diff services/nodemanager/arvnodeman/computenode/driver/ec2.py services/nodemanager/arvnodeman/computenode/driver/gce.py diff services/nodemanager/tests/test_computenode_driver_ec2.py services/nodemanager/tests/test_computenode_driver_gce.py diff services/nodemanager/doc/ec2.example.cfg services/nodemanager/doc/gce.example.cfg </pre> Brett Smith 79 0 4138 5 36 -c-a 5
5194 5194 [Documentation] User guide doesn't link to or acknowledge new "Getting Started" section Resolved Resolved 3 5194 36 85 1	5243 5243 Review branch 5194-quickfix-disambiguate-gettingstarted-sections This is a quick fix only. I disambiguated the new "getting started" section from the "getting started" subsection in the "user guide" section. I added "next steps" to "getting started" guide Radhika Chippada 72 0 5194 3 36 -c-a 5
5090 5090 Documentation 58 Documentation [Documentation] Add some direct links on doc.arvados.org to the most useful redmine pages Resolved Resolved 3 0.5 0.5 0.5 5090 36 85 2 0.5	5196 5196 Review branch 5090-doc-to-homepage Changed navbars, homepage, to let /doc/index.html function as homepage. Specifically: homepage title is "Arvados", top navbar links to "Developer Site" instead of arvados.org, homepage "license" footer text appears on right side to even out the two columns, various links added. I put 5092 instead of 5090 by accident in the branch name and am not sure how to fix that. 0 5090 3 36 -c-a 5 5440 5440 [Documentation] Revert doc-to-homepage 1. Title -> Arvados \| Documentation 2. Top left link: Arvados -> Arvados Docs 3. Top right link: Developer Site -> arvados.org Anything else? 0 5090 3 36 -c-a 5
4759 4759 Radhika Chippada Radhika Chippada 72 [API] greater than filter is treated as greater than or equals for modification time. Resolved Resolved 3 0.5 0.5 0.5 4759 36 72 85 1 0.5	5213 5213 Review branch: 4759-timestamp-precision Radhika Chippada 72 0 4759 3 36 -c-a 5
5220 5220 Radhika Chippada Radhika Chippada 72 [Workbench] for collections shared anonymously, do not show the 'Sharing and Permissions' box Resolved Resolved 3 0.5 0.5 0.5 5220 36 72 85 1 0.5	5225 5225 Review branch: 5220-no-sharing-box-for-anon-user Ward Vandewege 1 0 5220 3 36 -c-a 5
5189 5189 Radhika Chippada Radhika Chippada 72 [Workbench] Manage account does not show repositories directly owned by user Resolved Resolved 3 0.5 0.5 0.5 5189 36 72 85 1 0.5	5223 5223 Review branch: 5189-manage-acct-shows-own-repos Radhika Chippada 72 0 5189 3 36 -c-a 5
3410 3410 Tom Clegg Tom Clegg 3 API 57 API [API] Rename collection "desired replication" attributes, add model constraints Resolved Resolved 3 1.0 1.0 1.0 3410 36 3 85 6 1.0	5100 5100 Write migration Tom Clegg 3 0 3410 3 36 -c-a 5 5101 5101 Write tests Tom Clegg 3 0 3410 3 36 -c-a 5 5165 5165 Review 3410-replication-attrs commit:d302307 Radhika Chippada 72 0 3410 3 36 -c-a 5
4464 4464 Brett Smith Brett Smith 79 Workbench 56 Workbench [Workbench] Collections tab loads forever on a specific project Resolved Resolved 3 0.5 0.5 0.5 4464 36 79 85 1 0.5	5140 5140 Review 4464-api-project-contents-wip Radhika Chippada 72 0 4464 3 36 -c-a 5
5095 5095 Peter Amstutz Peter Amstutz 47 FUSE 69 FUSE [FUSE] arv-mount takes up too much memory and occassionally crashes when listing large 'home' directory Resolved Resolved 3 2.0 2.0 2.0 5095 36 47 85 1 2.0	5216 5216 Review 5095-fuse-ls-takes-forever Peter Amstutz 47 0 5095 3 36 -c-a 5
5186 5186 Radhika Chippada Radhika Chippada 72 [Workbench] Collection viewing broken for collections with empty (but non-nil) properties attribute Resolved Resolved 3 0.5 0.5 0.5 5186 36 72 85 1 0.5	5221 5221 Review branch: 5186-collection-with-empty-properties Radhika Chippada 72 0 5186 3 36 -c-a 5
5187 5187 Workbench 56 Workbench [Workbench] PNG is not available/is empty when it really isnt Resolved Resolved 3 5187 36 85 1
5011 5011 Tom Clegg Tom Clegg 3 SDKs 54 SDKs [SDKs] arv-put accepts --replication argument (saved with collection record) Resolved Resolved 3 0.5 0.5 0.5 5011 36 3 85 2 0.5	5121 5121 Add replication=X kwarg to CollectionWriter and friends Tom Clegg 3 0 5011 3 36 -c-a 5 5120 5120 Add --replication arg to arv-put Tom Clegg 3 0 5011 3 36 -c-a 5 5122 5122 Update docs Tom Clegg 3 0 5011 3 36 -c-a 5 5123 5123 Review 5011-arv-put-replication Tom Clegg 3 0 5011 3 36 -c-a 5 5124 5124 Use server-provided default replication level as default in arv-put and CollectionWriter classes Tom Clegg 3 0 5011 3 36 -c-a 5 5206 5206 Review 5011-thread-safe-test Tom Clegg 3 0 5011 3 36 -c-a 5
5200 5200 Tom Clegg Tom Clegg 3 Workbench 56 Workbench [Workbench] on public project, collection downloads require login Resolved Resolved 3 0.5 0.5 0.5 5200 36 3 85 1 0.5	5205 5205 Review 5200-anonymous-download 0 5200 3 36 -c-a 5
5037 5037 Tom Clegg Tom Clegg 3 SDKs 54 SDKs [SDKs] Improve Python SDK thread safety and document pitfalls Resolved Resolved 3 0.5 0.5 0.5 5037 36 3 85 1 0.5	5117 5117 Review 5037-python-sdk-thread-safe Tom Clegg 3 0 5037 3 36 -c-a 5 5116 5116 Remove connection pooling feature from arvados.api() Tom Clegg 3 0 5037 3 36 -c-a 5 5166 5166 Remove cache=False from Python SDK clients (bump version requirements instead) Tom Clegg 3 0 5037 3 36 -c-a 5 5170 5170 Review 5037-nonocache Tom Clegg 3 0 5037 3 36 -c-a 5
5030 5030 Tom Clegg Tom Clegg 3 Workbench 56 Workbench [Workbench] Stats graph display for job without stats is confusing Resolved Resolved 3 0.5 0.5 0.5 5030 36 3 85 1 0.5	5168 5168 Hide graph div until data points arrive Tom Clegg 3 0 5030 3 36 -c-a 5 5172 5172 Review 5030-hide-graph-until-data Tom Clegg 3 0 5030 3 36 -c-a 5
4904 4904 Peter Amstutz Peter Amstutz 47 Sample Pipelines 55 Sample Pipelines [SDKs] Use websockets to restart/reconfigure a web service running on a VM whenever the content of a project changes Resolved Resolved 3 2.0 2.0 2.0 4904 36 47 85 6 2.0	4947 4947 Provide example to science team Peter Amstutz 47 0 4904 3 36 -c-a 5 5018 5018 Review 4904-arv-web Peter Amstutz 47 0 4904 3 36 -c-a 5 4946 4946 Fix project directory refresh bug in FUSE Peter Amstutz 47 0 4904 3 36 -c-a 5
4839 4839 Tom Clegg Tom Clegg 3 Node Manager 66 Node Manager [Node Manager] Should look at Arvados node's crunch_worker_state, not info['slurm_state'] Resolved Resolved 3 0.5 0.5 0.5 4839 36 3 85 1 0.5	5169 5169 Heed crunch_worker_state Tom Clegg 3 0 4839 3 36 -c-a 5 5203 5203 Review 4839-worker-state Tom Clegg 3 0 4839 3 36 -c-a 5
4988 4988 Brett Smith Brett Smith 79 SDKs 54 SDKs [SDKs] Fix incompatibility between Python packages and newer virtualenv Closed Closed 5 0.5 0.5 0.5 4988 36 79 85 1 0.5
4951 4951 Radhika Chippada Radhika Chippada 72 Workbench 56 Workbench [Workbench] When a user does not have access to a VM, provide a way to contact the site admin to request access. Resolved Resolved 3 1.0 1.0 1.0 4951 36 72 85 2 1.0	5144 5144 Review branch: 4951-request-vm Radhika Chippada 72 0 4951 3 36 -c-a 5
5110 5110 Radhika Chippada Radhika Chippada 72 Workbench 56 Workbench [Workbench] Use full text search instead of 'ilike' when searching with a user-entered query Resolved Resolved 3 1.0 1.0 1.0 5110 36 72 85 2 1.0	5161 5161 Review branch: 4523-full-text-search This has the server-side full text search implementation. Once this branch is merged into master, the workbench can be updated to use full text search. Tom Clegg 3 0 5110 3 36 -c-a 5 5184 5184 Review branch: 5110-workbench-full-text-search Radhika Chippada 72 0 5110 3 36 -c-a 5
5185 5185 Radhika Chippada Radhika Chippada 72 [Workbench] Project lists all collections as zero size as of 6bf9ae1 Resolved Resolved 3 0.5 0.5 0.5 5185 36 72 85 1 0.5	5195 5195 Review branch: 5185-collections-tab-zero-size-issue Tom Clegg 3 0 5185 3 36 -c-a 5
5183 5183 Radhika Chippada Radhika Chippada 72 Workbench 56 Workbench [Workbench] Job reuse modal test fails intermittently Resolved Resolved 3 5183 36 72 85 1
4958 4958 Tom Clegg Tom Clegg 3 Workbench 56 Workbench [Workbench] Show an alert message below top nav when running in an unsupported browser. Resolved Resolved 3 0.5 0.5 0.5 4958 36 3 85 1 0.5	5171 5171 Check UA string and JS features, throw alert if missing. Tom Clegg 3 0 4958 3 36 -c-a 5 5174 5174 Add test case Tom Clegg 3 0 4958 3 36 -c-a 5 5173 5173 Review 4958-old-browser-alert commit:b48e7f0 Tom Clegg 3 0 4958 3 36 -c-a 5
5160 5160 Brett Smith Brett Smith 79 API 57 API [API] Non-admin can't share a project publicly on qr1hi Resolved Resolved 3 0.5 0.5 0.5 5160 36 79 85 1 0.5	5163 5163 Review 5160-share-with-anonymous-group-wip Brett Smith 79 0 5160 3 36 -c-a 5
5175 5175 Tom Clegg Tom Clegg 3 SDKs 54 SDKs [SDKs] Fix Python package versions so they can be uploaded to pypi Resolved Resolved 3 0.5 0.5 0.5 5175 36 3 85 1 0.5	5177 5177 Review 5175-pep-440 (arvados + arvados-dev) commit:f8067dd Tom Clegg 3 0 5175 3 36 -c-a 5
5070 5070 Brett Smith Brett Smith 79 [Support] Fix bugs and write tests (first half) Resolved Resolved 3 5.0 5.0 5.0 5070 36 79 85 3 5.0
3021 3021 Tests 59 Tests [Tests] Make jenkins tests run faster. Resolved Resolved 3 1.0 1.0 1.0 3021 36 85 6 1.0	4971 4971 Review 3021-test-perf on arvados-dev Tom Clegg 3 0 3021 3 36 -c-a 5 4896 4896 Review 3021-more-phantomjs Tom Clegg 3 0 3021 3 36 -c-a 5 4895 4895 Convert selenium tests to phantomjs where possible Tom Clegg 3 0 3021 3 36 -c-a 5 4902 4902 Use a fixture instead of calling "run a pipeline" in the set of 8 "rerun pipeline instance" tests Radhika Chippada 72 0 3021 3 36 -c-a 5 4974 4974 Review 3021-workbench-perf See #3021 note-19 Tom Clegg 3 0 3021 3 36 -c-a 5 4975 4975 Review 3021-api-performance See #3021 note-20 Tim Pierce 44 0 3021 3 36 -c-a 5 5092 5092 Review branch: 4902-use-fixture-to-rerun-pipelines 0 3021 3 36 -c-a 5 4986 4986 Refactor run_test_server.py to use API database reset rather than restarting the server Tom Clegg 3 0 3021 3 36 -c-a 5 5130 5130 Review 3021-leave-api-running (arvados-dev) This can't be merged/used without 3021-leave-api-running (arvados) Tom Clegg 3 0 3021 3 36 -c-a 5 5129 5129 Review 3021-leave-api-running (arvados) Tom Clegg 3 0 3021 3 36 -c-a 5
5141 5141 Workbench 56 Workbench [Workbench] Should permit all numeric values for numeric pipeline inputs, not just integers Resolved Resolved 3 0.5 0.5 0.5 5141 36 85 1 0.5	5143 5143 Review 5141-workbench-numeric-inputs-wip Peter Amstutz 47 0 5141 3 36 -c-a 5
5131 5131 Brett Smith Brett Smith 79 Crunch 52 Crunch [Crunch] run-command passes escape backslashes through Resolved Resolved 3 0.5 0.5 0.5 5131 36 79 85 1 0.5	5132 5132 Review 5131-run-command-consume-backslashes-wip Peter Amstutz 47 0 5131 3 36 -c-a 5
5069 5069 Brett Smith Brett Smith 79 Documentation 58 Documentation [Documentation] (recurring) Write blog post in first three days of sprint Resolved Resolved 3 0.5 0.5 0.5 5069 36 79 85 6 0.5
4562 4562 Brett Smith Brett Smith 79 Documentation 58 Documentation [Documentation] Wiki page: explain appropriate use cases for arv-run vs. run-command vs. writing your own crunch script. Resolved Resolved 3 1.0 1.0 1.0 4562 36 79 85 1 1.0	5015 5015 Review branch 4562-crunch-tools-docs-wip At https://arvados.org/projects/arvados/wiki/How_to_submit_compute_work_to_Arvados Brett Smith 79 0 4562 3 36 -c-a 5
4922 4922 Brett Smith Brett Smith 79 Crunch 52 Crunch [Crunch] google-api-python-client fails to install Resolved Resolved 3 0.5 0.5 0.5 4922 36 79 85 1 0.5		5012 5012 Confirm fix Rerun job once deployed on qr1hi Tim Pierce 44 0 4922 5 36 -c-a 5
5135 5135 Brett Smith Brett Smith 79 Workbench 56 Workbench [Workbench] Ensure modal button order is consistent Resolved Resolved 3 0.5 0.5 0.5 5135 36 79 85 1 0.5