Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422024-03-15T20:38:09ZArvados
Redmine Arvados - Bug #21601 (In Progress): fpm virtualenv packages not using branch versions for depende...https://dev.arvados.org/issues/216012024-03-15T20:38:09ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://dev.arvados.org/issues/19744#note-30">https://dev.arvados.org/issues/19744#note-30</a></p>
<p>The python3-arvados-cwl-runner_2.8.0~dev20240314145937-1_amd64.deb package has arvados-python-client 2.7.1 and crunchstat-summary 2.7.1, when it should have the dev versions from the same commit.</p>
<p>I went back and looked at earlier packages: python3-arvados-cwl-runner_2.7.1~rc3-1_amd64.deb has arvados-python-client 2.7.1rc3 (as expected) and python3-arvados-cwl-runner_2.7.0~dev20230908133938-1_amd64.deb has arvados-python-client 2.7.0.dev20230908133938 (also as expected).</p>
<p>My current theory is that this behavior got lost in the changes made in 20846-package-build-fixes, but I need to find out how it worked before.</p> Arvados - Bug #21434 (New): Follow up fix to schema saladhttps://dev.arvados.org/issues/214342024-01-31T16:46:46ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://github.com/common-workflow-language/schema_salad/issues/766">https://github.com/common-workflow-language/schema_salad/issues/766</a></p> Arvados - Feature #20982 (In Progress): Write a blog post with highlights of new releasehttps://dev.arvados.org/issues/209822023-09-20T15:33:51ZPeter Amstutzpeter.amstutz@curii.comArvados - Feature #19982 (In Progress): Ability to know when a container died because of spot ins...https://dev.arvados.org/issues/199822023-01-25T17:04:12ZPeter Amstutzpeter.amstutz@curii.com
<p>New arvados-cwl-runner behavior when spot instances are enabled</p>
<ul>
<li>When submitting spot instance, don't retry</li>
<li>Ability to detect when a container failed due to reclaimed spot instance (<a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: Detect and log spot instance interruption notices (Resolved)" href="https://dev.arvados.org/issues/19961">#19961</a>)</li>
<li>Exit code to indicate workflow failed due to spot instance</li>
<li>Option to automatically re-submit as reserved instance</li>
</ul> Tapestry - Feature #8707 (In Progress): Arvados job: download data from remote site into Keephttps://dev.arvados.org/issues/87072016-03-15T21:35:13ZTom Cleggtom@curii.com
<p>...to satisfy an API request like <a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: Accept authenticated API calls from data providers to add datasets to a public profile (Resolved)" href="https://dev.arvados.org/issues/8688">#8688</a></p>
<a name="Implementation"></a>
<h2 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h2>
<p>One task per requested file -- this avoids retrying <em>everything</em> whenever one file fails</p>
<p>Use writable FUSE (task output dir)</p>
<p>Run wget or curl, probably with some sort of batch-progress flag</p> Tapestry - Feature #8697 (In Progress): Check for finished download/processing jobshttps://dev.arvados.org/issues/86972016-03-15T17:50:45ZTom Cleggtom@curii.com
<p>Implement as a long-running process that polls and then waits {configurable} seconds</p>
When a download job succeeds, create a UserFile record for each file in the output collection that
<ul>
<li>has location == output of download job</li>
<li>belongs to the appropriate participant</li>
<li>appears on the participant's public profile page</li>
<li>appears on the participant's "data files" list</li>
</ul>
When a processing job succeeds,
<ul>
<li>update report_content_locator = job output</li>
<li>send email to user</li>
<li>add entry to user log</li>
</ul>
When any job fails,
<ul>
<li>send email to site admins</li>
</ul>
<a name="Implementation"></a>
<h2 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h2>
New UserFile attributes
<ul>
<li><pre>
t.integer "study_id"
</pre></li>
</ul> Tapestry - Feature #8689 (In Progress): Trigger an arvados job to retrieve the data and import it...https://dev.arvados.org/issues/86892016-03-15T00:28:10ZWard Vandewegeward@curii.com
When the API call described in <a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: Accept authenticated API calls from data providers to add datasets to a public profile (Resolved)" href="https://dev.arvados.org/issues/8688">#8688</a> is received, that should trigger
<ul>
<li>an email to the participant saying the transfer has been requested/started</li>
<li>a user log entry to that effect</li>
<li>an Arvados job that will ingest the data (when that is done, Tapestry needs to get notified or notice; see <a class="issue tracker-2 status-2 priority-4 priority-default parent" title="Feature: Check for finished download/processing jobs (In Progress)" href="https://dev.arvados.org/issues/8697">#8697</a>)</li>
</ul> GET-Evidence - Bug #5883 (New): migrate to updated oauthhttps://dev.arvados.org/issues/58832015-05-01T18:31:33ZAbram Connellyabram.connelly@gmail.com
<p>There is a warning on that says OAuth2 is going away. We need to migrate the authentication on GET-Evidence to make sure login is still possible.</p> GET-Evidence - Feature #5873 (New): GET-Evidence pipeline code should be publichttps://dev.arvados.org/issues/58732015-04-30T22:55:20ZAbram Connellyabram.connelly@gmail.com
<p>The GET-Evidence pipeline should be put under a public git repository.</p>
<p>Currently the GET-Evidence repository is under my (Abram Connelly) private Arvados git repository. This should probably remain but some hooks should be added somewhere so that the repository is pushed to a public Curoverse repository.</p> GET-Evidence - Bug #5872 (In Progress): 23andMe report pipeline is brokenhttps://dev.arvados.org/issues/58722015-04-30T22:52:53ZAbram Connellyabram.connelly@gmail.com
<p>When participants upload their 23andMe data, this kicks off a pipeline run through Arvados (now) through the same code path that the dataset submission goes through from Tapestry. The pipeline is failing to correctly generate reports for 23andMe data.</p>
<p>Porting the pipeline over to Arvados, the format for the input was changed and that's why the pipelines are failing on 23andMe data. I think the best thing to do going forward is to be more liberal about what input collection is specified so that 23andMe reports along with older reports can be run (or re-run) without having to worry about updating to the new format.</p>
<p>The old format just specifies a portable data hash. The new format expects <PDH>/<filename>.</p> Arvados - Idea #3304 (New): [Workbench] Add checkboxes to multi-type chooser modal to filter by o...https://dev.arvados.org/issues/33042014-07-18T21:04:07ZTom Cleggtom@curii.comArvados - Bug #3257 (In Progress): [Workbench] Hard to find user agreement after it has been signedhttps://dev.arvados.org/issues/32572014-07-16T14:33:47ZPjotr Prins
<p>After logging in the user agreement should be available in a straightforward way. Add an item to one of the dropdown menus at the top right.</p>
<p>There's a plan to consolidate these menus based on feedback from our UX/UI person. Tom can tell you more about that.</p>
<p>The user agreement is just a collection stored in Keep. Users can actually see it if they go to their 'Links' page and look for the entry with link class 'Signature'. But of course that's not user friendly.</p> Tapestry - Idea #2850 (New): Comprehensive "Sectioning" of Tapestryhttps://dev.arvados.org/issues/28502014-05-22T13:24:56ZPhil Hodgsonphil@curoverse.com
<p>Continuing the work done as proof of concept in Story <a class="issue tracker-6 status-3 priority-4 priority-default closed parent" title="Idea: Make it possible to disable/enable "sections" (Resolved)" href="https://dev.arvados.org/issues/2503">#2503</a>, a comprensive sectioning off of Tapestry will force us to assess what parts of Tapestry are out of use and how the pieces fit together conceptually. The ideal will be a small set of "sections", where each one is optional.</p>
<p>The methodology is:</p>
<ol>
<li>Identify "modules" and assign config.yml keys</li>
<li>Reference configuration when rendering partials and cells</li>
<li>Reference configuration when authorizing in controllers</li>
</ol>
<p>At each point we risk encountering questions of architecture because it will be evident that some reorganization of the code being done at the same time as adding these "sections" would make sense. Therefore this is a non-trivial but important story overall.</p> Tapestry - Idea #2552 (New): Configurable mini-consent form and validationshttps://dev.arvados.org/issues/25522014-04-08T12:13:43ZPhil Hodgsonphil@curoverse.comTapestry - Idea #2544 (New): The "site override" feature is testedhttps://dev.arvados.org/issues/25442014-04-07T13:02:53ZPhil Hodgsonphil@curoverse.com