Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422023-12-21T16:55:13ZArvados
Redmine Arvados - Bug #21314 (New): a-d-c should cancel a container if it can't be loadedhttps://dev.arvados.org/issues/213142023-12-21T16:55:13ZTom Cleggtom@curii.com
<p>If a container's "mounts" field is invalid, a-d-c logs this, and keeps trying.</p>
<code class="json syntaxhl"><span class="p">{</span><span class="nl">"ClusterID"</span><span class="p">:</span><span class="s2">"irdev"</span><span class="p">,</span><span class="nl">"ContainerUUID"</span><span class="p">:</span><span class="s2">"<a href="https://arvadosapi.com/xxxxx-dz642-xxxxxxxxxxxxxxx">xxxxx-dz642-xxxxxxxxxxxxxxx</a>"</span><span class="p">,</span><span class="nl">"PID"</span><span class="p">:</span><span class="mi">2037423</span><span class="p">,</span><span class="nl">"error"</span><span class="p">:</span><span class="s2">"json: cannot unmarshal array into Go struct field Container.mounts of type arvados.Mount"</span><span class="p">,</span><span class="nl">"level"</span><span class="p">:</span><span class="s2">"warning"</span><span class="p">,</span><span class="nl">"msg"</span><span class="p">:</span><span class="s2">"error getting mounts"</span><span class="p">,</span><span class="nl">"time"</span><span class="p">:</span><span class="s2">"2023-12-13T20:34:41.064140517Z"</span><span class="p">}</span><span class="w">
</span></code>
<p>In this situation, the offending container should be cancelled.</p> Arvados - Feature #19564 (In Progress): AutoReloadConfig usable in productionhttps://dev.arvados.org/issues/195642022-09-21T15:55:52ZPeter Amstutzpeter.amstutz@curii.com
<p>For all services:</p>
<p>If the config file on disk changes, it checks that the config file is valid (config-check), then (if it is valid) restarts the service.</p>
<p>If the config file is not valid, that is reported via health check (possibly already implemented?)</p>
<p>Ideally, restarting the service means allowing existing in-flight requests to complete, while new requests use the config.</p>
<p>When running under "arvados-server boot", this is currently handled by a supervisor process. When not running under "arvados-server boot" services need to monitor the config file & be able to reload/restart themselves.</p> Arvados - Idea #18997 (New): ability to migrate arvbox to arvados-server boot dev modehttps://dev.arvados.org/issues/189972022-04-13T14:33:12ZPeter Amstutzpeter.amstutz@curii.com
<p>arvados-server boot dev mode should be able to support all these commands:</p>
<pre>
Arvados-in-a-box https://doc.arvados.org/install/arvbox.html
start|run <config> [tag] start arvbox container
stop stop arvbox container
restart <config> stop, then run again
status print some information about current arvbox
ip print arvbox docker container ip address
host print arvbox published host
shell enter shell as root
ashell enter shell as 'arvbox'
psql enter postgres console
open open arvbox workbench in a web browser
root-cert get copy of root certificate
update <config> stop, pull latest image, run
build <config> build arvbox Docker image
reboot <config> stop, build arvbox Docker image, run
rebuild <config> build arvbox Docker image, no layer cache
checkpoint create database backup
restore restore checkpoint
hotreset reset database and restart API without restarting container
reset delete arvbox arvados data (be careful!)
destroy delete all arvbox code and data (be careful!)
log <service> tail log of specified service
ls <options> list directories inside arvbox
cat <files> get contents of files inside arvbox
pipe run a bash script piped in from stdin
sv <start|stop|restart> <service>
change state of service inside arvbox
clone <from> <to> clone dev arvbox
adduser <username> <email> [password]
add a user login
removeuser <username>
remove user login
listusers list user logins
</pre> Arvados - Idea #17472 (In Progress): Architecture for displaying container metrics in workbenchhttps://dev.arvados.org/issues/174722021-03-12T20:56:15ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://docs.google.com/document/d/15JDlp1ZzxHGxsV-y1gmNd0sjtuEYdIQz-YgTOqeyLqs/edit?usp=sharing">https://docs.google.com/document/d/15JDlp1ZzxHGxsV-y1gmNd0sjtuEYdIQz-YgTOqeyLqs/edit?usp=sharing</a></p> Arvados - Feature #17185 (New): [adc] add broken node metricshttps://dev.arvados.org/issues/171852020-12-02T17:42:15ZWard Vandewegeward@curii.com
<p>Add a broken-node metric</p>
<p>(counter) VMs that are determined to be "broken nodes"</p>
<p>Add a label to separate VMs marked as broken before the first container is started on them (likely boot problem) and after (likely container related problem).</p>
<p>Note that we already have a boot outcome metric. Make sure that we increment the broken node counter ("before first container" label) when we have a boot outcome == failed, though not in the timeout case.</p> Tapestry - Feature #8707 (In Progress): Arvados job: download data from remote site into Keephttps://dev.arvados.org/issues/87072016-03-15T21:35:13ZTom Cleggtom@curii.com
<p>...to satisfy an API request like <a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: Accept authenticated API calls from data providers to add datasets to a public profile (Resolved)" href="https://dev.arvados.org/issues/8688">#8688</a></p>
<a name="Implementation"></a>
<h2 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h2>
<p>One task per requested file -- this avoids retrying <em>everything</em> whenever one file fails</p>
<p>Use writable FUSE (task output dir)</p>
<p>Run wget or curl, probably with some sort of batch-progress flag</p> Tapestry - Feature #8697 (In Progress): Check for finished download/processing jobshttps://dev.arvados.org/issues/86972016-03-15T17:50:45ZTom Cleggtom@curii.com
<p>Implement as a long-running process that polls and then waits {configurable} seconds</p>
When a download job succeeds, create a UserFile record for each file in the output collection that
<ul>
<li>has location == output of download job</li>
<li>belongs to the appropriate participant</li>
<li>appears on the participant's public profile page</li>
<li>appears on the participant's "data files" list</li>
</ul>
When a processing job succeeds,
<ul>
<li>update report_content_locator = job output</li>
<li>send email to user</li>
<li>add entry to user log</li>
</ul>
When any job fails,
<ul>
<li>send email to site admins</li>
</ul>
<a name="Implementation"></a>
<h2 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h2>
New UserFile attributes
<ul>
<li><pre>
t.integer "study_id"
</pre></li>
</ul> Arvados - Task #5861 (New): Allow editing pipeline parameters in workbenchhttps://dev.arvados.org/issues/58612015-04-30T15:39:10ZTom Cleggtom@curii.comArvados - Task #3192 (New): Clarify requirements/implementationhttps://dev.arvados.org/issues/31922014-07-04T15:53:06ZTom Cleggtom@curii.comArvados - Task #3189 (New): [Blocker] Clarify design/requirementshttps://dev.arvados.org/issues/31892014-07-04T15:20:47ZTom Cleggtom@curii.comGET-Evidence - Feature #925 (In Progress): Add EVS frequency datahttps://dev.arvados.org/issues/9252012-05-16T15:30:32ZTom Cleggtom@curii.com
<p>Raw data at <a class="external" href="http://evs.gs.washington.edu/EVS/">http://evs.gs.washington.edu/EVS/</a></p>
<p>Pre-processed data with GET-Evidence AA predictions at <a class="external" href="http://mball.personalgenomes.org/~mball/allele_freq_data/ESP5400_getev-aa-changes_allele_freqs.txt">http://mball.personalgenomes.org/~mball/allele_freq_data/ESP5400_getev-aa-changes_allele_freqs.txt</a></p> GET-Evidence - Bug #495 (New): Allele frequency missing for rs41281314https://dev.arvados.org/issues/4952010-05-12T16:03:48ZMadeleine Ballmpball@gmail.com
<p>This is in 1000Genomes preliminary data according to dbSNP:<br /><a class="external" href="http://www.ncbi.nlm.nih.gov/projects/SNP/snp_ss.cgi?ss=ss130398918">http://www.ncbi.nlm.nih.gov/projects/SNP/snp_ss.cgi?ss=ss130398918</a></p>
<p>But we don't have the allele frequency showing up in GET-Evidence page: <br /><a class="external" href="http://evidence.personalgenomes.org/CDH23-T1209A">http://evidence.personalgenomes.org/CDH23-T1209A</a></p>
<p>(Allele frequency of 0.120 in pilot.1.YRI - WEST AFRICA)</p> GET-Evidence - Feature #488 (New): Allow non-nsSNP variantshttps://dev.arvados.org/issues/4882010-05-08T16:02:23ZTom Cleggtom@curii.com
Need to stop restricting database to nsSNP variants.
<ul>
<li>Support lookup/create by dbSNP id</li>
<li>Support lookup/create by {chromosome, position, ref_seq, variant_seq}</li>
</ul>
Considerations:
<ul>
<li>Is "strand" helpful? Or should "chr1:1234:-:A:C" just be described as "chr1:1234:T:G"?</li>
<li>How to deal with different references?
<ul>
<li>Include a reference identifier in database key -- "hg18:chr:pos:refseq:varseq" </li>
<li>Extend database schema to include multiple identifiers per locus, so "hg18:chr1:1234" and "hg19:chr1:1235" can map to the same variant ID</li>
</ul>
</li>
<li>Ensure only one page per variant. E.g., if rs1234 maps to chr1:1234:T:G and causes ABCD-A1T, then there should only be one GET-Evidence page regardless of which order those identifiers are looked up / used to create a page.</li>
</ul> GET-Evidence - Feature #482 (New): add “affects self (hom or dominant)” checkbox (vs “affects off...https://dev.arvados.org/issues/4822010-05-07T18:51:38ZTom Cleggtom@curii.com
<p>add “affects self (hom or dominant)” checkbox (vs “affects offspring”) on result page</p> GET-Evidence - Feature #476 (New): allow “lab members” group to upload/view publication PDFshttps://dev.arvados.org/issues/4762010-05-07T17:31:32ZTom Cleggtom@curii.com
<p>allow “lab members” group to upload/view publication PDFs</p>