Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422024-03-27T16:11:09ZArvados
Redmine Arvados - Task #21631 (New): Reviewhttps://dev.arvados.org/issues/216312024-03-27T16:11:09ZPeter Amstutzpeter.amstutz@curii.comArvados - Bug #21434 (New): Follow up fix to schema saladhttps://dev.arvados.org/issues/214342024-01-31T16:46:46ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://github.com/common-workflow-language/schema_salad/issues/766">https://github.com/common-workflow-language/schema_salad/issues/766</a></p> Arvados - Task #20510 (In Progress): Reviewhttps://dev.arvados.org/issues/205102023-05-10T20:46:40ZPeter Amstutzpeter.amstutz@curii.comArvados - Idea #20435 (In Progress): CWL user guide releasehttps://dev.arvados.org/issues/204352023-04-26T16:03:36ZPeter Amstutzpeter.amstutz@curii.comArvados - Support #20053 (In Progress): schema-salad codegen returns line numbershttps://dev.arvados.org/issues/200532023-02-01T18:12:43ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://github.com/common-workflow-language/schema_salad/pull/647">https://github.com/common-workflow-language/schema_salad/pull/647</a></p> Arvados - Feature #19982 (In Progress): Ability to know when a container died because of spot ins...https://dev.arvados.org/issues/199822023-01-25T17:04:12ZPeter Amstutzpeter.amstutz@curii.com
<p>New arvados-cwl-runner behavior when spot instances are enabled</p>
<ul>
<li>When submitting spot instance, don't retry</li>
<li>Ability to detect when a container failed due to reclaimed spot instance (<a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: Detect and log spot instance interruption notices (Resolved)" href="https://dev.arvados.org/issues/19961">#19961</a>)</li>
<li>Exit code to indicate workflow failed due to spot instance</li>
<li>Option to automatically re-submit as reserved instance</li>
</ul> Arvados - Feature #16316 (In Progress): a-c-r handles resource range requests (especially CPU) an...https://dev.arvados.org/issues/163162020-04-08T12:48:43ZPeter Amstutzpeter.amstutz@curii.com
<p>Implement a version of <a href="https://github.com/common-workflow-language/cwltool/blob/72e4ffb7814ec846c12ec15df82a4243654fe053/cwltool/executors.py#L286" class="external">select_resources</a> for Arvados.</p>
<p>You can get a dictionary of instance types with this:</p>
<p><code>api.config()["InstanceTypes"]</code></p>
<p>The select_resources method should, at minimum, accept a range of CPU core values (e.g. coresMin: 4, coresMax: 16) and then check the available InstanceTypes and assign the greatest core count available. For example, if the system is only configured with 2, 4, and 8 core nodes, it should assign 8 cores since it is in the range (4 - 16).</p>
<p>RAM and disk can also have a range. Just return the minimum value for now (this is the existing behavior).</p>
<p>Tell cwltool to use your <code>select_resources</code> method by setting the object field <code>runtimeContext.select_resources</code>.</p> GET-Evidence - Bug #14622 (New): Annotate l7g CWL pipelinehttps://dev.arvados.org/issues/146222018-12-17T14:13:38ZBen Carr
<p>Old Ticket:<br />Tiling workflow documentation is out of date. It can be auto-generated partly directly from the cwl workflows.</p>
<p>----<br />SWZ:</p>
<p>CWL annotated:<br />[...]</p>
<p>CWL: Not annotated yet, and not experimental<br />[...]</p>
<p>I would like you to annotate the ones in the following folders as noted by the main workflows (and those cwl they call) called in the AD overview documents:</p>
<pre><code class="text syntaxhl">/cwl-version/filter/cwl
/cwl-version/clean/cwl
/cwl-version/convert2fastj/gvcf_version/cwl
/cwl-version/tilelib/cwl
/cwl-version/checks/check-sglf/cwl
/cwl-version/cgf3/cwl
/cwl-version/checks/check-cgf/gvcf/cwl/
/cwl-version/npy/cwl
</code></pre>
<p>Also please spellcheck and check for existing typos.<br />----<br />BHCC:</p>
<p>Quote Edit Delete<br />As of 20181212 the Following is Annotated :</p>
<pre><code class="text syntaxhl">./cgf3/cwl/createcgf.cwl
./cgf3/cwl/getdirs20.cwl
./cgf3/cwl/tiling_convert2cgf.cwl
./cgf3/cwl/getdirs.cwl
./checks/check-cgf/gvcf/cwl/validate-conversion-gvcf-cgf-chrom_workflow.cwl
./checks/check-cgf/gvcf/cwl/gather_validate-conversion-gvcf-cgf.cwl
./checks/check-cgf/gvcf/cwl/validate-conversion-gvcf-cgf-chrom.cwl
./checks/check-sglf/cwl/sglf-sanity-check.cwl
./tagset/l7g-tagset.cwl
./tagset/tagset.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf.cwl
./convert2fastj/gvcf_version/cwl/convertgvcf.cwl
./convert2fastj/gvcf_version/cwl/getdirs.cwl
./clean/cwl/tiling_clean_gvcf.cwl
./clean/cwl/getdirs.cwl
./clean/cwl/cleangvcf.cwl
./filter/cwl/filter.cwl
./filter/cwl/getCollections.cwl
./filter/cwl/tiling_filtergvcf.cwl
./tilelib/cwl/getpaths_chunk.cwl
./tilelib/cwl/tiling_createsglf_chunk-scatter_v2.cwl
./tilelib/cwl/merge-tilelib.cwl
./tilelib/cwl/createsglf_chunkv2.cwl
./npy/cwl/tiling_npy-wf.cwl
./npy/cwl/cwl_steps/tiling_create-npy.cwl
./npy/cwl/cwl_steps/tiling_consol-npy.cwl
</code></pre>
<p>BHCC:<br />Te branch we are working on is</p>
<p>14386-cwl-docum<br />I have reviewed the list for typographical errors, including grep'ing out all labels and running them through spellcheck. The only place where camel case was used is where it is used by convention for clarity (NumPy) of the package or object.</p>
<p>Things needed to be addressed:<br />Check spelling one more time<br />make sure every label starts with a capital<br />Remove any unnecessary Camel Case<br />Use the YML files as a guide, as variable names do not always reflect what is going on<br />Remove and Doc blocks that do not add anything beyond what the label covers.<br />Shorten any labels as much as possible, the more concise the better<br />Don't include the type of code C++ / bash unless absolutely necessary<br />Will likely need a new ticket for the automated generation of figures and may need to run view.cwl locally to make that work. Will test automated generation once the labels are merged to master as vew.cwl only currently works off GitHub-master.<br />We should consider renaming the "gff" in the variable names of the following scripts:</p>
<pre><code class="text syntaxhl">./filter/cwl/filter.cwl
./filter/cwl/tiling_filtergvcf.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf_named.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf.cwl
./convert2fastj/gvcf_version/cwl/convertgvcf.cwl
</code></pre><br />to reflect gVCF status Tapestry - Idea #6257 (In Progress): Add public facing database dump, cron job and Arvados uploadhttps://dev.arvados.org/issues/62572015-06-05T20:56:01ZAbram Connellyabram.connelly@gmail.com
<p>There is code to do a dump of the public facing data. Merge it into Tapestry, create a cron job to automatically run on a daily basis and have it upload to an Arvados project.</p> GET-Evidence - Bug #6143 (New): Add 'nofollow' to download links for reporthttps://dev.arvados.org/issues/61432015-05-22T21:29:54ZAbram Connellyabram.connelly@gmail.com
<p>Web crawlers hammer the keep servers asking for the data files from links that appear in the reports. Add 'nofollow' on relevant links from the report page.</p> GET-Evidence - Bug #5883 (New): migrate to updated oauthhttps://dev.arvados.org/issues/58832015-05-01T18:31:33ZAbram Connellyabram.connelly@gmail.com
<p>There is a warning on that says OAuth2 is going away. We need to migrate the authentication on GET-Evidence to make sure login is still possible.</p> GET-Evidence - Feature #5873 (New): GET-Evidence pipeline code should be publichttps://dev.arvados.org/issues/58732015-04-30T22:55:20ZAbram Connellyabram.connelly@gmail.com
<p>The GET-Evidence pipeline should be put under a public git repository.</p>
<p>Currently the GET-Evidence repository is under my (Abram Connelly) private Arvados git repository. This should probably remain but some hooks should be added somewhere so that the repository is pushed to a public Curoverse repository.</p> GET-Evidence - Bug #5872 (In Progress): 23andMe report pipeline is brokenhttps://dev.arvados.org/issues/58722015-04-30T22:52:53ZAbram Connellyabram.connelly@gmail.com
<p>When participants upload their 23andMe data, this kicks off a pipeline run through Arvados (now) through the same code path that the dataset submission goes through from Tapestry. The pipeline is failing to correctly generate reports for 23andMe data.</p>
<p>Porting the pipeline over to Arvados, the format for the input was changed and that's why the pipelines are failing on 23andMe data. I think the best thing to do going forward is to be more liberal about what input collection is specified so that 23andMe reports along with older reports can be run (or re-run) without having to worry about updating to the new format.</p>
<p>The old format just specifies a portable data hash. The new format expects <PDH>/<filename>.</p> Lightning - Idea #5001 (New): Create a API interface to the public tile library including a "beacon"https://dev.arvados.org/issues/50012015-01-16T18:29:36ZSarah Guthriesallyeguthrie@gmail.comLightning - Idea #4827 (New): Automatic Deployment Implementationhttps://dev.arvados.org/issues/48272014-12-15T22:59:30ZSarah Guthriesallyeguthrie@gmail.com
<p>Goal: have an automatic process to run tests and setup Lightning that is run when a commit is pushed to master</p>
Steps:
<ol>
<li>Research how to structure the git repository</li>
<li>Research/communicate with Ward how to set up automatic deployment</li>
<li>Start writing tests and deploying</li>
</ol>