Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422024-03-27T16:11:09ZArvados
Redmine Arvados - Task #21631 (New): Reviewhttps://dev.arvados.org/issues/216312024-03-27T16:11:09ZPeter Amstutzpeter.amstutz@curii.comArvados - Bug #21434 (New): Follow up fix to schema saladhttps://dev.arvados.org/issues/214342024-01-31T16:46:46ZPeter Amstutzpeter.amstutz@curii.com
<p><a class="external" href="https://github.com/common-workflow-language/schema_salad/issues/766">https://github.com/common-workflow-language/schema_salad/issues/766</a></p> Arvados - Bug #17434 (New): project not shown in hierarchy on the left of wb2https://dev.arvados.org/issues/174342021-03-01T21:07:39ZDaniel Kutyła
<p>Project that is being shared is not visible in the left tree menu</p> Arvados - Bug #17396 (New): Favorites copy dialog further issueshttps://dev.arvados.org/issues/173962021-02-16T22:31:18ZDaniel Kutyła
<p><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: Favorites in copy dialog is different to favorite list (Resolved)" href="https://dev.arvados.org/issues/17306">#17306</a> solved main issues reported by our users but there are some stuff that came out while fixing those:</p>
<p>- We can not copy element which is located in a subdir without copying other elements as parent folder will not be selected and thus it will be removed <br />- For a large collections files copy operation might be very slow as we remove non selected elements in a post copy operation</p> Arvados - Bug #17300 (New): array of array of File causes 'NoneType' object has no attribute 'res...https://dev.arvados.org/issues/173002021-01-29T14:19:14ZStanislaw Adaszewski
<p>When running an ExpressionTool which has an output of the following type:</p>
<pre>
outputs:
subject_images:
type:
type: array # subjects
items:
type: array # timepoints
items: File
</pre>
<p>I receive the following error:</p>
<pre>
ERROR Unhandled error:
'NoneType' object has no attribute 'resolved'
Traceback (most recent call last):
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/main.py", line 833, in main
logger=_logger)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 767, in arv_executor
self.final_output, self.final_output_collection = self.make_output_collection(self.output_name, storage_classes, self.output_tags, self.final_output)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 467, in make_output_collection
adjustFileObjs(outputObj, rewrite)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/pathmapper.py", line 48, in adjustFileObjs
visit_class(rec, ("File",), op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 221, in visit_class
visit_class(rec[d], cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 224, in visit_class
visit_class(d, cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 224, in visit_class
visit_class(d, cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 219, in visit_class
op(rec)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 461, in rewrite
fileobj["location"] = generatemapper.mapper(fileobj["location"]).target
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/pathmapper.py", line 332, in mapper
return MapperEnt(v.resolved, v.target + src[i:], v.type, v.staged)
AttributeError: 'NoneType' object has no attribute 'resolved'
</pre>
<p>If I replace type: File with type: string and store locations instead of File objects the request finishes successfully and returns the expected list of locations.</p> Arvados - Bug #16228 (New): NullPointerException while uploading file using Java SDKhttps://dev.arvados.org/issues/162282020-03-10T07:28:41ZKrzysztof Majewski
<p>When trying to upload a files using Java SDK I get the following error:</p>
<pre>
2020-03-10 08:07:23 ERROR - Cannot upload file to Keep server.
org.arvados.client.exception.ArvadosApiException: java.net.ConnectException: Failed to connect to keep1.ardev.roche.com/10.115.164.103:25107
at org.arvados.client.api.client.BaseApiClient.getResponseBody(BaseApiClient.java:76)
at org.arvados.client.api.client.BaseApiClient.newCall(BaseApiClient.java:49)
at org.arvados.client.api.client.KeepServerApiClient.upload(KeepServerApiClient.java:42)
at org.arvados.client.logic.keep.FileTransferHandler.put(FileTransferHandler.java:35)
at org.arvados.client.logic.keep.KeepClient.lambda$put$1(KeepClient.java:110)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.net.ConnectException: Failed to connect to keep1.ardev.roche.com/10.115.164.103:25107
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:270)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:176)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:236)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:109)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:77)
at okhttp3.internal.connection.Transmitter.newExchange$okhttp(Transmitter.kt:162)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:35)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:71)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
at okhttp3.RealCall.execute(RealCall.kt:66)
at org.arvados.client.api.client.BaseApiClient.getResponseBody(BaseApiClient.java:63)
... 10 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:126)
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:268)
... 29 more
</pre>
<p>The keep1 server is taken from: <a class="external" href="https://api.ardev.roche.com/arvados/v1/keep_services/accessible">https://api.ardev.roche.com/arvados/v1/keep_services/accessible</a>. If I manipulate the response and remove it, leaving just keep2 everything works fine.</p>
<p>This is my code used for this example:</p>
<pre>
String serverUrl = "ardev.roche.com";
ExternalConfigProvider config = ExternalConfigProvider.builder()
.apiToken("api_token")
.apiHost(format("api.%s", serverUrl))
.apiPort(443)
.keepWebHost(format("collections.%s", serverUrl))
.keepWebPort(443)
.apiProtocol("https")
.apiHostInsecure(true)
.fileSplitSize(64)
.fileSplitDirectory(new File("C:\\file-split"))
.numberOfCopies(2)
.numberOfRetries(0)
.build();
List<File> files = asList(new File("C:\\samples\\sample.fcs"));
Collection collection = new CollectionsApiClient(config).get("collection_uuid");
new ArvadosFacade(config).uploadToExistingCollection(files, collection.getUuid());
</pre> Arvados - Bug #15183 (New): [Workbench2] No errors reported when file is missing a blockhttps://dev.arvados.org/issues/151832019-05-01T21:39:26ZTom Morristfmorris@veritasgenetics.com
<p>When looking at collection <a href="https://arvadosapi.com/su92l-4zz18-44qy35pg0lxshkf">su92l-4zz18-44qy35pg0lxshkf</a>/cwl.input.yml the file is cwl.output.json is listed as containing 361 bytes, but when downloaded from Workbench, it produces a zero byte file, but without giving any error or warning.</p>
<pre>
$ md5sum keep/by_id/su92l-4zz18-44qy35pg0lxshkf/*
a890894f5a748d2a93440fd046c73069 keep/by_id/su92l-4zz18-44qy35pg0lxshkf/cwl.input.yml
md5sum: keep/by_id/su92l-4zz18-44qy35pg0lxshkf/cwl.output.json: Input/output error
d41d8cd98f00b204e9800998ecf8427e keep/by_id/su92l-4zz18-44qy35pg0lxshkf/hu27FD1F_var-GS000034212-ASM.bed
b02868b9ac1264bd1f04b61d04518f57 keep/by_id/su92l-4zz18-44qy35pg0lxshkf/workflow.cwl
</pre> GET-Evidence - Bug #14622 (New): Annotate l7g CWL pipelinehttps://dev.arvados.org/issues/146222018-12-17T14:13:38ZBen Carr
<p>Old Ticket:<br />Tiling workflow documentation is out of date. It can be auto-generated partly directly from the cwl workflows.</p>
<p>----<br />SWZ:</p>
<p>CWL annotated:<br />[...]</p>
<p>CWL: Not annotated yet, and not experimental<br />[...]</p>
<p>I would like you to annotate the ones in the following folders as noted by the main workflows (and those cwl they call) called in the AD overview documents:</p>
<pre><code class="text syntaxhl">/cwl-version/filter/cwl
/cwl-version/clean/cwl
/cwl-version/convert2fastj/gvcf_version/cwl
/cwl-version/tilelib/cwl
/cwl-version/checks/check-sglf/cwl
/cwl-version/cgf3/cwl
/cwl-version/checks/check-cgf/gvcf/cwl/
/cwl-version/npy/cwl
</code></pre>
<p>Also please spellcheck and check for existing typos.<br />----<br />BHCC:</p>
<p>Quote Edit Delete<br />As of 20181212 the Following is Annotated :</p>
<pre><code class="text syntaxhl">./cgf3/cwl/createcgf.cwl
./cgf3/cwl/getdirs20.cwl
./cgf3/cwl/tiling_convert2cgf.cwl
./cgf3/cwl/getdirs.cwl
./checks/check-cgf/gvcf/cwl/validate-conversion-gvcf-cgf-chrom_workflow.cwl
./checks/check-cgf/gvcf/cwl/gather_validate-conversion-gvcf-cgf.cwl
./checks/check-cgf/gvcf/cwl/validate-conversion-gvcf-cgf-chrom.cwl
./checks/check-sglf/cwl/sglf-sanity-check.cwl
./tagset/l7g-tagset.cwl
./tagset/tagset.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf.cwl
./convert2fastj/gvcf_version/cwl/convertgvcf.cwl
./convert2fastj/gvcf_version/cwl/getdirs.cwl
./clean/cwl/tiling_clean_gvcf.cwl
./clean/cwl/getdirs.cwl
./clean/cwl/cleangvcf.cwl
./filter/cwl/filter.cwl
./filter/cwl/getCollections.cwl
./filter/cwl/tiling_filtergvcf.cwl
./tilelib/cwl/getpaths_chunk.cwl
./tilelib/cwl/tiling_createsglf_chunk-scatter_v2.cwl
./tilelib/cwl/merge-tilelib.cwl
./tilelib/cwl/createsglf_chunkv2.cwl
./npy/cwl/tiling_npy-wf.cwl
./npy/cwl/cwl_steps/tiling_create-npy.cwl
./npy/cwl/cwl_steps/tiling_consol-npy.cwl
</code></pre>
<p>BHCC:<br />Te branch we are working on is</p>
<p>14386-cwl-docum<br />I have reviewed the list for typographical errors, including grep'ing out all labels and running them through spellcheck. The only place where camel case was used is where it is used by convention for clarity (NumPy) of the package or object.</p>
<p>Things needed to be addressed:<br />Check spelling one more time<br />make sure every label starts with a capital<br />Remove any unnecessary Camel Case<br />Use the YML files as a guide, as variable names do not always reflect what is going on<br />Remove and Doc blocks that do not add anything beyond what the label covers.<br />Shorten any labels as much as possible, the more concise the better<br />Don't include the type of code C++ / bash unless absolutely necessary<br />Will likely need a new ticket for the automated generation of figures and may need to run view.cwl locally to make that work. Will test automated generation once the labels are merged to master as vew.cwl only currently works off GitHub-master.<br />We should consider renaming the "gff" in the variable names of the following scripts:</p>
<pre><code class="text syntaxhl">./filter/cwl/filter.cwl
./filter/cwl/tiling_filtergvcf.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf_named.cwl
./convert2fastj/gvcf_version/cwl/tiling_convert2fastj_gvcf.cwl
./convert2fastj/gvcf_version/cwl/convertgvcf.cwl
</code></pre><br />to reflect gVCF status GET-Evidence - Bug #6143 (New): Add 'nofollow' to download links for reporthttps://dev.arvados.org/issues/61432015-05-22T21:29:54ZAbram Connellyabram.connelly@gmail.com
<p>Web crawlers hammer the keep servers asking for the data files from links that appear in the reports. Add 'nofollow' on relevant links from the report page.</p> GET-Evidence - Bug #5883 (New): migrate to updated oauthhttps://dev.arvados.org/issues/58832015-05-01T18:31:33ZAbram Connellyabram.connelly@gmail.com
<p>There is a warning on that says OAuth2 is going away. We need to migrate the authentication on GET-Evidence to make sure login is still possible.</p> GET-Evidence - Feature #5873 (New): GET-Evidence pipeline code should be publichttps://dev.arvados.org/issues/58732015-04-30T22:55:20ZAbram Connellyabram.connelly@gmail.com
<p>The GET-Evidence pipeline should be put under a public git repository.</p>
<p>Currently the GET-Evidence repository is under my (Abram Connelly) private Arvados git repository. This should probably remain but some hooks should be added somewhere so that the repository is pushed to a public Curoverse repository.</p> Lightning - Idea #5001 (New): Create a API interface to the public tile library including a "beacon"https://dev.arvados.org/issues/50012015-01-16T18:29:36ZSarah Guthriesallyeguthrie@gmail.comLightning - Idea #4827 (New): Automatic Deployment Implementationhttps://dev.arvados.org/issues/48272014-12-15T22:59:30ZSarah Guthriesallyeguthrie@gmail.com
<p>Goal: have an automatic process to run tests and setup Lightning that is run when a commit is pushed to master</p>
Steps:
<ol>
<li>Research how to structure the git repository</li>
<li>Research/communicate with Ward how to set up automatic deployment</li>
<li>Start writing tests and deploying</li>
</ol> GET-Evidence - Bug #835 (New): Error when saving edits or submittinghttps://dev.arvados.org/issues/8352012-02-06T16:44:19ZMadeleine Ballmpball@gmail.com
<p>When saving or submitting we're seeing this error message now: "Save failed -- received bogus response from server."</p>
<p>Seems to date to changes made in commit <a class="changeset" title="if session is expired when "save draft" is clicked, pop-up a login window instead of falsely proc..." href="https://dev.arvados.org/projects/get-evidence/repository/8/revisions/5dc9adee7bd5cde0d1c9b16652cf0b5e8d6229d3">5dc9adee</a> ?</p>
<p>Sasha pointed out that the changes still get saved, but it's still a serious bug -- especially since we just sent the paper out for review and were planning to encourage people to edit...</p> GET-Evidence - Bug #468 (New): latest-flat incorrectly displays non single-base substitution vari...https://dev.arvados.org/issues/4682011-02-15T11:15:42ZMadeleine Ballmpball@gmail.com
<p>I haven't checked, but this problem likely extends to deletion & other new length changing or multiple-aa variants.</p>
<p>This bug blocks creating a flatfile version of gff_getevidence_map.py</p>
<p>For example, FIG4-K278Shift is showing up like this (2nd and 3rd columns are incorrect):<br />FIG4 278 278 pathogenic Moderate clinical importance, Uncertain pathogenic recessive 0 0 0 0 0 3 Y -- - Y 0 Y 3 - 2 - Y - - - - Y 4 Y 2N 1 0 This variant is predicted to cause a frameshift and may cause Charcot-Marie-Tooth Disease Type 4J in an autosomal recessive manner. Other variants in this gene which cause frameshift and premature termination have been implicated in causing this disease when compound heterozygous with another FIG4 variant.</p>
<p>While MYL2-A13T shows up like this:<br />MYL2 Ala13Thr A13T pathogenic Low clinical importance, Uncertain pathogenic dominant 1 1 1 6 6 3 Y 4Y 0 Y ! Y 4 Y 3 Y Y Y - - Y Y 1 - Familial Hypertrophic Cardiomyopathy 4596 1 455 3.054 4 N 0 0 This rare variant is implicated in causing late-onset familial hypertrophic cardiomyopathy. The variant has been found in five affected Caucasian individuals (in four families), but affected non-carriers and unaffected carriers have also been observed. No statistically significant enrichment of this variant in cases vs. controls has been shown.</p>