Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422024-03-28T17:14:24ZArvados
Redmine Arvados - Idea #21634 (New): Go through WB2 and ensure that loading indicators behave consistentl...https://dev.arvados.org/issues/216342024-03-28T17:14:24ZPeter Amstutzpeter.amstutz@curii.comArvados - Task #21625 (New): Review at engineering meetinghttps://dev.arvados.org/issues/216252024-03-27T16:08:13ZPeter Amstutzpeter.amstutz@curii.comArvados - Feature #21623 (New): Display filters active on a data table as chips on the column headerhttps://dev.arvados.org/issues/216232024-03-27T15:55:43ZPeter Amstutzpeter.amstutz@curii.comArvados - Bug #21622 (New): Mail delivery failure should not cause API calls to failhttps://dev.arvados.org/issues/216222024-03-27T15:52:03ZPeter Amstutzpeter.amstutz@curii.comArvados Workbench 2 - Feature #21621 (New): Inputs/Outputs panel allows JSON data to be copied to...https://dev.arvados.org/issues/216212024-03-27T14:19:45ZLucas Di Pentimalucas.dipentima@curii.com
<p>Sometimes it's necessary to get that data, specially on inputs when you need to re-run something from the CLI. Having a copy-to-clipboard button would make it trivial.</p> Arvados Workbench 2 - Idea #21615 (New): Details Panel should show details for every type of reso...https://dev.arvados.org/issues/216152024-03-22T13:25:49ZLisa Knox
<p>The details panel currently only shows details when viewing a project, workflow, process, or collection and shows empty everywhere else. This includes on the Shell Access view, Instance Types, and a few other places where there is really nothing to display, but the details panel will remain open if it was open when navigating to that view.</p>
<p>The details panel should display something useful if the current view allows for it, or should close automatically when navigating to a view where it is irrelevant. Having it display an empty element with the word "Projects" at the top in all non-specified cases is not helpful.</p> Arvados - Feature #21614 (New): User can open things in new tab with middle-click/Ctrl+clickhttps://dev.arvados.org/issues/216142024-03-21T14:54:40ZBrett Smithbrett.smith@curii.com
<p>Please please please, this would help a lot for situations where Workbench is struggling under the load of a large item and I need to get at adjacent items.</p>
<ul>
<li>Breadcrumbs parent links</li>
<li>The "Open collection" button of a process' logs pane</li>
<li>Subprocesses</li>
<li>Files in a collection listing</li>
</ul> Arvados - Bug #21612 (New): a-c-r with --debug can try to log entire input/output objects, which ...https://dev.arvados.org/issues/216122024-03-20T20:22:22ZBrett Smithbrett.smith@curii.com
<p>User got this error while running aws-s3-bulk-download.cwl with >6K input URLs, using <code>a-c-r --submit --debug</code>.</p>
<p>I don't think it actually interfered with the workflow's run at all, but it clogs the logs and looks scary.</p>
<p>IMO a-c-r (along with the rest of our code) should not try to log data that can be arbitrarily large.</p>
<p>Three instances where this came up:</p>
<pre>
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.7/logging/__init__.py", line 1037, in emit
stream.write(msg + self.terminator)
BlockingIOError: [Errno 11] write could not complete without blocking
Call stack:
File "/usr/bin/arvados-cwl-runner", line 8, in <module>
sys.exit(main())
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/__init__.py", line 440, in main
input_required=not workflow_op)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/main.py", line 1302, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 874, in arv_executor
self.start_run(runnable, runtimeContext)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 248, in start_run
self.workflow_eval_lock, self.stop_polling)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/task_queue.py", line 85, in add
task()
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/command_line_tool.py", line 202, in run
self.output_callback(cast(Optional[CWLObjectType], ev), "success")
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 321, in wrapped_callback
cb(obj, st)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow.py", line 429, in receive_output
output_callback(output, processStatus)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 564, in receive_output
_logger.debug("[%s] produced output %s", step.name, json_dumps(jobout, indent=4))
</pre>
<pre>--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.7/logging/__init__.py", line 1037, in emit
stream.write(msg + self.terminator)
BlockingIOError: [Errno 11] write could not complete without blocking
Call stack:
File "/usr/bin/arvados-cwl-runner", line 8, in <module>
sys.exit(main())
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/__init__.py", line 440, in main
input_required=not workflow_op)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/main.py", line 1302, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 863, in arv_executor
for runnable in jobiter:
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow.py", line 175, in job
yield from job.job(builder.job, output_callbacks, runtimeContext)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 821, in job
for newjob in step.iterable:
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 751, in try_make_job
yield from jobs
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 77, in job
yield from self.step.job(joborder, output_callback, runtimeContext)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow.py", line 462, in job
runtimeContext,
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow.py", line 175, in job
yield from job.job(builder.job, output_callbacks, runtimeContext)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 821, in job
for newjob in step.iterable:
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 735, in try_make_job
json_dumps(inputobj, indent=4),
</pre>
<pre>--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.7/logging/__init__.py", line 1037, in emit
stream.write(msg + self.terminator)
BlockingIOError: [Errno 11] write could not complete without blocking
Call stack:
File "/usr/bin/arvados-cwl-runner", line 8, in <module>
sys.exit(main())
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/__init__.py", line 440, in main
input_required=not workflow_op)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/main.py", line 1302, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 874, in arv_executor
self.start_run(runnable, runtimeContext)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 248, in start_run
self.workflow_eval_lock, self.stop_polling)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/task_queue.py", line 85, in add
task()
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/command_line_tool.py", line 202, in run
self.output_callback(cast(Optional[CWLObjectType], ev), "success")
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 321, in wrapped_callback
cb(obj, st)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow.py", line 429, in receive_output
output_callback(output, processStatus)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 582, in receive_output
self.do_output_callback(final_output_callback)
File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/workflow_job.py", line 541, in do_output_callback
_logger.debug("[%s] outputs %s", self.name, json_dumps(wo, indent=4))
</pre> Arvados - Bug #17434 (New): project not shown in hierarchy on the left of wb2https://dev.arvados.org/issues/174342021-03-01T21:07:39ZDaniel Kutyła
<p>Project that is being shared is not visible in the left tree menu</p> Arvados - Bug #17396 (New): Favorites copy dialog further issueshttps://dev.arvados.org/issues/173962021-02-16T22:31:18ZDaniel Kutyła
<p><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: Favorites in copy dialog is different to favorite list (Resolved)" href="https://dev.arvados.org/issues/17306">#17306</a> solved main issues reported by our users but there are some stuff that came out while fixing those:</p>
<p>- We can not copy element which is located in a subdir without copying other elements as parent folder will not be selected and thus it will be removed <br />- For a large collections files copy operation might be very slow as we remove non selected elements in a post copy operation</p> Arvados - Bug #17300 (New): array of array of File causes 'NoneType' object has no attribute 'res...https://dev.arvados.org/issues/173002021-01-29T14:19:14ZStanislaw Adaszewski
<p>When running an ExpressionTool which has an output of the following type:</p>
<pre>
outputs:
subject_images:
type:
type: array # subjects
items:
type: array # timepoints
items: File
</pre>
<p>I receive the following error:</p>
<pre>
ERROR Unhandled error:
'NoneType' object has no attribute 'resolved'
Traceback (most recent call last):
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/main.py", line 833, in main
logger=_logger)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 767, in arv_executor
self.final_output, self.final_output_collection = self.make_output_collection(self.output_name, storage_classes, self.output_tags, self.final_output)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 467, in make_output_collection
adjustFileObjs(outputObj, rewrite)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/pathmapper.py", line 48, in adjustFileObjs
visit_class(rec, ("File",), op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 221, in visit_class
visit_class(rec[d], cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 224, in visit_class
visit_class(d, cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 224, in visit_class
visit_class(d, cls, op)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/utils.py", line 219, in visit_class
op(rec)
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 461, in rewrite
fileobj["location"] = generatemapper.mapper(fileobj["location"]).target
File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/pathmapper.py", line 332, in mapper
return MapperEnt(v.resolved, v.target + src[i:], v.type, v.staged)
AttributeError: 'NoneType' object has no attribute 'resolved'
</pre>
<p>If I replace type: File with type: string and store locations instead of File objects the request finishes successfully and returns the expected list of locations.</p> Arvados - Bug #16228 (New): NullPointerException while uploading file using Java SDKhttps://dev.arvados.org/issues/162282020-03-10T07:28:41ZKrzysztof Majewski
<p>When trying to upload a files using Java SDK I get the following error:</p>
<pre>
2020-03-10 08:07:23 ERROR - Cannot upload file to Keep server.
org.arvados.client.exception.ArvadosApiException: java.net.ConnectException: Failed to connect to keep1.ardev.roche.com/10.115.164.103:25107
at org.arvados.client.api.client.BaseApiClient.getResponseBody(BaseApiClient.java:76)
at org.arvados.client.api.client.BaseApiClient.newCall(BaseApiClient.java:49)
at org.arvados.client.api.client.KeepServerApiClient.upload(KeepServerApiClient.java:42)
at org.arvados.client.logic.keep.FileTransferHandler.put(FileTransferHandler.java:35)
at org.arvados.client.logic.keep.KeepClient.lambda$put$1(KeepClient.java:110)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.net.ConnectException: Failed to connect to keep1.ardev.roche.com/10.115.164.103:25107
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:270)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:176)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:236)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:109)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:77)
at okhttp3.internal.connection.Transmitter.newExchange$okhttp(Transmitter.kt:162)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:35)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:71)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
at okhttp3.RealCall.execute(RealCall.kt:66)
at org.arvados.client.api.client.BaseApiClient.getResponseBody(BaseApiClient.java:63)
... 10 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:126)
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:268)
... 29 more
</pre>
<p>The keep1 server is taken from: <a class="external" href="https://api.ardev.roche.com/arvados/v1/keep_services/accessible">https://api.ardev.roche.com/arvados/v1/keep_services/accessible</a>. If I manipulate the response and remove it, leaving just keep2 everything works fine.</p>
<p>This is my code used for this example:</p>
<pre>
String serverUrl = "ardev.roche.com";
ExternalConfigProvider config = ExternalConfigProvider.builder()
.apiToken("api_token")
.apiHost(format("api.%s", serverUrl))
.apiPort(443)
.keepWebHost(format("collections.%s", serverUrl))
.keepWebPort(443)
.apiProtocol("https")
.apiHostInsecure(true)
.fileSplitSize(64)
.fileSplitDirectory(new File("C:\\file-split"))
.numberOfCopies(2)
.numberOfRetries(0)
.build();
List<File> files = asList(new File("C:\\samples\\sample.fcs"));
Collection collection = new CollectionsApiClient(config).get("collection_uuid");
new ArvadosFacade(config).uploadToExistingCollection(files, collection.getUuid());
</pre> Arvados - Bug #15183 (New): [Workbench2] No errors reported when file is missing a blockhttps://dev.arvados.org/issues/151832019-05-01T21:39:26ZTom Morristfmorris@veritasgenetics.com
<p>When looking at collection <a href="https://arvadosapi.com/su92l-4zz18-44qy35pg0lxshkf">su92l-4zz18-44qy35pg0lxshkf</a>/cwl.input.yml the file is cwl.output.json is listed as containing 361 bytes, but when downloaded from Workbench, it produces a zero byte file, but without giving any error or warning.</p>
<pre>
$ md5sum keep/by_id/su92l-4zz18-44qy35pg0lxshkf/*
a890894f5a748d2a93440fd046c73069 keep/by_id/su92l-4zz18-44qy35pg0lxshkf/cwl.input.yml
md5sum: keep/by_id/su92l-4zz18-44qy35pg0lxshkf/cwl.output.json: Input/output error
d41d8cd98f00b204e9800998ecf8427e keep/by_id/su92l-4zz18-44qy35pg0lxshkf/hu27FD1F_var-GS000034212-ASM.bed
b02868b9ac1264bd1f04b61d04518f57 keep/by_id/su92l-4zz18-44qy35pg0lxshkf/workflow.cwl
</pre> GET-Evidence - Bug #835 (New): Error when saving edits or submittinghttps://dev.arvados.org/issues/8352012-02-06T16:44:19ZMadeleine Ballmpball@gmail.com
<p>When saving or submitting we're seeing this error message now: "Save failed -- received bogus response from server."</p>
<p>Seems to date to changes made in commit <a class="changeset" title="if session is expired when "save draft" is clicked, pop-up a login window instead of falsely proc..." href="https://dev.arvados.org/projects/get-evidence/repository/8/revisions/5dc9adee7bd5cde0d1c9b16652cf0b5e8d6229d3">5dc9adee</a> ?</p>
<p>Sasha pointed out that the changes still get saved, but it's still a serious bug -- especially since we just sent the paper out for review and were planning to encourage people to edit...</p> GET-Evidence - Bug #468 (New): latest-flat incorrectly displays non single-base substitution vari...https://dev.arvados.org/issues/4682011-02-15T11:15:42ZMadeleine Ballmpball@gmail.com
<p>I haven't checked, but this problem likely extends to deletion & other new length changing or multiple-aa variants.</p>
<p>This bug blocks creating a flatfile version of gff_getevidence_map.py</p>
<p>For example, FIG4-K278Shift is showing up like this (2nd and 3rd columns are incorrect):<br />FIG4 278 278 pathogenic Moderate clinical importance, Uncertain pathogenic recessive 0 0 0 0 0 3 Y -- - Y 0 Y 3 - 2 - Y - - - - Y 4 Y 2N 1 0 This variant is predicted to cause a frameshift and may cause Charcot-Marie-Tooth Disease Type 4J in an autosomal recessive manner. Other variants in this gene which cause frameshift and premature termination have been implicated in causing this disease when compound heterozygous with another FIG4 variant.</p>
<p>While MYL2-A13T shows up like this:<br />MYL2 Ala13Thr A13T pathogenic Low clinical importance, Uncertain pathogenic dominant 1 1 1 6 6 3 Y 4Y 0 Y ! Y 4 Y 3 Y Y Y - - Y Y 1 - Familial Hypertrophic Cardiomyopathy 4596 1 455 3.054 4 N 0 0 This rare variant is implicated in causing late-onset familial hypertrophic cardiomyopathy. The variant has been found in five affected Caucasian individuals (in four families), but affected non-carriers and unaffected carriers have also been observed. No statistically significant enrichment of this variant in cases vs. controls has been shown.</p>