https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422017-12-17T23:21:26ZArvadosArvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=585942017-12-17T23:21:26ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/58594/diff?detail_id=56003">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=585952017-12-17T23:39:10ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><p>work in progress here</p>
<p>12308-go-fuse @ <a class="changeset" title="12308: Update parent dir modtime when adding/removing/renaming. Arvados-DCO-1.1-Signed-off-by: T..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/aa18bbe2333f293d329efdae4a13ff79b03a1d8c">aa18bbe2333f293d329efdae4a13ff79b03a1d8c</a></p>
<pre>
go get -d git.curoverse.com/arvados.git/lib/crunchstat ;# clone arvados to your gopath
(cd $GOPATH/src/git.curoverse.com/arvados.git && git checkout origin/12308-go-fuse)
go get git.curoverse.com/arvados.git/cmd/arvados-client
# (set up your ARVADOS_API_HOST and ARVADOS_API_TOKEN env vars)
$GOPATH/arvados-client mount --experimental /tmp/mnt &
cd /tmp/mnt/by_id/$some_existing_collection_uuid/ && git clone file:///home/path/to/arvados.git && sync .
</pre> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=586522017-12-19T05:37:42ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/58652/diff?detail_id=56068">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=587852017-12-23T07:10:03ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/58785/diff?detail_id=56201">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=591162018-01-08T00:59:28ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-3 priority-4 priority-default closed" href="/issues/12876">Feature #12876</a>: [CLI] arvados-client command-line tool</i> added</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=602612018-02-15T20:06:20ZAbram Connellyabram.connelly@gmail.com
<ul></ul><p>Using the new arv mount functionality, I created a new keep mount via:</p>
<pre>
$ mkdir keepgo
$ arvados-client mount --experimental keepgo/ -d 2> arv-mount-experimental.log
</pre>
<p>Going into the <code>keepgo/home</code> and doing some <code>ls</code> commands intermittantly gives the following error message:</p>
<pre>
abram@lightning-dev1:~/keepgo/home$ ls
ls: reading directory '.': Input/output error
00-example-shell.cwl input
Saved at 2015-05-06 21:13:44 UTC by crunch@01b0dfdb2f15
...
</pre>
<p>Where, when the error occurs, it gives a partial list of the directory.</p>
<p>I don't see anything of note in the log so I haven't provided it here.</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=618732018-04-13T20:46:32ZTom Cleggtom@curii.com
<ul></ul><p>The non-fuse-related code here has been extracted and used in <a class="issue tracker-6 status-3 priority-4 priority-default closed parent" title="Idea: [WebDAV] Support browsing of project hierarchies (Resolved)" href="https://dev.arvados.org/issues/13111">#13111</a>.</p>
<p>Fuse parts are rebased against <a class="issue tracker-6 status-3 priority-4 priority-default closed parent" title="Idea: [WebDAV] Support browsing of project hierarchies (Resolved)" href="https://dev.arvados.org/issues/13111">#13111</a>, now 12308-cgofuse @ <a class="changeset" title="12308: Add "arvados-client mount" command via cgofuse. Arvados-DCO-1.1-Signed-off-by: Tom Clegg ..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/c5633c850d664d2f78e0efccf9ec9734b4e32de5">c5633c850d664d2f78e0efccf9ec9734b4e32de5</a>.</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=809612020-01-18T00:39:50ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> set to <i>2020-02-12 Sprint</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=813122020-01-23T20:52:53ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-1 priority-4 priority-default" href="/issues/16082">Idea #16082</a>: Port client tools to Go</i> added</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=814672020-01-29T17:06:03ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=814702020-01-29T17:13:40ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> changed from <i>Peter Amstutz</i> to <i>Tom Clegg</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=814962020-01-30T16:44:51ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/81496/diff?detail_id=78164">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=815502020-02-04T14:43:39ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/81550/diff?detail_id=78225">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=815522020-02-04T14:57:57ZTom Cleggtom@curii.com
<ul></ul><p>12308-cgofuse @ <a class="changeset" title="12308: Merge branch 'master' Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>" href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/9a4fcabed1adeff0044d419977d5136c5cb1db3e">9a4fcabed1adeff0044d419977d5136c5cb1db3e</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1715/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1715/">developer-run-tests: #1715 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1715" alt="" /></a></a></p>
<p>Adds a "mount" subcommand to arvados-client, with limitations noted in the issue description above.</p>
<p>Our currently supported platforms (linux/amd64) work fine with the usual build process; <a class="source" href="https://dev.arvados.org/projects/arvados/repository/arvados/entry/cmd/arvados-client/Makefile">source:cmd/arvados-client/Makefile</a> has a recipe for cross-compiling binaries for linux/macos/windows on i386/amd64 using their respective fuse/fuse-like libraries.</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=815962020-02-06T14:44:36ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Gave it a light first pass look, a couple of issues:</p>
<ul>
<li>Tests failed on <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1715/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1715/">developer-run-tests: #1715 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1715" alt="" /></a></a></li>
<li>Getting checksum mismatch when trying to run make on <code>cmd/arvados-client/</code></li>
</ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=818122020-02-06T21:38:22ZTom Cleggtom@curii.com
<ul></ul><p>Fixed incorrect test assertion that failed build 1715 from <a class="issue tracker-6 status-3 priority-4 priority-default closed parent" title="Idea: [FUSE] Golang-based fuse driver (Resolved)" href="https://dev.arvados.org/issues/12308#note-14">#12308#note-14</a></p>
<p>12308-cgofuse @ <a class="changeset" title="12308: Fix wrong error check. Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>" href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/e54bbc170b78f3f4c90be7c8b314d58e559cd73c">e54bbc170b78f3f4c90be7c8b314d58e559cd73c</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1718/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1718/">developer-run-tests: #1718 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1718" alt="" /></a></a></p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=818132020-02-06T21:42:39ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>I have been testing a binary compiled for OSX, using <code>osxfuse</code> version 3.10.4 and OSX 10.14.6</p>
<p>The test consists on cloning arvados' repository from github.</p>
<ul>
<li>Against ce8i5 (5 Mbps uplink) fails crashing the program without debug info</li>
<li>Using arvbox
<ul>
<li>Unbounded virtual network: OK, 30secs</li>
<li>50 Mbps up/down: OK, 36 secs</li>
<li>25 Mbps up/down: OK, 50 secs</li>
<li>12 Mbps up/down: OK, 65 secs</li>
<li>5 Mbps up/down: FAILED (with crash and no debug info): 2min14secs, , got to write 43 of the 73 MB before crashing.</li>
</ul></li>
</ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=818142020-02-06T21:45:06ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Repeating the test of cloning arvados' repo, I got one crash with debug information after 6 seconds, the error I got from the 'git' command execution was <code>'fatal: write error: Device not configured.'</code>:</p>
<pre>
fatal error: concurrent map read and map write
goroutine 50 [running, locked to thread]:
runtime.throw(0x468fd7a, 0x21)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc0020d6a58 sp=0xc0020d6a28 pc=0x4031912
runtime.mapaccess1_faststr(0x45c5d80, 0xc0003ca180, 0xc002270081, 0x5, 0xc00228c0e0)
/usr/local/go/src/runtime/map_faststr.go:21 +0x44f fp=0xc0020d6ac8 sp=0xc0020d6a58 pc=0x40152df
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).Child(0xc000104300, 0xc002270081, 0x5, 0xc002269c50, 0x0, 0x40306ba, 0x44cd4fc, 0xc000104360)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/arvados/fs_base.go:259 +0x56 fp=0xc0020d6b38 sp=0xc0020d6ac8 pc=0x44b3256
git.arvados.org/arvados.git/sdk/go/arvados.(*vdirnode).Child(0xc0000c2cc0, 0xc002270081, 0x5, 0x0, 0xc0020d6c00, 0x40f1883, 0xc0022700b1, 0xa)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/arvados/fs_site.go:191 +0x9a fp=0xc0020d6b88 sp=0xc0020d6b38 pc=0x44c452a
git.arvados.org/arvados.git/sdk/go/arvados.rlookup.func1(0xc0020d6cc0, 0xc0020d6c80, 0x0, 0x0, 0x0, 0x0)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/arvados/fs_base.go:606 +0xc1 fp=0xc0020d6c10 sp=0xc0020d6b88 pc=0x44c78d1
git.arvados.org/arvados.git/sdk/go/arvados.rlookup(0x4758e00, 0xc0000c2cc0, 0xc002270080, 0x3b, 0x4758e00, 0xc0000c2cc0, 0xc0003c0288, 0x88)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/arvados/fs_base.go:607 +0x1bd fp=0xc0020d6ca0 sp=0xc0020d6c10 pc=0x44b5e7d
git.arvados.org/arvados.git/sdk/go/arvados.(*fileSystem).Stat(0xc00012a9a0, 0xc002270080, 0x3b, 0x0, 0x0, 0x0, 0x0)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/arvados/fs_base.go:439 +0x4f fp=0xc0020d6cf0 sp=0xc0020d6ca0 pc=0x44b492f
git.arvados.org/arvados.git/lib/mount.(*keepFS).Getattr(0xc0003c0240, 0xc002270080, 0x3b, 0xc00226fd40, 0xffffffffffffffff, 0x0)
/ext-go/2/src/git.arvados.org/arvados.git/lib/mount/fs.go:240 +0x16d fp=0xc0020d6d70 sp=0xc0020d6cf0 pc=0x4545f2d
github.com/arvados/cgofuse/fuse.hostGetattr(0x7002b60, 0x70000ffd2be0, 0x0)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:119 +0x123 fp=0xc0020d6e18 sp=0xc0020d6d70 pc=0x453a2b3
github.com/arvados/cgofuse/fuse.go_hostGetattr(...)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host_cgo.go:717
github.com/arvados/cgofuse/fuse._cgoexpwrap_ecb3c7988e70_go_hostGetattr(0x7002b60, 0x70000ffd2be0, 0x12800028)
_cgo_gotypes.go:592 +0x35 fp=0xc0020d6e40 sp=0xc0020d6e18 pc=0x45406a5
runtime.call32(0x0, 0x70000ffd2a10, 0x70000ffd2aa0, 0x18)
/usr/local/go/src/runtime/asm_amd64.s:539 +0x3b fp=0xc0020d6e70 sp=0xc0020d6e40 pc=0x405fa3b
runtime.cgocallbackg1(0x0)
/usr/local/go/src/runtime/cgocall.go:314 +0x1b7 fp=0xc0020d6f58 sp=0xc0020d6e70 pc=0x4005a37
runtime.cgocallbackg(0x0)
/usr/local/go/src/runtime/cgocall.go:191 +0xc1 fp=0xc0020d6fc0 sp=0xc0020d6f58 pc=0x40057e1
runtime.cgocallback_gofunc(0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/runtime/asm_amd64.s:793 +0x9b fp=0xc0020d6fe0 sp=0xc0020d6fc0 pc=0x406100b
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc0020d6fe8 sp=0xc0020d6fe0 pc=0x4061731
goroutine 1 [syscall, 5 minutes]:
github.com/arvados/cgofuse/fuse._Cfunc_hostMount(0x3, 0xc0003bc6c0, 0x4f01310, 0x0)
_cgo_gotypes.go:515 +0x4d
github.com/arvados/cgofuse/fuse.c_hostMount.func1(0xc000000003, 0xc0003bc6c0, 0x4f01310, 0x4f01310)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host_cgo.go:701 +0x97
github.com/arvados/cgofuse/fuse.c_hostMount(0xc000000003, 0xc0003bc6c0, 0x4f01310, 0xc0001a52c0)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host_cgo.go:701 +0x3d
github.com/arvados/cgofuse/fuse.(*FileSystemHost).Mount(0xc000168e80, 0x0, 0x0, 0xc00015f190, 0x1, 0x1, 0x0)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:666 +0x4ca
git.arvados.org/arvados.git/lib/mount.(*cmd).RunCommand(0x4b38dd0, 0xc000184120, 0x28, 0xc00015f180, 0x2, 0x2, 0x4743220, 0xc0000b0000, 0x4743240, 0xc0000b0008, ...)
/ext-go/2/src/git.arvados.org/arvados.git/lib/mount/command.go:81 +0x5c4
git.arvados.org/arvados.git/lib/cmd.Multi.RunCommand(0xc00015f0b0, 0x7ffeefbff4b0, 0x22, 0xc00015f170, 0x3, 0x3, 0x4743220, 0xc0000b0000, 0x4743240, 0xc0000b0008, ...)
/ext-go/2/src/git.arvados.org/arvados.git/lib/cmd/cmd.go:89 +0x280
main.main()
/ext-go/2/src/git.arvados.org/arvados.git/cmd/arvados-client/cmd.go:65 +0xe1
goroutine 6 [select]:
git.arvados.org/arvados.git/sdk/go/keepclient.(*cachedSvcList).poll(0xc00000ebc0)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/keepclient/discover.go:94 +0x154
created by git.arvados.org/arvados.git/sdk/go/keepclient.(*KeepClient).discoverServices
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/keepclient/discover.go:150 +0x542
goroutine 35 [syscall, 5 minutes]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:144 +0x96
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41
goroutine 48 [chan receive, 5 minutes]:
github.com/arvados/cgofuse/fuse.(*FileSystemHost).Mount.func3(0xc000168e80, 0xc0001a52c0)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:653 +0x41
created by github.com/arvados/cgofuse/fuse.(*FileSystemHost).Mount
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:652 +0x44b
goroutine 42 [select]:
git.arvados.org/arvados.git/sdk/go/keepclient.(*cachedSvcList).poll.func1(0xc0001a4d80, 0xc0001a4de0, 0xc00000ebc0)
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/keepclient/discover.go:77 +0x1a9
created by git.arvados.org/arvados.git/sdk/go/keepclient.(*cachedSvcList).poll
/ext-go/2/src/git.arvados.org/arvados.git/sdk/go/keepclient/discover.go:73 +0xaf
goroutine 51 [runnable, locked to thread]:
github.com/arvados/cgofuse/fuse._Cfunc_hostCstatFromFusestat(0x70000f9aebc0, 0x0, 0x0, 0x1000041ed, 0x14000001f6, 0x0, 0x8, 0x5e3c87a4, 0x3441ab98, 0x5e3c87a4, ...)
_cgo_gotypes.go:430 +0x45
github.com/arvados/cgofuse/fuse.c_hostCstatFromFusestat(...)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host_cgo.go:669
github.com/arvados/cgofuse/fuse.copyCstatFromFusestat(0x70000f9aebc0, 0xc0022d2000)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:80 +0x151
github.com/arvados/cgofuse/fuse.hostGetattr(0x6910210, 0x70000f9aebc0, 0x0)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host.go:120 +0x14b
github.com/arvados/cgofuse/fuse.go_hostGetattr(...)
/go/pkg/mod/github.com/arvados/cgofuse@v1.2.0/fuse/host_cgo.go:717
github.com/arvados/cgofuse/fuse._cgoexpwrap_ecb3c7988e70_go_hostGetattr(0x6910210, 0x70000f9aebc0, 0x4f7c3f0)
_cgo_gotypes.go:592 +0x35
goroutine 15611 [IO wait]:
internal/poll.runtime_pollWait(0x6d79f38, 0x72, 0xffffffffffffffff)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000104118, 0x72, 0x800, 0x83c, 0xffffffffffffffff)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000104100, 0xc0000e4900, 0x83c, 0x83c, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:169 +0x22b
net.(*netFD).Read(0xc000104100, 0xc0000e4900, 0x83c, 0x83c, 0x203000, 0x580020000000000, 0x0)
/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000154058, 0xc0000e4900, 0x83c, 0x83c, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:184 +0x68
crypto/tls.(*atLeastReader).Read(0xc000476180, 0xc0000e4900, 0x83c, 0x83c, 0xc0020d38c0, 0x4019f3e, 0xc0020d38a0)
/usr/local/go/src/crypto/tls/conn.go:780 +0x60
bytes.(*Buffer).ReadFrom(0xc0000a8958, 0x47429c0, 0xc000476180, 0x400c9d5, 0x45dd5a0, 0x465c1a0)
/usr/local/go/src/bytes/buffer.go:204 +0xb4
crypto/tls.(*Conn).readFromUntil(0xc0000a8700, 0x6d7a0d8, 0xc000154058, 0x5, 0xc000154058, 0x12)
/usr/local/go/src/crypto/tls/conn.go:802 +0xec
crypto/tls.(*Conn).readRecordOrCCS(0xc0000a8700, 0x0, 0x0, 0x3)
/usr/local/go/src/crypto/tls/conn.go:609 +0x124
crypto/tls.(*Conn).readRecord(...)
/usr/local/go/src/crypto/tls/conn.go:577
crypto/tls.(*Conn).Read(0xc0000a8700, 0xc0017ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1255 +0x161
net/http.(*persistConn).Read(0xc0022cc5a0, 0xc0017ee000, 0x1000, 0x1000, 0xc000096120, 0xc0020d3c20, 0x40075a5)
/usr/local/go/src/net/http/transport.go:1752 +0x75
bufio.(*Reader).fill(0xc0001419e0)
/usr/local/go/src/bufio/bufio.go:100 +0x103
bufio.(*Reader).Peek(0xc0001419e0, 0x1, 0x0, 0x0, 0x1, 0xc008c5e800, 0x0)
/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*persistConn).readLoop(0xc0022cc5a0)
/usr/local/go/src/net/http/transport.go:1905 +0x1d6
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1574 +0xafe
goroutine 15612 [select]:
net/http.(*persistConn).writeLoop(0xc0022cc5a0)
/usr/local/go/src/net/http/transport.go:2204 +0x123
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1575 +0xb23
</pre> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=818162020-02-06T22:00:23ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Running 3 concurrent 'git clone' commands was enough to make it crash again with the same error.</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=820352020-02-07T20:00:07ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/82035/diff?detail_id=78723">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=821852020-02-10T20:05:35ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>My latest 'black box' tests from last week were done with the linux binary and the issues found on the Mac version didn't happened: running multiple concurrent <code>git clone</code> operations worked great, also writing through a simulated slow link didn't crash the client.</p>
<p>Will give the code another look.</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=821892020-02-10T21:37:21ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Code review:</p>
<ul>
<li><code>lib/mount/command.go</code>
<ul>
<li>Line 29: Typo on comment</li>
<li>Line 60: Could that be simplified by using arvadosclient.MakeArvadosClient()?</li>
</ul>
</li>
<li><code>lib/mount/fs.go</code>
<ul>
<li>Line 65: Why should <code>lookupFH()</code> lock the filesystem while just reading data? — NOTE: As I’m learning, concurrent reads AND writes are not possible on maps, but could we use <code>RWMutex</code> to allow concurrent reads?</li>
<li>Line 255: Does this conditional make the return value to be “Not implemented” when the fh is a regular file or dir? A comment would help future readers.</li>
<li>Line 257: The <code>else</code> clause isn’t necessary. In which case the flow would reach this? Asking because if we’re not implementing this, returning 0 is telling the SO that the operation succeeded.</li>
</ul></li>
</ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=823682020-02-12T16:24:07ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>2020-02-12 Sprint</i> to <i>2020-02-26 Sprint</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=824202020-02-12T20:53:15ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/82420/diff?detail_id=79130">diff</a>)</li></ul><blockquote>
<ul>
<li><code>lib/mount/command.go</code>
<ul>
<li>Line 29: Typo on comment</li>
</ul></li>
</ul>
</blockquote>
<p>Fixed</p>
<blockquote>
<ul>
<li>Line 60: Could that be simplified by using arvadosclient.MakeArvadosClient()?</li>
</ul>
</blockquote>
<p>Not sure I follow. We could use MakeArvadosClient(), but we'd need to call NewClientFromEnv() anyway to get the client var, and once we have that, it seems cleaner to derive the ac/kc clients from it...</p>
<blockquote>
<ul>
<li><code>lib/mount/fs.go</code>
<ul>
<li>Line 65: Why should <code>lookupFH()</code> lock the filesystem while just reading data? — NOTE: As I’m learning, concurrent reads AND writes are not possible on maps, but could we use <code>RWMutex</code> to allow concurrent reads?</li>
</ul></li>
</ul>
</blockquote>
<p>Yes, good point -- reads probably outnumber writes, so RWMutex is probably better / less blocking. Changed.</p>
<blockquote>
<ul>
<li>Line 255: Does this conditional make the return value to be “Not implemented” when the fh is a regular file or dir? A comment would help future readers.</li>
</ul>
</blockquote>
<p>Added comments. It's ENOSYS when changing mode from file to dir, or vice versa.</p>
<blockquote>
<ul>
<li>Line 257: The <code>else</code> clause isn’t necessary. In which case the flow would reach this? Asking because if we’re not implementing this, returning 0 is telling the SO that the operation succeeded.</li>
</ul>
</blockquote>
<p>Added comments. This is the case where chmod is expected to succeed (it's only changing permission bits). It's a no-op because we don't save permission bits. We could return ENOSYS if the mode isn't 0755. I suspect that would make lots of things (like tar xzf) fail instead of doing the obvious thing, though. Perhaps we should have "strict/loose" modes? Meanwhile I've added this to the list of shortcomings.</p>
<p>12308-cgofuse @ <a class="changeset" title="12308: Fix typo. Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>" href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/3a2006d29fc38596a4dfb19b331bf2c86a9185ae">3a2006d29fc38596a4dfb19b331bf2c86a9185ae</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1728/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1728/">developer-run-tests: #1728 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1728" alt="" /></a></a></p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=824372020-02-13T17:06:15ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Thanks for the clarifications and comments, they'll be helpful.<br />About <code>arvadosclient.MakeArvadosClient()</code> I missed that the <code>client</code> var was used elsewhere, sorry.</p>
<p>This LGTM, thanks!</p> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=826482020-02-26T15:31:45ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> deleted (<del><i>2020-02-26 Sprint</i></del>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=837172020-04-23T15:12:51ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/83717/diff?detail_id=80484">diff</a>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=861192020-08-19T15:03:36ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed parent" href="/issues/16727">Bug #16727</a>: [FUSE] [cgofuse] Refresh signatures / reload collection instead of using expired blob signatures</i> added</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=876262020-10-07T02:11:32ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Release</strong> set to <i>25</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=877032020-10-07T02:13:13ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Release</strong> deleted (<del><i>25</i></del>)</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=940792021-06-30T20:44:22ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-1 priority-4 priority-default" href="/issues/17849">Idea #17849</a>: FUSE driver v2</i> added</li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=1118832023-02-14T22:22:45ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Release</strong> set to <i>60</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=1232612024-03-01T21:12:18ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> set to <i>Future</i></li></ul> Arvados - Idea #12308: [FUSE] Golang-based fuse driverhttps://dev.arvados.org/issues/12308?journal_id=1239542024-03-07T15:15:03ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Release</strong> deleted (<del><i>60</i></del>)</li><li><strong>Target version</strong> deleted (<del><i>Future</i></del>)</li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>Marking this resolved because an implementation does exist already</p>