Bug #8712
closed
[FUSE] Read-only collections appear empty after they are evicted from the cache
Added by Abram Connelly about 8 years ago.
Updated almost 8 years ago.
Description
On one of my shell nodes (lightning-dev1.su92l) I cannot see files that should be there through my keep mount in a collection. They show up in workbench and logging into another shell node, the files are visible.
On lightning-dev1
:
abram@lightning-dev1:~$ hostname
lightning-dev1.shell.su92l.arvadosapi.com
abram@lightning-dev1:~$ cd keep/by_id/a2198905927b8d24e162abe42e541f37+173
abram@lightning-dev1:~/keep/by_id/a2198905927b8d24e162abe42e541f37+173$ ls
abram@lightning-dev1:~/keep/by_id/a2198905927b8d24e162abe42e541f37+173$ ls -la
total 1
dr-xr-xr-x 1 abram abram 0 Jan 1 1970 .
dr-xr-xr-x 1 abram abram 0 Feb 22 20:40 ..
abram@lightning-dev1:~/keep/by_id/a2198905927b8d24e162abe42e541f37+173$
On abe.shell.su92l
:
abram@abe.su92l:~$ hostname
abe.shell.su92l.arvadosapi.com
abram@abe.su92l:~$ cd keep/by_id/a2198905927b8d24e162abe42e541f37+173
abram@abe.su92l:~/keep/by_id/a2198905927b8d24e162abe42e541f37+173$ ls -la
total 57016
dr-xr-xr-x 1 abram abram 0 Jan 1 1970 .
dr-xr-xr-x 1 abram abram 0 Feb 22 22:50 ..
-r-xr-xr-x 1 abram abram 32302 Jan 1 1970 assembly.00.hg19.fw.fwi
-r-xr-xr-x 1 abram abram 58305282 Jan 1 1970 assembly.00.hg19.fw.gz
-r-xr-xr-x 1 abram abram 41784 Jan 1 1970 assembly.00.hg19.fw.gz.gzi
-r-xr-xr-x 1 abram abram 2712 Jan 1 1970 README
abram@abe.su92l:~/keep/by_id/a2198905927b8d24e162abe42e541f37+173$
- Category set to FUSE
- Assigned To set to Peter Amstutz
- Target version set to 2016-03-30 sprint
- Target version changed from 2016-03-30 sprint to 2016-04-13 sprint
- Subject changed from Disappearing files in keep mount to [FUSE?] Disappearing files in keep mount
The bug is when a read-only collection has been spilled from the cache and then needs to be reloaded:
2016-04-04 10:25:02 arvados.arvados_fuse[10263] DEBUG: Updating None
2016-04-04 10:25:02 arvados.arvados_fuse[10263] ERROR: arv-mount b1eb3a7572b5fbbf04335bbf0ade7209+85: error
Traceback (most recent call last):
File "/home/peter/work/scripts/venv/local/lib/python2.7/site-packages/arvados_fuse-0.1.20160322160927-py2.7.egg/arvados_fuse/fusedir.py", line 410, in update
self.collection.update()
File "/home/peter/work/scripts/venv/local/lib/python2.7/site-packages/arvados_python_client-0.1.20160322001610-py2.7.egg/arvados/arvfile.py", line 238, in synchronized_wrapper
return orig_func(self, *args, **kwargs)
File "/home/peter/work/scripts/venv/local/lib/python2.7/site-packages/arvados_python_client-0.1.20160322001610-py2.7.egg/arvados/retry.py", line 153, in num_retries_setter
return orig_func(self, *args, **kwargs)
File "/home/peter/work/scripts/venv/local/lib/python2.7/site-packages/arvados_python_client-0.1.20160322001610-py2.7.egg/arvados/collection.py", line 1228, in update
self.apply(baseline.diff(other))
File "/home/peter/work/scripts/venv/local/lib/python2.7/site-packages/arvados_python_client-0.1.20160322001610-py2.7.egg/arvados/arvfile.py", line 389, in must_be_writable_wrapper
raise IOError(errno.EROFS, "Collection is read-only.")
IOError: [Errno 30] Collection is read-only.
reviewing 55778ad90a9b869f514e729b3f85c12383f4e2a7
I'm confused about if clear() should return a boolean or not...
because:
(class Directory) clear() line 166 does a
if not oldentries[n].clear(force)
and _entries dict could contain a CollectionDirectory correct?
if that's the case code should be something like:
def clear(self, force=False):
rtn = super(CollectionDirectoryBase, self).clear(force)
self.collection = None
return rtn
and tests should include more complex examples of a full Directory._entries
- Subject changed from [FUSE?] Disappearing files in keep mount to [FUSE] Read-only collections appear empty after they are evicted from the cache
- Status changed from New to In Progress
- Story points set to 0.5
- Status changed from In Progress to Resolved
- % Done changed from 50 to 100
Applied in changeset arvados|commit:b8ae9ce66c24055f3bf66b0956e15c44c9dd51f0.
Also available in: Atom
PDF