Bug #9244

[FUSE] .arvados#collection is missing or outdated after writing inside collection

Added by Brett Smith over 4 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
05/20/2016
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

If you read .arvados#collection before you write, it will have those contents after the write, even though they're no longer accurate (e.g., the manifest_text is out-of-date). If you don't read it before you write, you will get an I/O error when you try to read it later.

With a fresh mount at mnt, reading after writing:

% cd mnt
% cd home/
% cd Hello\ Files/
% cat alice.txt
hello, alice
% echo 'hello, bob' >bob.txt
% cat .arvados#collection
cat: .arvados#collection: Input/output error

Exception in the logs that generated the error:

2016-05-20 09:08:26 arvados.arvados_fuse[19721] ERROR: Unhandled exception during FUSE o
peration
Traceback (most recent call last):
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse
-0.1.20160322160927-py2.7.egg/arvados_fuse/__init__.py", line 277, in catch_exceptions_w
rapper
    return orig_func(self, *args, **kwargs)
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse
-0.1.20160322160927-py2.7.egg/arvados_fuse/__init__.py", line 466, in lookup
    inode = p[name].inode
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse-0.1.20160322160927-py2.7.egg/arvados_fuse/fresh.py", line 20, in use_counter_wrapper
    return orig_func(self, *args, **kwargs)
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse-0.1.20160322160927-py2.7.egg/arvados_fuse/fresh.py", line 29, in check_update_wrapper
    return orig_func(self, *args, **kwargs)
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse-0.1.20160322160927-py2.7.egg/arvados_fuse/fusedir.py", line 459, in __getitem__
    self.collection_record_file = ObjectFile(self.inode, self.collection_record)
  File "/home/brett/.local/virtualenv/arv/local/lib/python2.7/site-packages/arvados_fuse-0.1.20160322160927-py2.7.egg/arvados_fuse/fusefile.py", line 91, in __init__
    self.object_uuid = obj['uuid']
TypeError: 'NoneType' object has no attribute '__getitem__'
   unique: 571, error: -5 (Input/output error), outsize: 16

Reading, then writing:

% cd mnt/home/Hello\ Files/
% cat .arvados#collection
{
    "manifest_text": ". 03032680d3fa0561ef4f85071140861e+13+A... d820b9df970e1b498e7723c50b107e1b+11+A... 0:13:alice.txt 13:11:bob.txt\n",
    "uuid": "4xphq-4zz18-tiy3r9qvuk7u8c0",
    [some fields elided]
}
% echo 'hello, carol' >carol.txt
% cat .arvados#collection
{
    "manifest_text": ". 03032680d3fa0561ef4f85071140861e+13+A... d820b9df970e1b498e7723c50b107e1b+11+A... 0:13:alice.txt 13:11:bob.txt\n",
    "uuid": "4xphq-4zz18-tiy3r9qvuk7u8c0",
    [some fields elided]
}

There was no clear error in the logs in this case. Access tokens were the same in each manifest.


Related issues

Has duplicate Arvados - Bug #8679: [FUSE] Metafile .arvados#collection not updated when collection is modifiedDuplicate

History

#1 Updated by Brett Smith over 4 years ago

#8679 duplicates the second half of this.

#2 Updated by Tom Morris over 3 years ago

  • Target version set to Arvados Future Sprints

Also available in: Atom PDF