Bug #4669

[SDKs] arv-put should assume --no-resume when --raw is given

Added by Tim Pierce over 4 years ago. Updated over 4 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
SDKs
Target version:
Start date:
11/25/2014
Due date:
% Done:

0%

Estimated time:
Story points:
0.5

Description

The --raw option to arv-put is incompatible with the ResumeCollectionWriter, and raises an unhandled exception when run on a collection with a manifest larger than 64MB:

2014-11-24_20:17:48 4xphq-8i9sb-nseyfve44wcgn15 26498  wait for last 0 children to finish
2014-11-24_20:17:48 4xphq-8i9sb-nseyfve44wcgn15 26498  status: 61 done, 0 running, 0 todo
2014-11-24_20:17:48 4xphq-8i9sb-nseyfve44wcgn15 26498  Freeze not implemented
2014-11-24_20:17:48 4xphq-8i9sb-nseyfve44wcgn15 26498  collate
67108864 Traceback (most recent call last):
  File "/usr/local/bin/arv-put", line 4, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 450, in main
    writer.write_file(path, args.filename or os.path.basename(path))
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 317, in write_file
    super(ArvPutCollectionWriter, self).write_file(source, filename)
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 411, in write_file
    self.do_queued_work()
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 344, in do_queued_work
    self._work_file()
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 357, in _work_file
    self.write(buf)
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 662, in write
    return super(ResumableCollectionWriter, self).write(data)
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 427, in write
    self.flush_data()
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 304, in flush_data
    self.cache_state()
  File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 283, in cache_state
    state = self.dump_state()
  File "/usr/local/lib/python2.7/dist-packages/arvados/collection.py", line 631, in dump_state
    self._queued_file.tell())
IOError: [Errno 29] Illegal seek
128594 @shell.4xphq:~$ 
Collection saved as 'Saved at 2014-11-24 20:15:08 UTC by twp@shell.4xphq.arvadosapi.com'
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

History

#1 Updated by Tom Clegg over 4 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints

#2 Updated by Tom Clegg over 4 years ago

  • Story points set to 0.5

Also available in: Atom PDF