Bug #3427

[SDK] arv-put does not resume upload

Added by Abram Connelly about 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Brett Smith
Category:
SDKs
Target version:
Start date:
07/30/2014
Due date:
% Done:

100%

Estimated time:
Story points:
1.0

Description

I started an upload from my local machine to a remote keep store by issuing an 'arv-put' and killed it after it reported 192M uploaded. Resuming after the kill had the file upload restart from the beginning.

This was after a fresh 'sudo pip install --upgrade arvados-python-client'.

$ ls -lah tile_md5sum_hu154_sort.csv.gz 
-rw-r--r-- 1 abram abram 961M Jul 29 12:25 tile_md5sum_hu154_sort.csv.gz
$ arv-put tile_md5sum_hu154_sort.csv.gz 
arv-put: No --project-uuid or --name specified.  This data will be cached
in Keep.  You will need to find this upload by its locator(s) later.
192M / 960M 20.0% ^C
$ arv-put tile_md5sum_hu154_sort.csv.gz 
arv-put: No --project-uuid or --name specified.  This data will be cached
in Keep.  You will need to find this upload by its locator(s) later.
0M / 960M 0.0% 

The following might be relevant as well:

$ which arv-put
/usr/local/bin/arv-put
$ sudo pip freeze | grep arvados
arvados-python-client==0.1.1406400797.673a10d

Also:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       205G   74G  121G  38% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev            7.4G  4.0K  7.4G   1% /dev
tmpfs           1.5G  1.2M  1.5G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.5G  4.4M  7.5G   1% /run/shm
none            100M   12K  100M   1% /run/user
/dev/sda1       1.8T  1.6T  180G  90% /scratch
$ cd ~/.cache/arvados/arv-put
~/.cache/arvados/arv-put$ ls -lah
total 8.0K
drwx------ 2 abram abram 4.0K Jul 30 17:27 .
drwxrwxr-x 4 abram abram 4.0K Jul 24 16:59 ..
-rw-rw-r-- 1 abram abram    0 Jul 24 16:57 6513a274b009750531dc2aeaaa52d692
-rw-rw-r-- 1 abram abram    0 Jul 30 17:27 9e04856f702db1742d23c6907db465da
-rw-rw-r-- 1 abram abram    0 Jul 28 09:06 daf08842da8fa0415899872f983b914e

Associated revisions

Revision 7962de49 (diff)
Added by Brett Smith about 5 years ago

3427: Restore arv-put's resume functionality.

The refactoring in 0d1a48e accidentally introduced a bug where arv-put
wipes its resume cache clean before building the uploader. Because
the refactor always arranges `resume_cache = None` when the user
doesn't want to resume, we can fix the bug by just skipping the wipe.
Closes #3427.

History

#1 Updated by Peter Amstutz about 5 years ago

Possibly your Python package is out of date. The latest package is arvados-python-client-0.1.1406756091.698f00b

#2 Updated by Tom Clegg about 5 years ago

  • Subject changed from arv-put does not resume upload to [SDK] arv-put does not resume upload

#3 Updated by Tom Clegg about 5 years ago

  • Story points set to 1.0

#4 Updated by Tom Clegg about 5 years ago

  • Target version set to 2014-08-27 Sprint

#5 Updated by Brett Smith about 5 years ago

  • Assigned To set to Brett Smith

The packaged version is recent enough to have the most recent bugfix relating to resuming, so this needs further investigation.

#6 Updated by Tim Pierce about 5 years ago

  • Category set to SDKs

#7 Updated by Brett Smith about 5 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:7962de491af28d00a9c88412ad4d1e42be83432a.

Also available in: Atom PDF