Project

General

Profile

Actions

Bug #8867

closed

'arv-put' reports 100% upload but files only partially uploaded

Added by Peter Grandi almost 8 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
SDKs
Target version:
-
Story points:
-

Description

Curious situation where someone uploads around 4TiB to a mostly-full Keep with arv-put, it is reported as 100% complete fine, and the file size looks right at around 16GB:

librarian@sole$ ls -ld DDD_WGS_EGAD00001001114_3/DDDP106312.20131218.bam /local/library2/ddd-data/EGAD00001001114/DDDP106312.20131218.bam/DDDP106312.20131218.bam
-r-xr-xr-x 1 librarian librarian 16614852138 Mar 14 06:34 DDD_WGS_EGAD00001001114_3/DDDP106312.20131218.bam
-rw-r--r-- 1 librarian      9993 16614852138 Jun  6  2015 /local/library2/ddd-data/EGAD00001001114/DDDP106312.20131218.bam/DDDP106312.20131218.bam

But only the first 4GB of those 16GB are actually there:

librarian@sole$ cmp -l DDD_WGS_EGAD00001001114_3/DDDP106312.20131218.bam /local/library2/ddd-data/EGAD00001001114/DDDP106312.20131218.bam/DDDP106312.20131218.bam | head -50
cmp: EOF on DDD_WGS_EGAD00001001114_3/DDDP106312.20131218.bam
librarian@sole$ dd bs=1M if=DDD_WGS_EGAD00001001114_3/DDDP106312.20131218.bam of=/dev/null
3765+2 records in
3765+2 records out
3948298240 bytes (3.9 GB) copied, 82.117 s, 48.1 MB/s
librarian@sole$ expr 16614852138 - 3948298240
12666553898

Related issues

Related to Arvados - Idea #8937: [SDKs] Write integration test for when arv-put resumes from a cache with expired access tokensResolvedLucas Di Pentima04/13/2016Actions
Actions

Also available in: Atom PDF