Bug #4309
Updated by Peter Amstutz over 9 years ago
Originally https://arvados.org/issues/3699#note-43
I'm copying a pipeline with a 5990M collection. I noticed this code:
<pre>
data = src_keep.get(word)
dst_locator = dst_keep.put(data)
</pre>
See attached image, there's a very clear falloff between blocks -- doing this sequentially isn't optimal. Download and upload could proceed concurrently. Also, I suspect we could get better utilization if we downloaded 2 blocks at a time.
But in the interests of getting arv-copy out the door we probably shouldn't do anything about it now.