Idea #10968
closed
[arv-put] Notify user that resume cache is being used & provide progress indication
Added by Tom Morris about 7 years ago.
Updated about 7 years ago.
Description
Inform the user that resume cache is being used & calcluations.
If possible, provide period progress updates to the user (~10-15 seconds).
Timeboxed to 1 pt
- Target version set to 2017-02-15 sprint
- Assigned To set to Lucas Di Pentima
- Status changed from New to In Progress
Branch 10968-arvput-resume-progress at: ef328d9
Test run: https://ci.curoverse.com/job/developer-run-tests/154/
There were 2 steps that take too long when uploading a big collection of files:
- The
expected_bytes calculation (when there are a lot of files on disk to be uploaded)
- If resuming a partial upload,
arv-put will check if there's some previously uploaded file on the cached collection that is not on disk on the current run. This was taking too much time, and was fixed on #10932
The updates on this branch include an optimization on the progress reporting and a message to notify the user when it's calculating the expected_bytes count on a potential big file list.
The optimization on progress reporting is about disabling unnecessary checkpointing when the upload hasn't started yet, and using the skipped_bytes count to show a progress indication with the said count vs total.
The final effect is that when resuming, the progress indicator will start from 0% to whatever percentage was left on the previous run, updating every second to provide maximum feedback until the real upload begins, changing to "checkpointing mode" every minute like before.
LGTM, thanks! (Might as well fix errant space at EOL in comment, though... Coding Standards)
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados|commit:aaa45b09de0e9437743fce53d7c0bf8165074b5e.
Also available in: Atom
PDF
Updated by Tom Morris 
Updated by 
Updated by