Story #10968

[arv-put] Notify user that resume cache is being used & provide progress indication

Added by Tom Morris over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
01/24/2017
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0

Description

Inform the user that resume cache is being used & calcluations.
If possible, provide period progress updates to the user (~10-15 seconds).

Timeboxed to 1 pt


Subtasks

Task #11042: Review 10968-arvput-resume-progressResolvedLucas Di Pentima

Associated revisions

Revision aaa45b09
Added by Lucas Di Pentima over 2 years ago

Merge branch '10968-arvput-resume-progress'
Closes #10968

History

#1 Updated by Tom Morris over 2 years ago

  • Target version set to 2017-02-15 sprint

#2 Updated by Lucas Di Pentima over 2 years ago

  • Assigned To set to Lucas Di Pentima

#3 Updated by Lucas Di Pentima over 2 years ago

  • Status changed from New to In Progress

#4 Updated by Lucas Di Pentima over 2 years ago

Branch 10968-arvput-resume-progress at: ef328d9
Test run: https://ci.curoverse.com/job/developer-run-tests/154/

There were 2 steps that take too long when uploading a big collection of files:
  1. The expected_bytes calculation (when there are a lot of files on disk to be uploaded)
  2. If resuming a partial upload, arv-put will check if there's some previously uploaded file on the cached collection that is not on disk on the current run. This was taking too much time, and was fixed on #10932

The updates on this branch include an optimization on the progress reporting and a message to notify the user when it's calculating the expected_bytes count on a potential big file list.
The optimization on progress reporting is about disabling unnecessary checkpointing when the upload hasn't started yet, and using the skipped_bytes count to show a progress indication with the said count vs total.
The final effect is that when resuming, the progress indicator will start from 0% to whatever percentage was left on the previous run, updating every second to provide maximum feedback until the real upload begins, changing to "checkpointing mode" every minute like before.

#5 Updated by Tom Clegg over 2 years ago

LGTM, thanks! (Might as well fix errant space at EOL in comment, though... Coding Standards)

#6 Updated by Lucas Di Pentima over 2 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:aaa45b09de0e9437743fce53d7c0bf8165074b5e.

Also available in: Atom PDF