Bug #14900
closed
[arv-put] when reusing a local cache, arv-put does not check if the blocks exist and blindly creates the collection
Description
What I ran into today:
1) create an ephemeral arvados cluster with our kubernetes prototype
2) arv-put a file to it from my local system. Arv-put creates a local cache directory in .cache/arvados/arv-put/
3) blow away the arvados cluster, and re-create it
4) re-run the arv-put command from my local system:
$ arv-put 4xphq-8i9sb-fmwod1qn74cemdp.log.txt 2019-02-27 22:05:07 arvados.arv_put[103627] INFO: Resuming upload from cache file /home/wvandewege/.cache/arvados/arv-put/0a16eb3c98df444230504d71edb7d7c1 0M / 0M 100.0% 2019-02-27 22:05:07 arvados.arv_put[103627] INFO: 2019-02-27 22:05:07 arvados.arv_put[103627] INFO: Collection saved as 'Saved at 2019-02-27 22:05:07 UTC by wvandewege@shell.qr1hi.arvadosapi.com' vwxyz-4zz18-v5p53u2m8rdl17i
Arv-put happily creates the collection without checking the blocks. The blocks don't exist.
Workaround: blow away cache directory and re-run the arv-put command.
While this scenario is unusual today, we are going to a future with more ephemeral arvados installations, and this will become a more common occurrence.
Proposed solution: arv-put should double check that (some of? all of?) the blocks exist before creating a collection from a local cache.
Related issues
Updated by Ward Vandewege over 5 years ago
- Status changed from New to In Progress
Updated by Ward Vandewege over 5 years ago
- Status changed from In Progress to New
Updated by Tom Morris
Updated by Lucas Di Pentima over 5 years ago
- Assigned To set to Lucas Di Pentima
Updated by Lucas Di Pentima over 5 years ago
This seems to supposedly have being addressed on #14012, so maybe it's a regression?
Updated by Lucas Di Pentima over 5 years ago
- Related to Bug #14012: arv-put does not check cached signatures when resuming added
Updated by Lucas Di Pentima over 5 years ago
- Status changed from New to In Progress
Updated by Lucas Di Pentima over 5 years ago
After doing some tests on my own, didn't realized that I was using an old arv-put while bumping into the same issue described here.
Created a virtualenv from scratch and tried the latest arv-put:
(venv-arvados) lucas@veritas:~/arvados_local$ arv-put ../arvados/doc/.site/
2019-03-15 10:13:16 arvados.arv_put[23840] INFO: Calculating upload size, this could take some time...
2019-03-15 10:13:16 arvados.arv_put[23840] INFO: Resuming upload from cache file /home/lucas/.cache/arvados/arv-put/490dd3db31b4f11bf084de9b6711cca3
2019-03-15 10:13:20 arvados.arv_put[23840] ERROR: arv-put: Resume cache contains invalid signature: it may have expired
or been created with another Arvados user's credentials.
Switch user or use one of the following options to restart upload:
--no-resume to start a new resume cache.
--no-cache to disable resume cache. (X-Request-Id: req-g0lnb1xfvj6262dbzjx9)
(venv-arvados) lucas@veritas:~/arvados_local$ arv-put --version
/home/lucas/venv-arvados/bin/arv-put 1.3.1.20190308210512
Updated by Lucas Di Pentima over 5 years ago
- Status changed from In Progress to Closed
Ticket closed as the current arv-put version already has a fix for this issue.