Bug #11788
closed
[arv-put] should preserve directory structure when given directory references
Added by Tom Morris over 7 years ago.
Updated over 7 years ago.
Description
Neither of these examples worked to create a collection containing files with the subdirectory references to the Config and Interop directories preserved:
arv-put Config InterOp SampleSheet.csv
arv-put Config/ InterOp/ SampleSheet.csv
Instead the contents of both subdirectories were flattened into the top level.
If given a directory reference, that directory should be included in the paths stored in the resulting collection.
- Assigned To set to Lucas Di Pentima
Might be good if this API change could be toggled by the user. For example, if there is a file foo/bar/x.txt,
- rsync foo/bar/ dest:baz/ creates dest:baz/x.txt (analogous to existing arv-put behavior)
- tar cf - foo/bar/ creates a tarball with foo/bar/x.txt (analogous to proposed arv-put behavior)
- Status changed from New to In Progress
Updates at cf9874c59
Test run: https://ci.curoverse.com/job/developer-run-tests/334/
Fixed the specific case that when passing a directory reference to arv-put, it skips uploading the directory and instead goes ahead uploading its contents directly.
So, now the directory uploads behave like this:
- "
arv-put relative/dirpath1 another/relative/dirpath2/ afile.txt" → ./afile.txt, ./path1/..., ./path2/...
- "
arv-put /absolute/path/of/some/dir" → ./dir/...
To make arv-put behave more like rsync, I think we could add a new argument --destination-path, so that all referenced dirs/files are uploaded inside that named subcollection, allowing the previous behavior (dir flattening) by passing "--destination-path /", for example.
Offline discussion concluded we should behave like rsync. I think that means, if the current directory is "/cwd" and the following local files exist:
- /cwd/dir1/file1.txt
- /cwd/dir2/file2.txt
We should have:
| arv-put args |
resulting collection contents |
| dir1 |
/dir1/file1.txt |
| /cwd/dir1 |
/dir1/file1.txt |
| dir1/ |
/file1.txt |
| /cwd/dir1/ |
/file1.txt |
| dir1 dir2 |
/dir1/file1.txt
/dir2/file2.txt |
| dir1 dir2/ |
/dir1/file1.txt
/file2.txt |
| /cwd/dir1 /cwd/dir2/ |
/dir1/file1.txt
/file2.txt |
If/when we add a "--relative" flag, it should work like this:
| arv-put args |
resulting collection contents |
| --relative dir1 |
/dir1/file1.txt |
| --relative /cwd/dir1 |
/cwd/dir1/file1.txt |
| --relative dir1/ |
/dir1/file1.txt |
| --relative /cwd/dir1/ |
/cwd/dir1/file1.txt |
| --relative dir1 dir2 |
/dir1/file1.txt
/dir2/file2.txt |
| --relative dir1 dir2/ |
/dir1/file1.txt
/dir2/file2.txt |
| --relative /cwd/dir1 /cwd/dir2/ |
/cwd/dir1/file1.txt
/cwd/dir2/file2.txt |
| --relative /cwd/dir1 /cwd/./dir2/ |
/cwd/dir1/file1.txt
/dir2/file2.txt |
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:5598bbc3c71da60b7b7a665b36495b957d6a3c52.
Also available in: Atom
PDF
Updated by 
Updated by