Feature #8502
open[SDKs] Fetch all results with the CLI `list` command
Description
It would be nice if the arv CLI `list` command had a way to handle fetching more records than the API server allows, by doing the batching for you and just giving all of the matching records requested. Implementing this externally is not trivial, so I think it would make sense for the CLI to do it internally.
I'd suggest something like a `--batch` or `-b` option that enabled this behaviour.
If a `-l` limit is also given, that should still be respected, but there should also be a way to signify that you want ALL of the matching records. There are at least a few options for how that might work:
- the default behaviour of `-l` could be changed when `--batch` is in effect (such that not specifying `-l` implies no limit)
- a special value for `-l` could indicate "no limit" (e.g. `--limit -1` or `-l -1` although the readability of the latter leaves something to be desired)
- another new option `--all` that overrides the setting of `-l`
It may also be advantageous to be able to specify the batch size, so perhaps the `--batch` / `-b` options could take an (optional?) numeric argument.
It probably would be a good idea to also document what the expected behaviour would be if data changes between the first and last batch, although I suspect there is functionality missing from the API server that would make it easy to guarantee a consistent set of results (e.g. using cursors).
Updated by Brett Smith over 8 years ago
- Subject changed from automatically fetching all results with the CLI `list` command to [SDKs] Fetch all results with the CLI `list` command
Updated by Brett Smith over 8 years ago
- Target version set to Arvados Future Sprints
Updated by Ward Vandewege over 3 years ago
- Target version deleted (
Arvados Future Sprints)