Project

General

Profile

Actions

Feature #8502

open

[SDKs] Fetch all results with the CLI `list` command

Added by Joshua Randall about 8 years ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
SDKs
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

It would be nice if the arv CLI `list` command had a way to handle fetching more records than the API server allows, by doing the batching for you and just giving all of the matching records requested. Implementing this externally is not trivial, so I think it would make sense for the CLI to do it internally.

I'd suggest something like a `--batch` or `-b` option that enabled this behaviour.

If a `-l` limit is also given, that should still be respected, but there should also be a way to signify that you want ALL of the matching records. There are at least a few options for how that might work:
- the default behaviour of `-l` could be changed when `--batch` is in effect (such that not specifying `-l` implies no limit)
- a special value for `-l` could indicate "no limit" (e.g. `--limit -1` or `-l -1` although the readability of the latter leaves something to be desired)
- another new option `--all` that overrides the setting of `-l`

It may also be advantageous to be able to specify the batch size, so perhaps the `--batch` / `-b` options could take an (optional?) numeric argument.

It probably would be a good idea to also document what the expected behaviour would be if data changes between the first and last batch, although I suspect there is functionality missing from the API server that would make it easy to guarantee a consistent set of results (e.g. using cursors).

Actions #1

Updated by Brett Smith about 8 years ago

  • Subject changed from automatically fetching all results with the CLI `list` command to [SDKs] Fetch all results with the CLI `list` command
Actions #2

Updated by Brett Smith about 8 years ago

  • Target version set to Arvados Future Sprints
Actions #3

Updated by Ward Vandewege almost 3 years ago

  • Target version deleted (Arvados Future Sprints)
Actions #4

Updated by Peter Amstutz about 1 year ago

  • Release set to 60
Actions #5

Updated by Peter Amstutz about 2 months ago

  • Target version set to Future
Actions

Also available in: Atom PDF