Bug #3400

Updated by Tom Clegg almost 6 years ago

Background: If a developer writes something like @Link.where(...).each do { ... }@ and the API call results in more than the result limit (default 100), the code will only iterate over the first 100 results. This is error prone and leads to subtle bugs where code works as expected when there are less than 100 results but suddenly starts losing results when there are more than 100 results. The client libraries SDKs should clearly distinguish between cases where paging is desired vs iterating over the entire result set. This problem currently exists in all client SDKs: SDKs, including Python, Go, Ruby, Java, Perl, command line, and Rails (Workbench).

Highest priorities:
* Add and document "all pages" mode to Python and Rails/Workbench libraries
* Document the limitation in other libraries. If possible, show an example of how to do it with the @offset@ parameter, like the @$newtask_results@ code in source:sdk/cli/bin/crunch-job#L1056.