Project

General

Profile

Actions

Idea #21132

closed

Document request objects in arvados.api_resources

Added by Brett Smith 7 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Documentation
Start date:
11/18/2023
Due date:
Story points:
0.5
Release relationship:
Auto

Description

In arvados.api_resources, all the API methods are documented as returning an API resource directly. This isn't correct. Instead they return a request object that you must execute() to make the API query and get the response.

Document this object. The very simplest core of it is along the lines of:

T = TypeVar('T', bound=Dict)
class ArvadosAPIRequest(Generic[T]):
    def execute(self, num_retries: int=0) -> T: ...

But check googleapiclient to see if there are other methods, or execute arguments, worth documenting for users. (If there's an existing typestub for the library, we might consider using that as a base, if licensing allows it.) This definition can probably just go in the static part of the generated file in discovery2pydoc.py.

Once this is defined, all the API methods need their return type updated from T to ArvadosAPIRequest[T]. That's return_annotation in Method.signature in discovery2pydoc.


Subtasks 1 (0 open1 closed)

Task #21202: Review 21132-api-resources-fixesResolvedPeter Amstutz11/18/2023Actions
Actions #1

Updated by Brett Smith 6 months ago

  • Target version changed from To be scheduled to Development 2023-11-29 sprint
  • Assigned To set to Brett Smith
  • Status changed from New to In Progress

21132-api-resources-fixes @ 44f7408d23ccfbb51dbc06522302af5e4aa53da7 - developer-run-tests: #3912

  • All agreed upon points are implemented / addressed.
    • Yes. Note this branch includes fixes for both this ticket and #21136.
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • N/A
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • See above
  • Documentation has been updated.
    • N/A
  • Behaves appropriately at the intended scale (describe intended scale).
    • N/A
  • Considered backwards and forwards compatibility issues between client and server.
    • N/A
  • Follows our coding standards and GUI style guidelines.
    • discovery2pydoc.py doesn't have consistent docstrings because it's strictly internal tooling (although it does help method signatures to help you know what's what). Otherwise, yes.
Actions #2

Updated by Peter Amstutz 5 months ago

Brett Smith wrote in #note-1:

21132-api-resources-fixes @ 44f7408d23ccfbb51dbc06522302af5e4aa53da7 - developer-run-tests: #3912

  • All agreed upon points are implemented / addressed.
    • Yes. Note this branch includes fixes for both this ticket and #21136.
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • N/A
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • See above
  • Documentation has been updated.
    • N/A
  • Behaves appropriately at the intended scale (describe intended scale).
    • N/A
  • Considered backwards and forwards compatibility issues between client and server.
    • N/A
  • Follows our coding standards and GUI style guidelines.
    • discovery2pydoc.py doesn't have consistent docstrings because it's strictly internal tooling (although it does help method signatures to help you know what's what). Otherwise, yes.

It looks like the narrow goal of this ticket is just to make sure that the various API methods have an accurate return type (the newly defined generic ArvadosAPIRequest).

Strictly speaking the actual API returns the plain 'googleapiclient.http.HttpRequest' but what we're doing is declaring a subclass so something shows up on our doc page. Perhaps it would merit a line of explanation that ArvadosAPIRequest isn't a real class, but you should treat it as if it were the return type?

Otherwise this LGTM.

Actions #3

Updated by Brett Smith 5 months ago

Peter Amstutz wrote in #note-2:

Strictly speaking the actual API returns the plain 'googleapiclient.http.HttpRequest' but what we're doing is declaring a subclass so something shows up on our doc page. Perhaps it would merit a line of explanation that ArvadosAPIRequest isn't a real class, but you should treat it as if it were the return type?

Yes, and this is true of every class in the module. Literally everything in arvados.api_resources is fake and only exists as a channel to provide documentation. The module docstring kind of hints at this but I agree it could be clearer. What do you think about this?

_MODULE_PYDOC = '''Arvados API client documentation skeleton

This module documents the methods and return types provided by the Arvados API
client. Start with `ArvadosAPIClient`, which documents the methods available
from the API client objects constructed by `arvados.api`.

This module **only** exists to document the methods available on Arvados API
clients. The classes in this module do not have any implementation, and you
should not instantiate them in your code. The implementation of client objects
is generated dynamically at runtime from the Arvados API discovery document.
'''
Actions #4

Updated by Peter Amstutz 5 months ago

Brett Smith wrote in #note-3:

Peter Amstutz wrote in #note-2:

Strictly speaking the actual API returns the plain 'googleapiclient.http.HttpRequest' but what we're doing is declaring a subclass so something shows up on our doc page. Perhaps it would merit a line of explanation that ArvadosAPIRequest isn't a real class, but you should treat it as if it were the return type?

Yes, and this is true of every class in the module. Literally everything in arvados.api_resources is fake and only exists as a channel to provide documentation. The module docstring kind of hints at this but I agree it could be clearer. What do you think about this?

[...]

I would phrase it like "This module exists to document the shape and interface of objects returned by the Arvados API and should be used as a reference, however the actual runtime classes will be different because..."

Actions #5

Updated by Brett Smith 5 months ago

Peter Amstutz wrote in #note-4:

I would phrase it like "This module exists to document the shape and interface of objects returned by the Arvados API and should be used as a reference, however the actual runtime classes will be different because..."

I committed and pushed a version that tries to follow this structure and hit the same beats, but with more Python-familiar terminology. Thanks.

Actions #6

Updated by Brett Smith 5 months ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved
Actions #7

Updated by Brett Smith 5 months ago

  • Release set to 67
Actions

Also available in: Atom PDF