Arvados

Idea #21017

Updated by Brett Smith 8 months ago

A lot of code in the Python SDK cookbook that coordinates low-level API calls should just be methods that are defined in the SDK itself. Why force every user to copy-paste these code snippets when we can just give them the method directly? 

 Here are common general-purpose container/request functions that could be useful. I'm not sure what module they should go in. It might make sense to add them to a new @arvados.container@ module. 

 <pre><code class="python">def lookup( container_started( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
 ) -> Container | None: bool: ... 
 # If container is a str, it's a UUID for one of the other types, fetch that. 
 # If it's a container request, return False if container_uuid is None, else fetch that. 
 # Return the resulting container. 
 # Note that lookup should return None *only* if it handles a container 
 # request with no container_uuid set. 
 # Other problems like malformed UUID, other object types, container request 
 # without container_uuid selected, etc. should all raise their corresponding 
 # exceptions. 

 def container_started( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
 ) -> bool: 
     container = lookup(client, container) 
     return container['status'] `return status not in {'Queued', 'Locked'} 'Locked'}` 

 def container_finished( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
 ) -> bool: 
     container = lookup(client, container) 
     return container['status'] ... 
 # Same dance as above, `return status in {'Cancelled', 'Complete'} 'Complete'}` 

 def container_succeeded( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
         success: typing.Container[int]=frozenset([0]), 
 ) -> bool: 
     container = lookup(client, container) 
     return ( 
         container['status'] ... 
 # Same dance as above, `return status == 'Complete' 
         and container['exit_code'] exit_code in success 
     ) success` 

 def child_requests( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
         filters: list[list[str]]=[], 
         select: list[str]=[], 
         depth: int | None=None, 
 ) -> Iterator[ContainerRequest]: ... 
 # lookup container, yield from Same dance as above, return the obvious `keyset_list_all` calls, down to depth if given call. 

 def child_containers( 
         client: ArvadosAPIClient, 
         container: str | Container | ContainerRequest, 
         request_filters: list[list[str]]=[], 
         container_filters: list[list[str]]=[], 
         select: list[str]=[], 
         depth: int | None=None, 
 ) -> Iterator[Container]: ... 
 # Same dance as above, call `child_requests` to get container_uuids, then 
 # yield from return the obvious `keyset_list_all` calls call with those UUIDs, down to depth if given. UUIDs. 
 # Note that `child_requests` can be called with a very narrow `select` for 
 # optimization, and can add the filter `container_uuid is not null`. 
 </code></pre>

Back