Arvados

h1. Routing multi cluster requests

h2. Concept

The goal of federation is to present an interface that fuses multiple clusters into a single view.

This requires a router or proxy which determines which cluster(s) a request should go.  This could exist in several places: entirely in the client, in a dedicated request router service, on the local cluster's API server.

For the purposes of this discussion, we'll consider how this works when implemented entirely on the client side.  A router service is likely to have similar request handling behavior but has the additional requirement to act as a transparent API server proxy.

h2. Examples

My "home cluster" is qr1hi.  I have a token qr1hi-secretsecretsecret.

h3. I want to read a collection on c97qk using the Python SDK.

<pre>

c = CollectionReader("c97qk-...")

</pre>

# The CollectionReader calls the request router class.

# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com.

# The request router uses the "salted" token hmac(c97qk, qr1hi-secretsecretsecret) → qr1hi-secretsecretc97qk

# c97qk gets the token and notices the qr1hi prefix.

# c97qk contacts qr1hi to determine if the token is valid and what user is associated with the token.

# c97qk caches the token and sets current_user.  The request proceeds as normal.

# The request gets the response and returns it to CollectionReader.

#* manifest_text needs to be munged by either the c97qk or the request router to add the hint "+K@c97qk" so that blocks will be fetched from c97qk.

h3. I want to search for a collection across clusters

<pre>

c = router.collections().list(filters=[["name", "like", "sample-1234%"]]).execute()

</pre>

# The router has a "search list" of clusters (where does this come from??? maybe an attribute of the primary user account  on qr1hi?)

# The router sends the request to each cluster in parallel using federated identity / salted token described above.

# The router gathers the results.

# The router collates the results (will need to understand "order" option to do this properly)

# Collated results are returned

# Paging - ???  likely need to keep track of some state locally to be able to be able to issue correct follow-up requests to each cluster.  Can have consistent ordering within a page but not across pages unless all pages are fetched first.

h3. I want to create a collection on another cluster.

Provide "owner_uuid" of a project or group on a foreign cluster.

<pre>

router.collections().create(body={"owner_uuid": "c97qk-...."}).execute()

</pre>

# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .

# The cluster determines if the user has write access to the group or project and validates the create request as normal.

# The newly created record is returned.

No "owner_uuid" means creating the object on the "home" cluster.

h3. I want to update an object on another cluster.

<pre>

router.collections().update(uuid="c97qk-....", body={....}).execute()

</pre>

# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .

# The cluster determines if the user has write access to object and validates the update request as normal.

# The updated record is returned.

h3. I want to change the ownership of a remote object to a project on my home cluster.

The object is located on c97qk and currently owned by me, I'd like to make it owned by a project qr1hi-...

# Route the "update" as described above to c97qk.

# c97qk contacts qr1hi and asks if the user has write access to the project.

# The object is updated and returned to the user

(This suggests I can only share things with groups on the same home cluster as me.  hmm.)