Project

General

Profile

Routing multi cluster requests » History » Version 3

Peter Amstutz, 06/21/2017 03:39 PM

1 2 Peter Amstutz
h1. Routing multi cluster requests
2 1 Peter Amstutz
3 2 Peter Amstutz
h2. Concept
4
5 1 Peter Amstutz
The goal of federation is to present an interface that fuses multiple clusters into a single view.
6
7
This requires a router or proxy which determines which cluster(s) a request should go.  This could exist in several places: entirely in the client, in a dedicated request router service, on the local cluster's API server.
8
9 2 Peter Amstutz
For the purposes of this discussion, we'll consider how this works when implemented entirely on the client side.  A router service is likely to have similar request handling behavior but has the additional requirement to act as a transparent API server proxy.
10
11
h2. Examples
12
13
My "home cluster" is qr1hi.  I have a token qr1hi-secretsecretsecret.
14
15
h3. I want to read a collection on c97qk using the Python SDK.
16
17
<pre>
18
c = CollectionReader("c97qk-...")
19
</pre>
20
21
# The CollectionReader calls the request router class.
22
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com.
23
# The request router uses the "salted" token hmac(c97qk, qr1hi-secretsecretsecret) &rarr; qr1hi-secretsecretc97qk
24
# c97qk gets the token and notices the qr1hi prefix.
25
# c97qk contacts qr1hi to determine if the token is valid and what user is associated with the token.
26
# c97qk caches the token and sets current_user.  The request proceeds as normal.
27
# The request gets the response and returns it to CollectionReader.
28
#* manifest_text needs to be munged by either the c97qk or the request router to add the hint "+K@c97qk" so that blocks will be fetched from c97qk.
29
30
h3. I want to search for a collection across clusters
31
32
<pre>
33
c = router.collections().list(filters=[["name", "like", "sample-1234%"]]).execute()
34
</pre>
35
36
# The router has a "search list" of clusters (where does this come from??? maybe an attribute of the primary user account  on qr1hi?)
37
# The router sends the request to each cluster in parallel using federated identity / salted token described above.
38
# The router gathers the results.
39
# The router collates the results (will need to understand "order" option to do this properly)
40
# Collated results are returned
41
# Paging - ???  likely need to keep track of some state locally to be able to be able to issue correct follow-up requests to each cluster.  Can have consistent ordering within a page but not across pages unless all pages are fetched first.
42
43
h3. I want to create a collection on another cluster.
44
45
Provide "owner_uuid" of a project or group on a foreign cluster.
46
47
<pre>
48
router.collections().create(body={"owner_uuid": "c97qk-...."}).execute()
49
</pre>
50
51
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .
52
# The cluster determines if the user has write access to the group or project and validates the create request as normal.
53
# The newly created record is returned.
54
55
No "owner_uuid" means creating the object on the "home" cluster.
56
57
h3. I want to update an object on another cluster.
58
59
<pre>
60
router.collections().update(uuid="c97qk-....", body={....}).execute()
61
</pre>
62
63
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .
64
# The cluster determines if the user has write access to object and validates the update request as normal.
65
# The updated record is returned.
66
67 3 Peter Amstutz
h3. I want to change the ownership of a remote object to a project on my home cluster.
68 1 Peter Amstutz
69 3 Peter Amstutz
The object is located on c97qk and currently owned by me, I'd like to make it owned by a project qr1hi-...
70
71
# Route the "update" as described above to c97qk.
72
# c97qk contacts qr1hi and asks if the user has write access to the project.
73
# The object is updated and returned to the user
74
75
(This suggests I can only share things with groups on the same home cluster as me.  hmm.)