Project

General

Profile

Routing multi cluster requests » History » Version 5

Peter Amstutz, 06/21/2017 03:51 PM

1 2 Peter Amstutz
h1. Routing multi cluster requests
2 1 Peter Amstutz
3 2 Peter Amstutz
h2. Concept
4
5 1 Peter Amstutz
The goal of federation is to present an interface that fuses multiple clusters into a single view.
6
7
This requires a router or proxy which determines which cluster(s) a request should go.  This could exist in several places: entirely in the client, in a dedicated request router service, on the local cluster's API server.
8
9 2 Peter Amstutz
For the purposes of this discussion, we'll consider how this works when implemented entirely on the client side.  A router service is likely to have similar request handling behavior but has the additional requirement to act as a transparent API server proxy.
10
11
h2. Examples
12
13
My "home cluster" is qr1hi.  I have a token qr1hi-secretsecretsecret.
14
15
h3. I want to read a collection on c97qk using the Python SDK.
16
17
<pre>
18
c = CollectionReader("c97qk-...")
19
</pre>
20
21
# The CollectionReader calls the request router class.
22
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com.
23
# The request router uses the "salted" token hmac(c97qk, qr1hi-secretsecretsecret) &rarr; qr1hi-secretsecretc97qk
24
# c97qk gets the token and notices the qr1hi prefix.
25
# c97qk contacts qr1hi to determine if the token is valid and what user is associated with the token.
26
# c97qk caches the token and sets current_user.  The request proceeds as normal.
27
# The request gets the response and returns it to CollectionReader.
28
#* manifest_text needs to be munged by either the c97qk or the request router to add the hint "+K@c97qk" so that blocks will be fetched from c97qk.
29
30
h3. I want to search for a collection across clusters
31
32
<pre>
33
c = router.collections().list(filters=[["name", "like", "sample-1234%"]]).execute()
34
</pre>
35
36
# The router has a "search list" of clusters (where does this come from??? maybe an attribute of the primary user account  on qr1hi?)
37
# The router sends the request to each cluster in parallel using federated identity / salted token described above.
38
# The router gathers the results.
39
# The router collates the results (will need to understand "order" option to do this properly)
40
# Collated results are returned
41
# Paging - ???  likely need to keep track of some state locally to be able to be able to issue correct follow-up requests to each cluster.  Can have consistent ordering within a page but not across pages unless all pages are fetched first.
42
43 4 Peter Amstutz
Another case: I want to list the contents of a project across clusters.  Same query process.
44
45
<pre>
46
c = router.collections().list(owner_uuid="qr1hi-....").execute()
47
</pre>
48
49 2 Peter Amstutz
h3. I want to create a collection on another cluster.
50
51
Provide "owner_uuid" of a project or group on a foreign cluster.
52
53
<pre>
54
router.collections().create(body={"owner_uuid": "c97qk-...."}).execute()
55
</pre>
56
57
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .
58
# The cluster determines if the user has write access to the group or project and validates the create request as normal.
59
# The newly created record is returned.
60
61
No "owner_uuid" means creating the object on the "home" cluster.
62
63
h3. I want to update an object on another cluster.
64
65
<pre>
66
router.collections().update(uuid="c97qk-....", body={....}).execute()
67
</pre>
68
69
# The request router class examines the prefix c97qk and contacts c97qk.arvadosapi.com using federated identity / salted token described above .
70
# The cluster determines if the user has write access to object and validates the update request as normal.
71
# The updated record is returned.
72
73 3 Peter Amstutz
h3. I want to change the ownership of a remote object to a project on my home cluster.
74 1 Peter Amstutz
75 3 Peter Amstutz
The object is located on c97qk and currently owned by me, I'd like to make it owned by a project qr1hi-...
76
77 5 Peter Amstutz
# Route an "update" request to change "owner_uuid" to c97qk as described above.
78 3 Peter Amstutz
# c97qk contacts qr1hi and asks if the user has write access to the project.
79
# The object is updated and returned to the user
80
81
(This suggests I can only share things with groups on the same home cluster as me.  hmm.)
82 4 Peter Amstutz
83
h3. I want to change the ownership of an object on my home cluster object to a project on a remote cluster.
84
85
# Route the "update" as described above to qr1hi.
86
# qr1hi contacts c97qk _using the c97qk salted token_ and asks if the user has write access to the project.
87
# The object is updated and returned to the user
88
89
h3. I want to change the ownership of an object from one remote project (c97qk) to another (4xphq).
90
 
91
Can't be done directly (???) because c97qk and 4xphq don't talk to each other directly.  (The token given to c97qk is not valid for accessing 4xphq and likewise).  Could be done as a two-step process where ownership is assigned from c97qk to qr1hi, then from qr1hi to 4xphq.