Events API » History » Version 2

Tom Clegg, 10/19/2016 10:24 AM

1 1 Tom Clegg
h1. Events API
2 1 Tom Clegg
3 1 Tom Clegg
4 1 Tom Clegg
5 1 Tom Clegg
6 1 Tom Clegg
7 1 Tom Clegg
See also: [[Websocket server]]
8 1 Tom Clegg
9 1 Tom Clegg
h1. Purpose
10 1 Tom Clegg
11 1 Tom Clegg
The Events API serves to notify processes about events that interest them _as soon as possible after those events happen_.
12 1 Tom Clegg
13 1 Tom Clegg
(The history of events that have happened in the past is also interesting, but that's addressed by the Logs API, not the Events API.)
14 1 Tom Clegg
15 1 Tom Clegg
h1. Conceptual model
16 1 Tom Clegg
17 1 Tom Clegg
An event reports a change to the state of an object.
18 1 Tom Clegg
19 1 Tom Clegg
The fact that an object's state has changed is meaningful only when its previous state is known. For example, if a client asks "tell me the next time object X changes" at nearly the same time X changes, the response depends on whether the request arrives before or after the change occurs.
20 1 Tom Clegg
21 1 Tom Clegg
Therefore, the Events API should support operations like:
22 1 Tom Clegg
* "tell me the current state of X, and then notify me next time it changes"
23 1 Tom Clegg
* "tell me as soon as X differs from my cached copy that has Etag E"
24 1 Tom Clegg
25 1 Tom Clegg
An "event stream" is a sequence of events about an object, starting from an implicit or explicit known state.
26 1 Tom Clegg
27 1 Tom Clegg
h1. Essential features
28 1 Tom Clegg
29 1 Tom Clegg
h2. Multiple streams
30 1 Tom Clegg
31 1 Tom Clegg
The Events API supports multiplexing event streams on a single connection. The cost of setting up and maintaining an event channel can be non-trivial, and the sequence of events concerning multiple related objects may be significant.
32 1 Tom Clegg
33 1 Tom Clegg
It is possible to add and remove event streams on an existing connection, without interrupting other streams.
34 1 Tom Clegg
35 1 Tom Clegg
It is permitted to hold a connection open with no event streams, but the server may close such connections after some time threshold.
36 1 Tom Clegg
37 1 Tom Clegg
h2. Delivery guarantees
38 1 Tom Clegg
39 1 Tom Clegg
In general, the Events API cannot guarantee that every event will be delivered.
40 1 Tom Clegg
41 1 Tom Clegg
However, there are specific cases where it is possible to detect missed events and notify the client.
42 1 Tom Clegg
43 1 Tom Clegg
If some events are dropped but the event stream is still open (for example, a server-side buffer overflows when a client is receiving data too slowly) the stream must indicate this to the client as soon as possible, and no later than the next event.
44 1 Tom Clegg
* The "missed events" signal does not necessarily specify the number of missed events.
45 1 Tom Clegg
* The server is permitted to send a "missed events" signal even if no events were missed.
46 1 Tom Clegg
47 1 Tom Clegg
Depending on the application, a client might respond to a "missed events" signal by
48 1 Tom Clegg
* restarting the affected streams immediately
49 1 Tom Clegg
* restarting the affected streams only if they stay silent for some timeout period
50 1 Tom Clegg
* doing nothing
51 1 Tom Clegg
* hanging up
52 1 Tom Clegg
53 1 Tom Clegg
h2. Event message content
54 1 Tom Clegg
55 1 Tom Clegg
Each event includes the UUID and Etag of the changed object.
56 1 Tom Clegg
57 1 Tom Clegg
If requested by the client, and supported by the server, events may also include other object attributes.
58 1 Tom Clegg
59 1 Tom Clegg
h1. Additional features
60 1 Tom Clegg
61 1 Tom Clegg
h2. Event sequence
62 1 Tom Clegg
63 1 Tom Clegg
With the current API server, it may be possible to update an object twice in quick succession such that the modification timestamps are out of order: i.e., the current state of object X has modification time T1, even though the same object previously had modification time T2>T1. If this occurs, the Events API must return the T2 update before the T1 update (or not return the T2 update at all).
64 1 Tom Clegg
65 1 Tom Clegg
In order to support delivery mechanisms where messages are re-ordered in transit, the Events API should assign a strictly increasing integer ID to each event sent over a given connection. Client pseudocode:
66 1 Tom Clegg
67 1 Tom Clegg
68 1 Tom Clegg
receiveEvent(id, uuid, newEtag):
69 1 Tom Clegg
  if lastID[uuid] > id:
70 1 Tom Clegg
    # already received a newer update for this object
71 1 Tom Clegg
72 1 Tom Clegg
  currentEtag[uuid] = newEtag
73 1 Tom Clegg
  lastID[uuid] = id
74 1 Tom Clegg
75 1 Tom Clegg
76 1 Tom Clegg
Note these IDs are connection-specific: they cannot be used to reconnect and resume an event stream.
77 1 Tom Clegg
78 1 Tom Clegg
h2. Server-side event filters
79 1 Tom Clegg
80 1 Tom Clegg
Some clients will only be interested in a subset of possible changes. For example, a pipeline runner wants to know as soon as a container's "state" attribute changes, but might not care about other changes like "priority" or "progress".
81 1 Tom Clegg
82 1 Tom Clegg
Possible API features for reducing unnecessary work and network traffic:
83 1 Tom Clegg
# Allow clients to describe which attributes are interesting, e.g., @"select":["state"]@
84 1 Tom Clegg
# With each event, provide the list of changed attributes, e.g., @"changed":["state","output","log"]@, but not the attribute values themselves
85 1 Tom Clegg
86 1 Tom Clegg
These features might be tricky to implement efficiently for attributes that are computed on the fly. 
87 1 Tom Clegg
88 1 Tom Clegg
h2. Null stream
89 1 Tom Clegg
90 1 Tom Clegg
To simplify implementation of clients that subscribe to event streams but also retrieve some objects without listening for events, a client should be able to use the Events API to retrieve the current state of an object without subscribing to the object's event stream.
91 2 Tom Clegg
92 2 Tom Clegg
h2. Ownership-change events
93 2 Tom Clegg
94 2 Tom Clegg
Some clients need to know when an object is added or removed from a project.
95 2 Tom Clegg
96 2 Tom Clegg
When an object's owner_uuid changes, this event should be sent to:
97 2 Tom Clegg
# all clients subscribed to the object itself
98 2 Tom Clegg
# all clients subscribed to the old owner_uuid
99 2 Tom Clegg
# all clients subscribed to the new owner_uuid
100 2 Tom Clegg
101 2 Tom Clegg
Likewise, subscribing to stream X should cause clients to receive messages when a new object is created with owner_uuid=X, and when an object with owner_uuid=X is deleted.