Websocket v0 shim

Objective:
  • Go-based websocket server.
  • Implement just enough of the existing Rails/Puma server's API so it can be dropped in as a replacement without breaking existing clients.
  • Leave room to implement v1 Websocket server in the same server process, so new clients can use v1 while old clients continue using v0.

Functional

Endpoint

Provide v0 API at "/websocket".

Subscription requests

{"method":"subscribe", "last_log_id":12345, "filters":[["event_type","=","X"]]}
{"method":"subscribe", "last_log_id":12345}
{"method":"subscribe"}

Ignore filters requested by the client. Just send all events (subject to permissions).

Optional(?): Implement event_type filters as used by arv-mount. Otherwise, every arv-mount process will receive a lot of job/container log data that it can't use.

If last_log_id is given,
  • start listening for new events (but don't process any yet)
  • load old rows from the logs table with id >= last_log_id, and process them as if they were new
  • note the largest ID seen in the old rows (if any); save this in last_log_id
  • process new events as they arrive, but skip events with id <= last_log_id

Ignore "unsubscribe" messages.

Event messages

Send only these fields from each event/log:
  • id
  • object_uuid
  • object_kind
  • object_owner_uuid
  • properties (only if object is a container or job)
  • created_at

Slow clients

Allocate a finite sized buffer/channel of events for each connected client. If a client receives events too slowly and its buffer fills up:
  • print a log message.
  • (if possible) send an error message to the client.
  • terminate the connection.

Logging

Print JSON-formatted log entries on stderr.

Print a log entry when a client connects.

Print a log entry when a client disconnects. Show counters for
  • Number of events sent
  • Number of bytes sent

Implementation

Database access

Connect directly to PostgreSQL to receive event notifications.

Permission

Defer to the REST API for permission checks.

Before sending an event to a subscriber, check whether the subscriber's token is able to retrieve the object_uuid referenced in the event.

To check permission, perform a HEAD request for the event's object_uuid, using the subscriber's token.

Maintain a permission cache {(token, object_uuid) → bool}. When the cache grows bigger than the configured number of entries (default 128K?), clear it and emit a log message.

Configuration

Configuration will include
  • PostgreSQL connection info/credentials
  • API server endpoint (use arvados.Client like keep-balance et al.)
  • Maximum connection buffer size (buffered events per connection)
  • Maximum concurrent connections