Arvados

h1. Websocket server

(early draft)

{{toc}}

h2. Background

The Rails API server can function as a websocket server. Clients (notably Workbench, arv-mount, arv-ws) use it to listen for events without polling.

Problems with current implementation:

* Unreliable. See #9427, #8277

* Resource-heavy (one postgres connection per connected client, uses lots of memory)

* Logging is not very good

* Updates look like database records instead of API responses (e.g., computed fields are missing, collection manifest_text has no signatures)

* Offers an API for catching up on missed events after disconnecting/reconnecting, but this API (let alone the code) isn't enough to offer a "don't miss any events, don't send any events twice" guarantee. See #9388

#8460

h2. Desired features

Monotonically increasing event IDs, so clients can (meaningfully) request "all matching events since X"

h2. Design sketch

New server, written in Go.

One goroutine per connected client.

One database connection receiving notifications about new logs. (Possibly still N database connections serving "catch-up" messages to N clients.)

h2. Libraries

Websocket:

* https://godoc.org/golang.org/x/net/websocket

PostgreSQL:

* https://godoc.org/github.com/lib/pq via https://godoc.org/database/sql

* https://godoc.org/github.com/lib/pq#hdr-Notifications and https://godoc.org/github.com/lib/pq/listen_example

h2. Obstacles

#8565, #8566

h2. Related

It might be expedient to offload synchronization to some existing software that does this well.

* "Apache Zookeeper":https://zookeeper.apache.org/doc/trunk/zookeeperOver.html -- "Coordination services are notoriously hard to get right. They are especially prone to errors such as race conditions and deadlock. The motivation behind ZooKeeper is to relieve distributed applications the responsibility of implementing coordination services from scratch."

* Google "Chubby":http://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf paper

* NSQ http://nsq.io/