Message queue¶

Arvados desired capabilities for message queue:

• Support for WebSocket transport to enable use by browser
• Client support for languages used in Arvados (Go, Python, Ruby, Javascript)
• Can apply Arvados permission model
• Pub/sub of live change events on records
  • Transactional subscribe request that returns most recent record + subscribes to subsequent changes
• Pub/sub of live Container logs
  • Transactional subscribe request that returns recent recent log history + subscribes to subsequent logs
  • In-order delivery of logs
• Handle temporary disconnects
• Easy to deploy (fits into our stack)
• Scales to at least 1000 streams without trouble

Investigate WAMP:

http://wamp-proto.org/

It uses WebSockets as its principal transport, and has client library implementations for the languages we care about (Go, Python, Javascript)

https://github.com/gammazero/nexus

https://github.com/crossbario/autobahn-python

https://github.com/crossbario/autobahn-js

Probably what we want to do is run a "WAMP router" that sits in the middle of the log producers, the logging microservice, and log listeners (browser).