Project

General

Profile

Message queue » History » Revision 9

Revision 8 (Peter Amstutz, 11/01/2017 08:29 PM) → Revision 9/13 (Peter Amstutz, 11/01/2017 08:30 PM)

h1. Message queue 

 Arvados desired capabilities for message queue: 

 * Support for WebSocket transport to enable use by browser 
 * Client support for languages used in Arvados (Go, Python, Ruby, Javascript) 
 * Can apply Arvados permission model 
 * Pub/sub of live change events on records 
 ** Transactional subscribe request that returns most recent record + subscribes to subsequent changes 
 * Pub/sub of live Container logs 
 ** Transactional subscribe request that returns recent recent log history + subscribes to subsequent logs 
 ** In-order delivery of logs 
 * Handle temporary disconnects 
 * Easy to deploy (fits into our stack) 
 * Scales to at least 1000 streams without trouble Reasonable performance / resource usage 

 Investigate WAMP: 

 http://wamp-proto.org/ 

 It uses WebSockets as its principal transport, and has client library implementations for the languages we care about (Go, Python, Javascript) 

 https://github.com/gammazero/nexus 

 https://github.com/crossbario/autobahn-python 

 https://github.com/crossbario/autobahn-js 

 Probably what we want to do is run a "WAMP router" that sits in the middle of the log producers, the logging microservice, and log listeners (browser).