Project

General

Profile

Message queue » History » Version 9

Peter Amstutz, 11/01/2017 08:30 PM

1 1 Peter Amstutz
h1. Message queue
2
3 7 Peter Amstutz
Arvados desired capabilities for message queue:
4
5
* Support for WebSocket transport to enable use by browser
6
* Client support for languages used in Arvados (Go, Python, Ruby, Javascript)
7
* Can apply Arvados permission model
8
* Pub/sub of live change events on records
9
** Transactional subscribe request that returns most recent record + subscribes to subsequent changes
10
* Pub/sub of live Container logs
11
** Transactional subscribe request that returns recent recent log history + subscribes to subsequent logs
12
** In-order delivery of logs
13
* Handle temporary disconnects
14 8 Peter Amstutz
* Easy to deploy (fits into our stack)
15 9 Peter Amstutz
* Scales to at least 1000 streams without trouble
16 7 Peter Amstutz
17 1 Peter Amstutz
Investigate WAMP:
18
19
http://wamp-proto.org/
20
21 2 Peter Amstutz
It uses WebSockets as its principal transport, and has client library implementations for the languages we care about (Go, Python, Javascript)
22 1 Peter Amstutz
23
https://github.com/gammazero/nexus
24 2 Peter Amstutz
25
https://github.com/crossbario/autobahn-python
26
27
https://github.com/crossbario/autobahn-js
28 1 Peter Amstutz
29
Probably what we want to do is run a "WAMP router" that sits in the middle of the log producers, the logging microservice, and log listeners (browser).