Story #21146


Replace ws4py dependency from PySDK

Added by Brett Smith about 1 month ago. Updated 2 days ago.

In Progress
Assigned To:
Start date:
Due date:
% Done:


Estimated time:
(Total: 0.00 h)
Story points:


The GitHub project says "[Project on Hiatus]"

The last commit just removed a similar warning from the README.

This is becoming a liability because the project build structure relies on a stack that's more and more deprecated. It's becoming harder to install on newer distributions and that problem is only going to get worse over time.

Find and switch to a replacement while maintaining the API of and other affected modules.

Subtasks 1 (1 open0 closed)

Task #21242: Review 21146-pysdk-new-websocketsIn ProgressPeter Amstutz12/01/2023


Related issues

Related to Arvados - Story #20846: Support Ubuntu 22.04 LTSIn ProgressBrett Smith10/30/2023

Actions #1

Updated by Brett Smith about 1 month ago

Actions #2

Updated by Peter Amstutz about 1 month ago

  • Target version changed from To be groomed to Development 2024-01-03 sprint
Actions #3

Updated by Brett Smith 11 days ago

The good news is we built our own class that Has-A client underneath, rather than subclassing. Thanks to that, here is the entire API we need to support.

  • __init__ should start a daemon thread to run the connection.
  • Each f argument is a filter.
class EventClient(object):
    def __init__(self, url, filters, on_event_cb, last_log_id): ...
    def subscribe(self, f, last_log_id=None): ...
    def unsubscribe(self, f): ...
    def close(self, code=1000, reason='', timeout=0): ...
    def on_event(self, m): ...
    def on_closed(self): ...
    def run_forever(self): ...
Actions #4

Updated by Brett Smith 11 days ago

We can use the websockets package for this. It has a thread-based client that's pretty compatible with our needs. The main difference is that the API is not callback-oriented, so we'll need to write that glue code ourselves, but that's not hard.

The license is 3-clause BSD so it's compatible with our SDK. Support is available through Tidelift so it's a mature project. Last release was a month ago so it's actively maintained. And of course it's the top result when you search "websockets" on PyPI.

Actions #5

Updated by Brett Smith 6 days ago

  • Assigned To set to Brett Smith
  • Status changed from New to In Progress

I have a branch that passes existing tests. I need to add a couple tests (due to different behavior in the new library) and docs.

Actions #6

Updated by Brett Smith 2 days ago

21146-pysdk-new-websockets @ 187f790d8b71c5c7b54a599bec6e58950d7116dc - developer-run-tests: #3938

remainder rerun developer-run-tests-remainder: #4142 - This has a different failure, so the intersection of both runs pass all tests 🙄

  • All agreed upon points are implemented / addressed.
    • Yes
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • N/A
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • See above
  • Documentation has been updated.
    • Yes
  • Behaves appropriately at the intended scale (describe intended scale).
    • I have not benchmarked to compare performance of ws4py vs. websockets or whatever, but it seems unlikely to be a significant difference.
  • Considered backwards and forwards compatibility issues between client and server.
    • The new EventClient maintains its entire public API, including methods, argument names, attributes, and semantics. Some of the code organization came about specifically to maintain semantics. For example, I thought about moving the last_log_id updating from on_event to run, but then weird things might've happened if someone subclassed EventClient with their own on_event.
  • Follows our coding standards and GUI style guidelines.
    • Yes

Also available in: Atom PDF