Project

General

Profile

Actions

Bug #20507

closed

Fix websocket service

Added by Peter Amstutz 11 months ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
API
Story points:
-

Description

In the course of looking at #20449, it doesn't seem like workbench is receiving any events from websockets. It seems to be receiving a once-per-minute empty ping, but not any actual events.

I think this explains why most of the time the continuous-reloading behavior described in #20449 doesn't actually happen. (But sometimes it does, which means sometimes websockets works?)

We should either fix it or abandon it entirely and rely on polling.


Related issues

Related to Arvados - Bug #20449: Background refresh tasks of "all processes" issuesResolvedPeter Amstutz06/01/2023Actions
Has duplicate Arvados - Bug #20904: Investigate websockets issue seen by userDuplicatePeter AmstutzActions
Actions #1

Updated by Peter Amstutz 11 months ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz 11 months ago

  • Status changed from In Progress to New
Actions #3

Updated by Peter Amstutz 11 months ago

  • Category set to API
  • Description updated (diff)
Actions #4

Updated by Peter Amstutz 11 months ago

  • Description updated (diff)
Actions #5

Updated by Peter Amstutz 11 months ago

  • Related to Bug #20449: Background refresh tasks of "all processes" issues added
Actions #6

Updated by Brett Smith 10 months ago

I recently ran a toy websocket client against pirca (running 2.6.3), just subscribed to everything and dumping the records to JSON. It seemed to work, so websockets aren't completely broken.

Actions #7

Updated by Peter Amstutz 10 months ago

Brett Smith wrote in #note-6:

I recently ran a toy websocket client against pirca (running 2.6.3), just subscribed to everything and dumping the records to JSON. It seemed to work, so websockets aren't completely broken.

Tom and I chatted about this a couple weeks ago, I suspect there's a bug in Workbench 2 where it sometimes it sends an empty subscription and thus doesn't receive any events. Nobody has gone in to track it down yet.

Actions #8

Updated by Peter Amstutz 7 months ago

  • Subject changed from Fix websocket service or get rid of it to Fix websocket service
Actions #9

Updated by Peter Amstutz 7 months ago

Unfortunately, I don't yet know how to reproduce this.

Actions #10

Updated by Peter Amstutz 7 months ago

See graphs on #20930

Actions #11

Updated by Peter Amstutz 7 months ago

  • Target version changed from Future to Development 2023-09-13 sprint
Actions #13

Updated by Tom Clegg 7 months ago

  • Has duplicate Bug #20904: Investigate websockets issue seen by user added
Actions #14

Updated by Tom Clegg 7 months ago

Our listener pings create the conditions for a deadlock issue.

Specifically: our event loop can deadlock if enough (~32) server notifications arrive after the event loop decides to call Ping (e.g., while listener.Ping() is waiting for a response from the server, or in the time.Sleep() invoked by testSlowPing).

(*ListenerConn)listenerConnLoop() doesn't see the server's ping response until it finishes sending a previous notification through its internal queue to (*Listener)listenerConnLoop(), which is blocked on sending to our Notify channel, which is blocked on waiting for the Ping response.

(The lib/pq example uses "go listener.Ping()" so it doesn't deadlock.)

Actions #15

Updated by Tom Clegg 7 months ago

  • Assigned To set to Tom Clegg
  • Status changed from New to In Progress
Actions #16

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2023-09-13 sprint to Development 2023-09-27 sprint
Actions #17

Updated by Tom Clegg 7 months ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF