Project

General

Profile

Actions

Bug #16345

closed

Health check checks for clock and version skew

Added by Peter Amstutz over 2 years ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
05/11/2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

Health check returns current clock time in HTTP timestamp (e.g. 'Date' header)

Health check publishes a metric which is difference between earliest and latest clock times using HTTP timestamp in the response header

Health check client also checks and reports version skew among services

Client detailed dump (-yaml) should print metrics detail like config hash, version number, config timestamp, current clock.


Subtasks 2 (0 open2 closed)

Task #19059: Review 16345-check-clock-skewResolvedWard Vandewege05/11/2022

Actions
Task #19147: Review 16345-check-version-skewResolvedWard Vandewege05/20/2022

Actions

Related issues

Related to Arvados Epics - Story #18727: Avoid configuration skew between different services and hostsResolved03/01/202205/31/2022

Actions
Actions #1

Updated by Peter Amstutz over 2 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz over 2 years ago

  • Status changed from In Progress to New
Actions #4

Updated by Peter Amstutz about 2 years ago

  • Related to Story #16444: Improved error detection/reporting added
Actions #6

Updated by Peter Amstutz 5 months ago

  • Related to Story #18727: Avoid configuration skew between different services and hosts added
Actions #7

Updated by Peter Amstutz 5 months ago

  • Related to deleted (Story #16444: Improved error detection/reporting)
Actions #8

Updated by Peter Amstutz 4 months ago

  • Target version set to 2022-05-11 sprint
  • Description updated (diff)
Actions #9

Updated by Peter Amstutz 4 months ago

  • Assigned To set to Tom Clegg
Actions #10

Updated by Tom Clegg 3 months ago

  • Status changed from New to In Progress
Actions #12

Updated by Peter Amstutz 3 months ago

  • Description updated (diff)
  • Subject changed from Health check checks for clock skew to Health check checks for clock and version skew
Actions #13

Updated by Peter Amstutz 3 months ago

  • Description updated (diff)
Actions #14

Updated by Tom Clegg 3 months ago

  • Target version changed from 2022-05-11 sprint to 2022-05-25 sprint
Actions #15

Updated by Ward Vandewege 3 months ago

Tom Clegg wrote:

16345-check-clock-skew @ 10db2d51a25a80840f299f09baada5588af991f3 -- developer-run-tests: #3107

LGTM thanks!

Actions #16

Updated by Tom Clegg 3 months ago

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)
Actions #17

Updated by Ward Vandewege 3 months ago

Tom Clegg wrote:

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)

LGTM, thanks!

Actions #18

Updated by Tom Clegg 3 months ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados-private:commit:arvados|36cfafd6e7eae2784c22aefdd9df26783412d42a.

Actions

Also available in: Atom PDF