Project

General

Profile

Actions

Bug #16345

closed

Health check checks for clock and version skew

Added by Peter Amstutz almost 4 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
-
Release relationship:
Auto

Description

Health check returns current clock time in HTTP timestamp (e.g. 'Date' header)

Health check publishes a metric which is difference between earliest and latest clock times using HTTP timestamp in the response header

Health check client also checks and reports version skew among services

Client detailed dump (-yaml) should print metrics detail like config hash, version number, config timestamp, current clock.


Subtasks 2 (0 open2 closed)

Task #19059: Review 16345-check-clock-skewResolvedWard Vandewege05/11/2022Actions
Task #19147: Review 16345-check-version-skewResolvedWard Vandewege05/20/2022Actions

Related issues

Related to Arvados Epics - Idea #18727: Avoid configuration skew between different services and hostsResolved03/01/202205/31/2022Actions
Actions #1

Updated by Peter Amstutz almost 4 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz almost 4 years ago

  • Status changed from In Progress to New
Actions #4

Updated by Peter Amstutz almost 4 years ago

  • Related to Idea #16444: Improved error detection/reporting added
Actions #6

Updated by Peter Amstutz about 2 years ago

  • Related to Idea #18727: Avoid configuration skew between different services and hosts added
Actions #7

Updated by Peter Amstutz about 2 years ago

  • Related to deleted (Idea #16444: Improved error detection/reporting)
Actions #8

Updated by Peter Amstutz about 2 years ago

  • Target version set to 2022-05-11 sprint
  • Description updated (diff)
Actions #9

Updated by Peter Amstutz almost 2 years ago

  • Assigned To set to Tom Clegg
Actions #10

Updated by Tom Clegg almost 2 years ago

  • Status changed from New to In Progress
Actions #12

Updated by Peter Amstutz almost 2 years ago

  • Description updated (diff)
  • Subject changed from Health check checks for clock skew to Health check checks for clock and version skew
Actions #13

Updated by Peter Amstutz almost 2 years ago

  • Description updated (diff)
Actions #14

Updated by Tom Clegg almost 2 years ago

  • Target version changed from 2022-05-11 sprint to 2022-05-25 sprint
Actions #15

Updated by Ward Vandewege almost 2 years ago

Tom Clegg wrote:

16345-check-clock-skew @ 10db2d51a25a80840f299f09baada5588af991f3 -- developer-run-tests: #3107

LGTM thanks!

Actions #16

Updated by Tom Clegg almost 2 years ago

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)
Actions #17

Updated by Ward Vandewege almost 2 years ago

Tom Clegg wrote:

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)

LGTM, thanks!

Actions #18

Updated by Tom Clegg almost 2 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados-private:commit:arvados|36cfafd6e7eae2784c22aefdd9df26783412d42a.

Actions #19

Updated by Peter Amstutz over 1 year ago

  • Release set to 47
Actions

Also available in: Atom PDF