Project

General

Profile

Actions
Copy link

Bug #16345

closed

Health check checks for clock and version skew

Added by Peter Amstutz almost 3 years ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Tom Clegg
Category:
-
Target version:
2022-05-25 sprint
Start date:
05/11/2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release:
Arvados 2.5.0
Release relationship:
Auto

Description

Health check returns current clock time in HTTP timestamp (e.g. 'Date' header)

Health check publishes a metric which is difference between earliest and latest clock times using HTTP timestamp in the response header

Health check client also checks and reports version skew among services

Client detailed dump (-yaml) should print metrics detail like config hash, version number, config timestamp, current clock.

Subtasks 2 (0 open — 2 closed)

Task #19059: Review 16345-check-clock-skewResolvedWard Vandewege05/11/2022

Actions
Task #19147: Review 16345-check-version-skewResolvedWard Vandewege05/20/2022

Actions

Related issues

Related to Arvados Epics - Story #18727: Avoid configuration skew between different services and hostsResolved03/01/202205/31/2022

Actions
Actions
Copy link
 #1

Updated by Peter Amstutz almost 3 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #2

Updated by Peter Amstutz almost 3 years ago

  • Status changed from In Progress to New
Actions
Copy link
 #4

Updated by Peter Amstutz over 2 years ago

  • Related to Story #16444: Improved error detection/reporting added
Actions
Copy link
 #6

Updated by Peter Amstutz 11 months ago

  • Related to Story #18727: Avoid configuration skew between different services and hosts added
Actions
Copy link
 #7

Updated by Peter Amstutz 11 months ago

  • Related to deleted (Story #16444: Improved error detection/reporting)
Actions
Copy link
 #8

Updated by Peter Amstutz 10 months ago

  • Target version set to 2022-05-11 sprint
  • Description updated (diff)
Actions
Copy link
 #9

Updated by Peter Amstutz 10 months ago

  • Assigned To set to Tom Clegg
Actions
Copy link
 #10

Updated by Tom Clegg 9 months ago

  • Status changed from New to In Progress
Actions
Copy link
 #12

Updated by Peter Amstutz 9 months ago

  • Description updated (diff)
  • Subject changed from Health check checks for clock skew to Health check checks for clock and version skew
Actions
Copy link
 #13

Updated by Peter Amstutz 9 months ago

  • Description updated (diff)
Actions
Copy link
 #14

Updated by Tom Clegg 9 months ago

  • Target version changed from 2022-05-11 sprint to 2022-05-25 sprint
Actions
Copy link
 #15

Updated by Ward Vandewege 9 months ago

Tom Clegg wrote:

16345-check-clock-skew @ 10db2d51a25a80840f299f09baada5588af991f3 -- developer-run-tests: #3107

LGTM thanks!

Actions
Copy link
 #16

Updated by Tom Clegg 9 months ago

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)
Actions
Copy link
 #17

Updated by Ward Vandewege 9 months ago

Tom Clegg wrote:

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)

LGTM, thanks!

Actions
Copy link
 #18

Updated by Tom Clegg 9 months ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados-private:commit:arvados|36cfafd6e7eae2784c22aefdd9df26783412d42a.

Actions
Copy link
 #19

Updated by Peter Amstutz about 2 months ago

  • Release set to 47
Actions
Copy link

Also available in: Atom PDF