Project

General

Custom queries

Profile

Actions

Bug #16345

closed

Health check checks for clock and version skew

Added by Peter Amstutz almost 5 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
-
Release relationship:
Auto

Description

Health check returns current clock time in HTTP timestamp (e.g. 'Date' header)

Health check publishes a metric which is difference between earliest and latest clock times using HTTP timestamp in the response header

Health check client also checks and reports version skew among services

Client detailed dump (-yaml) should print metrics detail like config hash, version number, config timestamp, current clock.


Subtasks 2 (0 open2 closed)

Task #19059: Review 16345-check-clock-skewResolvedWard Vandewege05/11/2022Actions
Task #19147: Review 16345-check-version-skewResolvedWard Vandewege05/20/2022Actions

Related issues 1 (0 open1 closed)

Related to Arvados Epics - Idea #18727: Avoid configuration skew between different services and hostsResolved03/01/202205/31/2022Actions
Actions #15

Updated by Ward Vandewege almost 3 years ago

Tom Clegg wrote:

16345-check-clock-skew @ 10db2d51a25a80840f299f09baada5588af991f3 -- developer-run-tests: #3107

LGTM thanks!

Actions #16

Updated by Tom Clegg almost 3 years ago

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)
Actions #17

Updated by Ward Vandewege almost 3 years ago

Tom Clegg wrote:

16345-check-version-skew @ 7f9de270aa34467c1b1668be9333ec28d14b10a1 -- developer-run-tests: #3152
retry wb1 developer-run-tests-apps-workbench-integration: #3377

  • fail health check and provide details if any services don't match the version of the arvados-server check program itself
  • add Version, ConfigSourceSHA256, ConfigSourceTimestamp fields to -yaml output
  • tweak existing fields: don't lowercase keys, delete superfluous HTTPStatus field (HTTPStatusCode is enough)

LGTM, thanks!

Actions #18

Updated by Tom Clegg almost 3 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados-private:commit:arvados|36cfafd6e7eae2784c22aefdd9df26783412d42a.

Actions

Also available in: Atom PDF