Story #15003

[config] Go package & tool for preprocessing cluster config file

Added by Tom Clegg 2 months ago. Updated 5 days ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
04/26/2019
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
3.0

Description

This tool preprocesses cluster configs so individual components only need to understand a single config object, despite complications like supporting old config formats.

Initial functionality:
  • Read the cluster config file from disk
  • Apply defaults (perhaps by layering on-disk config after loading yaml file embedded in Go binary)
  • Migrate values from old configs to their new positions (e.g., NodeProfiles → Services)
  • Error out if the old configs cannot be migrated/ignored safely
Possible future functionality (not implemented here):
  • Load secrets from disk ("signing key is file:///etc/key")
  • Load pre-cluster-config-file configs like keepstore.yml
  • Layered configs
  • Includes

Subtasks

Task #15096: Review 15003-preprocess-configResolvedTom Clegg


Related issues

Related to Arvados - Bug #10282: [Go programs]Warn or error when unrecognized keys appear in config filesClosed

Blocks Arvados - Story #13648: [Epic] Use one cluster configuration file for all componentsNew

Associated revisions

Revision d235817f
Added by Tom Clegg 6 days ago

Merge branch '15003-preprocess-config'

refs #15003

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>

History

#1 Updated by Tom Clegg 2 months ago

  • Target version set to To Be Groomed

#2 Updated by Tom Clegg 2 months ago

  • Description updated (diff)
  • Story points set to 3.0

#3 Updated by Tom Clegg 2 months ago

  • Blocks Story #13648: [Epic] Use one cluster configuration file for all components added

#4 Updated by Tom Morris about 2 months ago

  • Target version changed from To Be Groomed to Arvados Future Sprints

#5 Updated by Tom Morris about 1 month ago

  • Assigned To set to Tom Clegg
  • Target version changed from Arvados Future Sprints to 2019-04-24 Sprint

#6 Updated by Tom Clegg about 1 month ago

  • Status changed from New to In Progress

#7 Updated by Tom Clegg 27 days ago

15003-preprocess-config @ cb13593decb097f501c0a1d64510a653b3233395 https://ci.curoverse.com/view/Developer/job/developer-run-tests/1209/
  • embeds the default config data in the arvados-server binary
  • adds "arvados-server config-dump" and "arvados-server config-check" commands

#8 Updated by Tom Clegg 27 days ago

  • Target version changed from 2019-04-24 Sprint to 2019-05-08 Sprint

#10 Updated by Lucas Di Pentima 18 days ago

This looks good AFAICT, the only thing I think it would be needed is documentation, both on the Admin site & the sub-commands' help text.

#11 Updated by Tom Clegg 13 days ago

  • Target version changed from 2019-05-08 Sprint to 2019-05-22 Sprint

#12 Updated by Tom Clegg 13 days ago

  • Related to Bug #10282: [Go programs]Warn or error when unrecognized keys appear in config files added

Also available in: Atom PDF