Project

General

Profile

Actions

Feature #17608

closed

[deployment][arvados-formula] run @arvados-server --config-check@ on all the roles but shell

Added by Javier Bértoli almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Javier Bértoli
Category:
Deployment
Target version:
Story points:
-
Release relationship:
Auto

Description

The arvados-server package should be installed in all the roles but shell, so the configuration file (/etc/arvados/config.yml) can be checked before the service is started.

#. Add arvados-server to all the roles
#. Add an only_if/unless control to the service before starting it


Subtasks 1 (0 open1 closed)

Task #17647: Review commit ee82273@arvados-formulaResolvedJavier Bértoli05/06/2021Actions

Related issues

Blocks Arvados - Idea #17512: Release Arvados 2.2ResolvedPeter Amstutz05/03/2021Actions
Actions #1

Updated by Javier Bértoli almost 3 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Javier Bértoli almost 3 years ago

We're already running a pre-deploy check of the configuration (commit 4bf9501a@arvados-formula).

Actions #4

Updated by Peter Amstutz almost 3 years ago

Actions #5

Updated by Javier Bértoli almost 3 years ago

  • % Done changed from 0 to 100

Changed from

/usr/bin/arvados-server config-dump -config

to

check_command: /usr/bin/arvados-server config-check -config

configurable through pillars (commit ee82273@arvados-formula)

Now, check is strict, so we make sure that configuration is arvados-compatible.

Test pass

Actions #6

Updated by Javier Bértoli almost 3 years ago

  • Status changed from In Progress to Feedback
Actions #7

Updated by Lucas Di Pentima almost 3 years ago

I've been reading the basics on SaltStack, so please if you detect some conceptual flaws on my questions, feel free to correct me :)

- General Q: The "Get the `tplroot` from `tpldir`" code block seem to be duped in every sls file. Could that be moved to the root init.sls file or some other shared place so it's declared once?
- General Q II: I had to clone the arvados-formula repo from github. Is it hosted also on our own repo? If not, should it be hosted there and mirrored to github?
- Typo in pillar.example comment: s/intalling/installing
- At pillar.example: shouldn't the config.check_command key have some default value? I ask because there're other keys with uncommented values, so someone using the example would need to set something on check_command for the lookup to work, right? AFAICT the arvados/config/file.sls formula doesn't have a fallback value for check_cmd if the pillar value isn't set, or would that produce a null value that will make the check_cmd primitive be disabled? If that's the case, should we try to avoid not checking the config? We even recommend checking the config in the pillar example, so maybe it's a good idea to make it a default?

Actions #8

Updated by Javier Bértoli almost 3 years ago

Lucas Di Pentima wrote:

- General Q: The "get the `tplroot` from `tpldir`" code block seem to be duped in every sls file. Could that be moved to the root init.sls file or some other shared place so it's declared once?

- General Q II: I had to clone the arvados-formula repo from github. Is it hosted also on our own repo? If not, should it be hosted there and mirrored to github?

Yes, we have 3 places atm:

  • our own repo,
  • the github one and
  • the saltstack-formulas one. But this last one, as it tries to follow a set of different rules that we follow regarding versioning and such, I'm managing it through PRs to keep it updated. And currently, is outdated, waiting for this set of fixes to be updated.

We refer everyone to github/arvados-formula in our docs. Mirroring it from our own would be a great improvement.

- Typo in pillar.example comment: s/intalling/installing

Thanks, fixed in another branch.

- At pillar.example: shouldn't the config.check_command key have some default value? I ask because there're other keys with uncommented values, so someone using the example would need to set something on check_command for the lookup to work, right? AFAICT the arvados/config/file.sls formula doesn't have a fallback value for check_cmd if the pillar value isn't set, or would that produce a null value that will make the check_cmd primitive be disabled? If that's the case, should we try to avoid not checking the config? We even recommend checking the config in the pillar example, so maybe it's a good idea to make it a default?

Defaults in saltstack formulas are set in <formula>/*yaml files, which are merged (through map.jinja) in certain order and finally with the pillar data, to get the configuration that will be applied to the minion.

In this case, the check_command: /usr/bin/arvados-server config-check -config set in arvados/defaults.yaml will be used if no value is set by the user.

Actions #9

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-05-12 sprint to 2021-05-26 sprint
Actions #10

Updated by Javier Bértoli almost 3 years ago

  • Status changed from Feedback to Resolved
Actions #11

Updated by Peter Amstutz almost 3 years ago

  • Release set to 38
Actions

Also available in: Atom PDF