Feature #17608

[deployment][arvados-formula] run @arvados-server --config-check@ on all the roles but shell

Added by Javier Bértoli 14 days ago. Updated about 10 hours ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Deployment
Target version:
Start date:
05/06/2021
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

The arvados-server package should be installed in all the roles but shell, so the configuration file (/etc/arvados/config.yml) can be checked before the service is started.

#. Add arvados-server to all the roles
#. Add an only_if/unless control to the service before starting it


Subtasks

Task #17647: Review commit ee82273@arvados-formulaResolvedJavier Bértoli


Related issues

Blocks Arvados - Story #17512: Release Arvados 2.2New05/03/2021

History

#1 Updated by Javier Bértoli 13 days ago

  • Status changed from New to In Progress

#2 Updated by Javier Bértoli 13 days ago

We're already running a pre-deploy check of the configuration (commit 4bf9501a@arvados-formula).

#4 Updated by Peter Amstutz 8 days ago

#5 Updated by Javier Bértoli 8 days ago

  • % Done changed from 0 to 100

Changed from

/usr/bin/arvados-server config-dump -config

to

check_command: /usr/bin/arvados-server config-check -config

configurable through pillars (commit ee82273@arvados-formula)

Now, check is strict, so we make sure that configuration is arvados-compatible.

Test pass

#6 Updated by Javier Bértoli 7 days ago

  • Status changed from In Progress to Feedback

#7 Updated by Lucas Di Pentima 6 days ago

I've been reading the basics on SaltStack, so please if you detect some conceptual flaws on my questions, feel free to correct me :)

- General Q: The "Get the `tplroot` from `tpldir`" code block seem to be duped in every sls file. Could that be moved to the root init.sls file or some other shared place so it's declared once?
- General Q II: I had to clone the arvados-formula repo from github. Is it hosted also on our own repo? If not, should it be hosted there and mirrored to github?
- Typo in pillar.example comment: s/intalling/installing
- At pillar.example: shouldn't the config.check_command key have some default value? I ask because there're other keys with uncommented values, so someone using the example would need to set something on check_command for the lookup to work, right? AFAICT the arvados/config/file.sls formula doesn't have a fallback value for check_cmd if the pillar value isn't set, or would that produce a null value that will make the check_cmd primitive be disabled? If that's the case, should we try to avoid not checking the config? We even recommend checking the config in the pillar example, so maybe it's a good idea to make it a default?

#8 Updated by Javier Bértoli 6 days ago

Lucas Di Pentima wrote:

- General Q: The "get the `tplroot` from `tpldir`" code block seem to be duped in every sls file. Could that be moved to the root init.sls file or some other shared place so it's declared once?

- General Q II: I had to clone the arvados-formula repo from github. Is it hosted also on our own repo? If not, should it be hosted there and mirrored to github?

Yes, we have 3 places atm:

  • our own repo,
  • the github one and
  • the saltstack-formulas one. But this last one, as it tries to follow a set of different rules that we follow regarding versioning and such, I'm managing it through PRs to keep it updated. And currently, is outdated, waiting for this set of fixes to be updated.

We refer everyone to github/arvados-formula in our docs. Mirroring it from our own would be a great improvement.

- Typo in pillar.example comment: s/intalling/installing

Thanks, fixed in another branch.

- At pillar.example: shouldn't the config.check_command key have some default value? I ask because there're other keys with uncommented values, so someone using the example would need to set something on check_command for the lookup to work, right? AFAICT the arvados/config/file.sls formula doesn't have a fallback value for check_cmd if the pillar value isn't set, or would that produce a null value that will make the check_cmd primitive be disabled? If that's the case, should we try to avoid not checking the config? We even recommend checking the config in the pillar example, so maybe it's a good idea to make it a default?

Defaults in saltstack formulas are set in <formula>/*yaml files, which are merged (through map.jinja) in certain order and finally with the pillar data, to get the configuration that will be applied to the minion.

In this case, the check_command: /usr/bin/arvados-server config-check -config set in arvados/defaults.yaml will be used if no value is set by the user.

#9 Updated by Peter Amstutz about 14 hours ago

  • Target version changed from 2021-05-12 sprint to 2021-05-26 sprint

#10 Updated by Javier Bértoli about 10 hours ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF