Feature #13611

Nodemanager reads instance sizes from the system config file to avoid duplication

Added by Lucas Di Pentima 6 months ago. Updated 2 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Node Manager
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
2.0

Description

Both crunch-dispatch-slurm and nodemanager need to a list of configured node sizes. Currently, this information is duplicated in both the nodemanager config file as the master arvados config.yml. To eliminate duplication nodemanager should read the node size configurations, and possibly the rest of its configuration, from the master arvados config.yml.


Related issues

Related to Arvados - Story #13569: Add units tera / giga / mega / kilo for configurationsResolved2018-06-29

Related to Arvados - Story #13648: [Epic] Use one cluster configuration file for all componentsNew

Blocked by Arvados - Bug #13674: [crunch-dispatch-slurm] Use hash instead of array for InstanceTypes configResolved

Blocked by Arvados - Story #13731: Design new nodemanager config data organization for cluster-wide configNew2018-08-13

History

#1 Updated by Peter Amstutz 6 months ago

  • Description updated (diff)

#2 Updated by Lucas Di Pentima 6 months ago

  • As described on https://dev.arvados.org/projects/arvados/wiki/Cluster_configuration, ProviderType attribute would be the same as the newly implemented instance_type on nodemanager. I think neither GoSDK nor dispatchcloud need to know about it, at least for now as the only user would be nodemanager.
  • Reading nodemanager’s cloud sizes from config.yml seems to be pretty straightforward.
  • Should we add a section about nodemanager’s specific configs on this story or leave that for another?
    • Cloud settings & credentials.
    • Daemon values (max nodes, poll time, dispatcher, etc).
    • Logging levels.
  • If going the hybrid path of having nodemanager’s config temporarily spread in 2 configuration files, should we ignore arvados-node-manager.ini’s cloud sizes?
  • Update nodemanager docs, possibly adding a migration guide.

#3 Updated by Lucas Di Pentima 6 months ago

  • Related to Story #13569: Add units tera / giga / mega / kilo for configurations added

#4 Updated by Tom Morris 6 months ago

  • Story points set to 2.0

#5 Updated by Tom Clegg 6 months ago

  • Related to Story #13648: [Epic] Use one cluster configuration file for all components added

#6 Updated by Tom Clegg 6 months ago

  • Related to Bug #13674: [crunch-dispatch-slurm] Use hash instead of array for InstanceTypes config added

#7 Updated by Tom Clegg 6 months ago

  • Related to deleted (Bug #13674: [crunch-dispatch-slurm] Use hash instead of array for InstanceTypes config)

#8 Updated by Tom Clegg 6 months ago

  • Blocked by Bug #13674: [crunch-dispatch-slurm] Use hash instead of array for InstanceTypes config added

#9 Updated by Tom Morris 5 months ago

  • Target version changed from To Be Groomed to Arvados Future Sprints

#10 Updated by Tom Clegg 5 months ago

  • Blocked by Story #13731: Design new nodemanager config data organization for cluster-wide config added

Also available in: Atom PDF