Story #13731

Design new nodemanager config data organization for cluster-wide config

Added by Tom Morris 5 months ago. Updated 2 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
08/13/2018
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

Add to Cluster configuration

Here's an example of the Puppet Hiera config, this is a yaml file used to populate the current .ini file, so one obvious approach would be to use the heira configuration format in "arvados.yml" instead:

arvados-node-manager:
  Logging:
    level: DEBUG
    apiclient: WARNING
    googleapiclient: WARNING
  Daemon:
    dispatcher: slurm
    max_nodes: 8
    poll_time: 10
    boot_fail_after: 1200
  Arvados:
    token: (redacted)
    timeout: 20
  Cloud:
    provider: ec2
    shutdown_windows: 21, 999999
  Cloud Credentials:
    key: (redacted)
    secret: "(redacted)" 
    region: us-east-1
    timeout: 20
  Cloud Create:
    ping_host: 4xphq.arvadosapi.com
    ex_keyname: ward
    image_id: ami-0a01b48b88d14541e
    subnet_id: subnet-24f5ad62
    security_groups: sg-4fc53e2a
  Cloud List:
    instance-state-name: running
    tag_arvados-class: dynamic-compute
    tag_cluster: 4xphq
  Size m4.medium:
    cores: 1
    scratch: 4000
  Size m4.large:
    cores: 2
    scratch: 32000
  Size m4.xlarge:
    cores: 4
    scratch: 80000

Subtasks

Task #13732: ReviewIn ProgressNico César


Related issues

Blocks Arvados - Feature #13611: Nodemanager reads instance sizes from the system config file to avoid duplicationNew

History

#1 Updated by Tom Morris 5 months ago

  • Status changed from New to In Progress

#2 Updated by Tom Morris 5 months ago

  • Status changed from In Progress to New
  • Story points set to 0.5

#3 Updated by Tom Clegg 5 months ago

  • Description updated (diff)

#4 Updated by Tom Clegg 5 months ago

  • Assigned To set to Tom Clegg

#5 Updated by Tom Clegg 5 months ago

  • Blocks Feature #13611: Nodemanager reads instance sizes from the system config file to avoid duplication added

#6 Updated by Peter Amstutz 5 months ago

  • Description updated (diff)

#7 Updated by Tom Clegg 5 months ago

  • Target version changed from 2018-07-18 Sprint to 2018-08-01 Sprint

#8 Updated by Lucas Di Pentima 5 months ago

  • Assigned To changed from Tom Clegg to Lucas Di Pentima

#9 Updated by Lucas Di Pentima 5 months ago

  • Status changed from New to In Progress

#10 Updated by Lucas Di Pentima 5 months ago

  • Target version changed from 2018-08-01 Sprint to 2018-08-15 Sprint

#11 Updated by Lucas Di Pentima 4 months ago

Added nodemanager's config items to the arvados-config.yml example.
You can check the differences here: https://dev.arvados.org/projects/arvados/wiki/Cluster_configuration/diff?utf8=✓&version=13&version_from=12&commit=View+differences (although it doesn't show indentation correctly)

As a summary:

  • Added [Logging] section as a top-level LoggingLevel key so that we can use default logging levels, maybe with the possibility of adding per-service values too
  • [Cloud Credentials] is replaced by the Providers key, I expanded it a little so that it's more clean how we can manage regions/services/credentials
  • Node sizes were already existing as top-level items
  • Moved all cloud node handling specifics inside a Cloud key on NodeManager's own key

#12 Updated by Lucas Di Pentima 4 months ago

  • Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint

#13 Updated by Tom Morris 3 months ago

  • Status changed from In Progress to New
  • Assigned To deleted (Lucas Di Pentima)
  • Target version changed from 2018-09-05 Sprint to Arvados Future Sprints

Moving this to the backlog since we're redesigning this corner of the world.

Also available in: Atom PDF