Arvados

Running that config file through arvados-server config-dump gives:

AutoReloadConfig: false
Clusters:
  harpo:
    API:
      AsyncPermissionsUpdateInterval: 0s
      DisabledAPIs: null
      KeepServiceRequestTimeout: 0s
      MaxConcurrentRequests: 0
      MaxIndexDatabaseRead: 0
      MaxItemsPerResponse: 0
      MaxKeepBlobBuffers: 0
      MaxRequestAmplification: 0
      MaxRequestSize: 0
      MaxTokenLifetime: 0s
      RequestTimeout: 0s
      SendTimeout: 0s
      VocabularyPath: "" 
      WebsocketClientEventQueue: 0
      WebsocketServerEventQueue: 0
...

This is very wrong, none of the defaults are there.

Indeed, after updating config.yml like this:

Clusters:
  harpo:

    SystemRootToken: "systemroottokenmushaveatleast32characters" 
    ManagementToken: "managementtokenmushaveatleast32characters" 

    API:
      DisabledAPIs: {}
...

and re-running arvados-server config-dump:

AutoReloadConfig: false
Clusters:
  harpo:
    API:
      AsyncPermissionsUpdateInterval: 20s
      DisabledAPIs: {}
      KeepServiceRequestTimeout: 15s
      MaxConcurrentRequests: 0
      MaxIndexDatabaseRead: 134217728
      MaxItemsPerResponse: 1000
      MaxKeepBlobBuffers: 128
      MaxRequestAmplification: 4
      MaxRequestSize: 134217728
      MaxTokenLifetime: 0s
      RequestTimeout: 5m
      SendTimeout: 1m
      VocabularyPath: "" 
      WebsocketClientEventQueue: 64
      WebsocketServerEventQueue: 4
...

which is as expected. The problem appears to be that the API: section has no values at all in config.yml. Setting any one value makes arvados-server config-dump do the right thing. For example, setting MaxItemsPerResponse:

AutoReloadConfig: False

Clusters:
  harpo:

    SystemRootToken: "systemroottokenmushaveatleast32characters" 
    ManagementToken: "managementtokenmushaveatleast32characters" 

    API:
      MaxItemsPerResponse: 50
...

results in

AutoReloadConfig: false
Clusters:
  harpo:
    API:
      AsyncPermissionsUpdateInterval: 20s
      DisabledAPIs: {}
      KeepServiceRequestTimeout: 15s
      MaxConcurrentRequests: 0
      MaxIndexDatabaseRead: 134217728
      MaxItemsPerResponse: 50
      MaxKeepBlobBuffers: 128
      MaxRequestAmplification: 4
      MaxRequestSize: 134217728
      MaxTokenLifetime: 0s
      RequestTimeout: 5m
      SendTimeout: 1m
      VocabularyPath: "" 
      WebsocketClientEventQueue: 64
      WebsocketServerEventQueue: 4

Okay, the problem is that the config:

AutoReloadConfig: False

Clusters:
  harpo:

    SystemRootToken: "systemroottokenmushaveatleast32characters" 
    ManagementToken: "managementtokenmushaveatleast32characters" 

    API:

    Collections:
      BlobSigningKey: "blobsigningkeymushaveatleast32characters" 
...

Is turned into

...
{
  "AutoReloadConfig": false,
  "Clusters": {
    "harpo": {
      "API": null,
      "Collections": {
        "BlobSigningKey": "blobsigningkeymushaveatleast32characters" 
      },
...

and then we call mergo.Merge(&merged, src, mergo.WithOverride) in lib/config/load.go which overwrites the default API structure with null. Doh.

Ready for review at e414e9d90df8756ab4fc9dab4d2a5729a51a525c on branch 18679-handle-keys-with-null-values-in-config

Tests passed at developer-run-tests: #2894 , minus wb integration tests, which passed on a re-run here: developer-run-tests-apps-workbench-integration: #3079

This looks like a templating problem to me. The template/operator didn't intend to put a null value in their config.yml, but they did.

FBOW these are documented ways to spell null in yaml:

key1:
key2: ~
key3: null

That said, we don't currently need null to mean anything other than "use the default" and we generally prefer to have a real value like "" or {} if the intent is to say "use the zero value instead of the default", so this seems like a way to accommodate templating artifacts that doesn't cost us anything.

LGTM, thanks

Tom Clegg wrote:

LGTM, thanks

Thanks, merged.