Project

General

Profile

Actions
Copy link

Bug #17590

closed

[configuration] EC2 credentials variables are not consistently named in the /etc/arvados/config.yml file

Added by Javier Bértoli about 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Tom Clegg
Category:
-
Target version:
2021-05-12 sprint
Story points:
-
Release:
Arvados 2.2.2
Release relationship:
Auto

Description

When setting up a new cluster in AWS I noticed the credentials variables for the Containers:CloudVMs:DriverParameters and Volumes:<volume>:DriverParameters sections are named inconsistently. This leads to involuntary, hard to debug, errors when creating the new config:

    Containers:
      CloudVMs:
        Driver: ec2
        DriverParameters:
          AccessKeyID: AKIA...5
          SecretAccessKey: 5x...MV

    Volumes:
      <cluster>-nyw5e-0000000000000000:
        DriverParameters:
          AccessKey: AKIA...5
          SecretKey: 5x...MV

The names of these variables should be consistent across the file.

Subtasks 1 (0 open — 1 closed)

Task #17639: Review 17590-ec2-config-keysResolvedTom Clegg05/06/2021Actions
Actions
Copy link
 #1

Updated by Peter Amstutz about 3 years ago

  • Target version changed from 2021-04-28 bughunt sprint to 2021-05-12 sprint
Actions
Copy link
 #2

Updated by Tom Clegg about 3 years ago

  • Assigned To set to Tom Clegg
Actions
Copy link
 #4

Updated by Tom Clegg about 3 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #6

Updated by Ward Vandewege about 3 years ago

Tom Clegg wrote:

17590-ec2-config-keys @ cef860fc944f66b4b52aa2ea6d66561944a7c065 -- developer-run-tests: #2459

  • I think specifying the new keys while keeping the old ones around should be an error.
  • Can you add a note to the upgrade guide?
  • The config-check command says "deprecated or unknown config entry" but it sure would be nice if that could also give the hing that they were simply renamed. Does it only do that when Keepstore starts perhaps? This is what I see on tordo:
tordo:~# ./arvados-server config-check
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.API.RailsSessionSecretToken" 
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.SecretKey" 
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey"

Otherwise, LGTM, thanks!

Actions
Copy link
 #7

Updated by Tom Clegg about 3 years ago

Oops, it turns out the logs were unhelpful because the migration code wasn't running at all -- it was looking for driver=s3 (as spelled in the example config file) instead of driver=S3 (the spelling that actually works to select the S3 driver). With that fixed (in both migration and example config), config-check does this on tordo:

Your configuration is relying on deprecated entries. Suggest making the following changes.
--- without-deprecated-configs
+++ relying-on-deprecated-configs
@@ -495,10 +495,10 @@
             ReadOnly: false
         Driver: S3
         DriverParameters:
-          AccessKey: AAAAAAAAAAAAAAAAAAAA
+          AccessKeyID: AAAAAAAAAAAAAAAAAAAA
           Bucket: tordo-nyw5e-000000000000000-volume
           Region: us-east-1
-          SecretKey: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+          SecretAccessKey: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
         ReadOnly: false
         Replication: 2
         StorageClasses: null
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.SecretKey" 
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey" 
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.API.RailsSessionSecretToken" 
time="2021-05-07T14:19:53Z" level=warning msg="using your old config keys tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey/SecretKey -- but you should rename them to AccessKeyID/SecretAccessKey"

Also made it an error to supply both old and new config keys, and added a note to the "upgrading" doc page.

17590-ec2-config-keys @ 659fa8bfd2f9820a9b7f955e9e9e63a1cc4275a1 -- developer-run-tests: #2463

Actions
Copy link
 #9

Updated by Tom Clegg about 3 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved
Actions
Copy link
 #10

Updated by Peter Amstutz over 2 years ago

  • Release set to 41
Actions
Copy link

Also available in: Atom PDF