Bug #17590
closed
[configuration] EC2 credentials variables are not consistently named in the /etc/arvados/config.yml file
Added by Javier Bértoli about 3 years ago.
Updated over 2 years ago.
Release relationship:
Auto
Description
When setting up a new cluster in AWS I noticed the credentials variables for the Containers:CloudVMs:DriverParameters and Volumes:<volume>:DriverParameters sections are named inconsistently. This leads to involuntary, hard to debug, errors when creating the new config:
Containers:
CloudVMs:
Driver: ec2
DriverParameters:
AccessKeyID: AKIA...5
SecretAccessKey: 5x...MV
Volumes:
<cluster>-nyw5e-0000000000000000:
DriverParameters:
AccessKey: AKIA...5
SecretKey: 5x...MV
The names of these variables should be consistent across the file.
- Target version changed from 2021-04-28 bughunt sprint to 2021-05-12 sprint
- Assigned To set to Tom Clegg
- Status changed from New to In Progress
Tom Clegg wrote:
17590-ec2-config-keys @ cef860fc944f66b4b52aa2ea6d66561944a7c065 -- developer-run-tests: #2459 
- I think specifying the new keys while keeping the old ones around should be an error.
- Can you add a note to the upgrade guide?
- The config-check command says "deprecated or unknown config entry" but it sure would be nice if that could also give the hing that they were simply renamed. Does it only do that when Keepstore starts perhaps? This is what I see on tordo:
tordo:~# ./arvados-server config-check
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.API.RailsSessionSecretToken"
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.SecretKey"
time="2021-05-07T13:52:12Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey"
Otherwise, LGTM, thanks!
Oops, it turns out the logs were unhelpful because the migration code wasn't running at all -- it was looking for driver=s3 (as spelled in the example config file) instead of driver=S3 (the spelling that actually works to select the S3 driver). With that fixed (in both migration and example config), config-check does this on tordo:
Your configuration is relying on deprecated entries. Suggest making the following changes.
--- without-deprecated-configs
+++ relying-on-deprecated-configs
@@ -495,10 +495,10 @@
ReadOnly: false
Driver: S3
DriverParameters:
- AccessKey: AAAAAAAAAAAAAAAAAAAA
+ AccessKeyID: AAAAAAAAAAAAAAAAAAAA
Bucket: tordo-nyw5e-000000000000000-volume
Region: us-east-1
- SecretKey: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+ SecretAccessKey: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ReadOnly: false
Replication: 2
StorageClasses: null
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.SecretKey"
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey"
time="2021-05-07T14:19:53Z" level=warning msg="deprecated or unknown config entry: Clusters.tordo.API.RailsSessionSecretToken"
time="2021-05-07T14:19:53Z" level=warning msg="using your old config keys tordo.Volumes.tordo-nyw5e-000000000000000.DriverParameters.AccessKey/SecretKey -- but you should rename them to AccessKeyID/SecretAccessKey"
Also made it an error to supply both old and new config keys, and added a note to the "upgrading" doc page.
17590-ec2-config-keys @ 659fa8bfd2f9820a9b7f955e9e9e63a1cc4275a1 -- developer-run-tests: #2463 
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by