Project

General

Profile

Actions
Copy link

Feature #21751

closed

Installer's Terraform code supports Customer-Managed Keys

Added by Lucas Di Pentima 9 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Lucas Di Pentima
Category:
Deployment
Target version:
Development 2024-09-25 sprint
Story points:
-
Release:
Arvados 3.0
Release relationship:
Auto

Description

When enabling CMK on AWS, some special permissions need to be set in order for Arvados to work properly.

Compute nodes

Compute nodes need access to the keys so that the ebs-autoscale feature can create EBS encrypted volumes correctly. Preliminary tests suggest that the following policy is enough:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:GenerateDataKey*" 
            ],
            "Resource": [
                "arn:aws:kms:us-east-1:1234567890:key/xxxxxx-kms-id" 
            ]
        },
        {
            "Effect": "Allow",
            "Action": "kms:CreateGrant",
            "Resource": [
                "arn:aws:kms:us-east-1:1234567890:key/xxxxxx-kms-id" 
            ],
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": true
                }
            }
        }
    ]
}

Cloud dispatcher

The a-d-c service might need a similar policy so that it can launch compute nodes with their storage volumes encrypted by default.

This feature should be optional, and disabled by default. The CMK's ARN should be set in Terraform's tfvars file.

Subtasks 1 (0 open — 1 closed)

Task #21757: Review 21751-installer-cmk-supportResolvedLucas Di Pentima09/17/2024Actions
Actions
Copy link
 #2

Updated by Lucas Di Pentima 9 months ago

  • Status changed from New to In Progress
Actions
Copy link
 #3

Updated by Peter Amstutz 9 months ago

  • Target version changed from Development 2024-05-22 sprint to Development 2024-06-05 sprint
Actions
Copy link
 #4

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2024-06-05 sprint to Development 2024-06-19 sprint
Actions
Copy link
 #5

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2024-06-19 sprint to Development 2024-07-03 sprint
Actions
Copy link
 #6

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2024-07-03 sprint to Development 2024-07-24 sprint
Actions
Copy link
 #7

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-07-24 sprint to Development 2024-08-07 sprint
Actions
Copy link
 #8

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-08-07 sprint to Development 2024-08-28 sprint
Actions
Copy link
 #9

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-08-28 sprint to Development 2024-09-11 sprint
Actions
Copy link
 #10

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-09-11 sprint to Development 2024-09-25 sprint
Actions
Copy link
 #11

Updated by Lucas Di Pentima 5 months ago

This has been partially finished for some time. I left it aside because I wasn't able to reproduce the issue as I originally remembered it, but I have just been able to do so, so it's now ready to be reviewed.

Changes at c6428ce - branch 21751-installer-cmk-support

  • All agreed upon points are implemented / addressed. Describe changes from pre-implementation design.
    • Yes
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • No
  • Code is tested and passing, both automated and manual, what manual testing was done is described.
    • We currently don't have automated multi-node+terraform tests. I've done manual tests on our sandbox account:
      1. Deployed a test cluster and confirmed it's working correctly with a diagnostics run.
      2. Changed the account's EC2 security configuration setting the EBS encryption to use a CMK.
      3. Ran another diagnostics test and confirmed it's failing with the following instance error message: "Client.InvalidKMSKey.InvalidState: The KMS key provided is in an incorrect state"
      4. Set the cmk_arn variable in terraform/services/terraform.tfvars and re-ran ./installer.sh terraform.
      5. Ran diagnostics once again, and got a successful run.
  • New or changed UX/UX and has gotten feedback from stakeholders.
    • None
  • Documentation has been updated.
    • This minor features are just documented on the terraform.tfvars file.
  • Behaves appropriately at the intended scale (describe intended scale).
    • No change in scale.
  • Considered backwards and forwards compatibility issues between client and server.
    • Backwards compatibility is maintained as this feature is disabled by default.
  • Follows our coding standards and GUI style guidelines.
    • Yes

Changes:

Actions
Copy link
 #12

Updated by Lucas Di Pentima 5 months ago

  • Release set to 70
Actions
Copy link
 #13

Updated by Brett Smith 5 months ago

Lucas Di Pentima wrote in #note-11:

Changes at c6428ce - branch 21751-installer-cmk-support

Just one naming suggestion about:

name = "${local.cluster_name}_dispatchercmk_access_attachment"

For consistency with the rest of the names around here, I think it would be nicer if we added an underscore to make it dispatcher_cmk. It's also a little easier to read.

This is good to merge, thanks.

Actions
Copy link
 #14

Updated by Lucas Di Pentima 5 months ago

Brett Smith wrote in #note-13:

For consistency with the rest of the names around here, I think it would be nicer if we added an underscore to make it dispatcher_cmk. It's also a little easier to read.

This is good to merge, thanks.

Thank you for spotting the typo, it was my intention to name it as you suggested.

Actions
Copy link
 #15

Updated by Lucas Di Pentima 5 months ago

  • Status changed from In Progress to Resolved
Actions
Copy link

Also available in: Atom PDF