Project

General

Profile

Actions
Copy link

Feature #21705

open

Update Go runtime and dependencies

Added by Tom Clegg 28 days ago. Updated 1 day ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Tom Clegg
Category:
-
Target version:
Development 2024-05-22 sprint
Story points:
-

Files

cloudtest.log.txt (6.08 KB) cloudtest.log.txt Tom Clegg, 05/15/2024 02:23 PM

Subtasks 1 (1 open — 0 closed)

Task #21729: ReviewNewBrett SmithActions
Actions
Copy link
 #1

Updated by Tom Clegg 28 days ago

  • Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Actions
Copy link
 #2

Updated by Tom Clegg 11 days ago

  • Status changed from New to In Progress
Actions
Copy link
 #3

Updated by Peter Amstutz 9 days ago

  • Target version changed from Development 2024-05-08 sprint to Development 2024-05-22 sprint
Actions
Copy link
 #4

Updated by Tom Clegg 3 days ago

21705-go-deps @ 03c62749d21315d1a2020ca663646cff185b63c3 -- developer-run-tests: #4217

  • Migrate keepstore S3 driver, dispatchcloud EC2 driver, and keep-web S3 test case to modern AWS SDK (aws-sdk-go-v2)
  • Update hashicorp/go-retryablehttp with our PR merged, remove our workaround

I have started another branch (21705-go122) with dependency updates that require go >= 1.21. Currently CI images have go 1.20, but if we rebuild after merging this branch (21705-go-deps) they will have 1.22.

Go 1.21 introduces toolchain versioning, so this sort of upgrade should get easier in future.

Actions
Copy link
 #5

Updated by Tom Clegg 2 days ago

arvados-server cloudtest result looks good: cloudtest.log.txt

The "cloud/driver Create succeeded, but instance is not in list" message is logged by the previous version too:

workbench.tordo:~# /tmp/arvados-server --version
/tmp/arvados-server 2.8.0~dev20240514203017 (go1.20.6)
workbench.tordo:~# /tmp/arvados-server cloudtest
...
ERRO[2024-05-15T14:24:31.031447498Z] cloud/driver Create succeeded, but instance is not in list  error="test instance missing from cloud provider's list" 
...

Evidently this is an "eventually consistent" situation that cloudtest doesn't expect. It might cause a bit of unnecessary churn in arvados-dispatch-cloud, but in the context of porting our EC2 driver to the new SDK it just indicates we have successfully preserved the existing driver's behavior.

Actions
Copy link
 #7

Updated by Tom Clegg 1 day ago

When we added "get credentials from instance metadata" support to the original s3 driver in #15599 we documented the following configuration scheme:
  1. If AccessKeyID and SecretAccessKey are both blank, s3 driver uses IAM role
  2. If IAMRole is also blank, s3 driver uses whatever IAM role is provided in instance metadata
  3. If IAMRole is not blank, s3 driver uses IAM role in instance metadata only if it matches
  4. If AccessKeyID / SecretAccessKey and IAMRole are both non-blank, error out at startup because it's unclear which credentials you want to use

However, #3 and #4 were never implemented in the "s3aws" driver, which has since become the only s3 driver. The IAMRole config is just completely ignored.

For #3: I think it would be painful to implement and maintain this with the new code base. The aws sdk does lots of things under the hood so it's hard to interfere/second-guess correctly.

For #4: If we don't have #3, then #4 reduces to "set this to true to assure us that your blank access/secret keys are not an error/oversight", which seems a bit silly. I am inclined to remove the IAMRole config, so the behaviors are
  • set AccessKeyID and SecretAccessKey → use explicit credentials
  • don't set them → use instance metadata

Again, this is what the code has been doing since April 2023, so it would just be a documentation update.

Actions
Copy link

Also available in: Atom PDF