Feature #17215

Support AWS roles in arvados-dispatch-cloud

Added by Peter Amstutz 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
01/20/2021
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Subtasks

Task #17235: reviewResolvedWard Vandewege


Related issues

Related to Arvados Epics - Story #16520: GxP QualificationIn Progress08/01/202004/30/2021

Associated revisions

Revision fbc95892
Added by Ward Vandewege 3 months ago

Merge branch '17215-aws-roles-a-d-c'

closes #17215

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <>

History

#1 Updated by Peter Amstutz 4 months ago

  • Assigned To set to Javier Bértoli

#2 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2021-01-06 Sprint to 2021-01-20 Sprint

#3 Updated by Peter Amstutz 3 months ago

  • Assigned To changed from Javier Bértoli to Tom Clegg
  • Category set to Crunch
  • Subject changed from Investigate & document support for using AWS roles for keepstore and arvados-dispatch-cloud to Support AWS roles in arvados-dispatch-cloud
  • Tracker changed from Support to Feature

#4 Updated by Peter Amstutz 3 months ago

#5 Updated by Peter Amstutz 3 months ago

  • Assigned To changed from Tom Clegg to Ward Vandewege

#6 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2021-01-20 Sprint to 2021-02-03 Sprint

#7 Updated by Ward Vandewege 3 months ago

  • Status changed from New to In Progress

Ready for review at 4c30d75e647f42318fd0069613b3ed4f82c70ea0 on branch 17215-aws-roles-a-d-c

Tested with an IAM role on tordo, it works.

#8 Updated by Tom Clegg 3 months ago

There seem to be a couple of not-quite-obvious behaviors:
  1. If the configured credentials are invalid (non-empty but unusable), it looks like the role will be used instead, which could produce surprising results if the operator isn't expecting it
  2. If the configured credentials are valid, and the magic AWS env vars are invalid (non-empty but unusable), setup will fail -- even though the magic env vars wouldn't end up being used if they were valid (at least this is my impression after skimming the source code for session.NewSession())
  3. It's not obvious to me what the error message will be for the "invalid credentials + no IAM role" case

None of this seems wrong per se, but might be worth mentioning explicitly in config.default.yml that we will try the explicit credentials first, then fall back to IAM role.

Other than that minor thing, LGTM!

#9 Updated by Ward Vandewege 3 months ago

Tom Clegg wrote:

There seem to be a couple of not-quite-obvious behaviors:
  1. If the configured credentials are invalid (non-empty but unusable), it looks like the role will be used instead, which could produce surprising results if the operator isn't expecting it

Actually, no, if invalid credentials are specified in the config file, a-d-c prints the following type of error in the logs and does not try to use the next authentication method:

Jan 21 19:14:27 tordo.arvadosapi.com arvados-dispatch-cloud1424: {"PID":1424,"error":"AuthFailure: AWS was not able to validate the provided access credentials\n\tstatus code: 401, request id: d32a9139-4827-480f-947f-151e7c449557","level":"warning","msg":"sync failed","time":"2021-01-21T19:14:27.969809836Z"}

If the credentials are empty (i.e. defined in config file but zero-length value), the role authentication method will be used.

  1. If the configured credentials are valid, and the magic AWS env vars are invalid (non-empty but unusable), setup will fail -- even though the magic env vars wouldn't end up being used if they were valid (at least this is my impression after skimming the source code for session.NewSession())

AWS-specific environment values are a different method to supply authentication information, and we don't support that method. So we probably don't need to worry about this scenario.

  1. It's not obvious to me what the error message will be for the "invalid credentials + no IAM role" case

See above, the IAM role does not come into play.

None of this seems wrong per se, but might be worth mentioning explicitly in config.default.yml that we will try the explicit credentials first, then fall back to IAM role.

Other than that minor thing, LGTM!

#10 Updated by Tom Clegg 3 months ago

Aha. The "chain" thing looked like it might try each method until one worked. But evidently it does the more predictable thing. And, agreed, the env vars seem like they won't interfere with anything in practice. So... LGTM

#11 Updated by Ward Vandewege 3 months ago

Tom Clegg wrote:

Aha. The "chain" thing looked like it might try each method until one worked. But evidently it does the more predictable thing. And, agreed, the env vars seem like they won't interfere with anything in practice. So... LGTM

Thanks will merge!

#12 Updated by Ward Vandewege 3 months ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

#13 Updated by Ward Vandewege 3 months ago

  • Release set to 37

Also available in: Atom PDF