Feature #17011

Add keep-web wildcard DNS to salt

Added by Peter Amstutz about 1 year ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Ward Vandewege
Category:
Deployment
Target version:
2020-12-16 Sprint
Start date:
11/25/2020
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0
Release:
Arvados 2.2.0
Release relationship:
Auto

Description

Keep-web supports virtual hosts to securely serve inline content to the browser (otherwise, everything is forced to be download-only to maintain same-origin security). This is also necessary for our S3 API support, as the preferred way to refer to buckets is with the bucket name as the first part of the domain name.

For each cluster zzzzz that we control:

  1. Configure DNS for *.collections.zzzzz.arvadosapi.com to go to keep-web
  2. Get a wildcard DNS cert for *.collections.zzzzz.arvadosapi.com
  3. Set Services.WebDAV.ExternalURL to "https://*.collections.zzzzz.arvadosapi.com"

More information at https://doc.arvados.org/v2.1/api/keep-web-urls.html

Subtasks

Task #17030: ReviewResolvedJavier Bértoli

Related issues

Blocks Arvados - Feature #17009: [keep-web] S3 API should accept bucket name as first component of domain nameResolved11/19/2020

Blocks Arvados - Story #17109: Support keep-web URLs with collection the domain nameResolved01/13/2021

Associated revisions

Revision e94ddba9 (diff)
Added by Ward Vandewege about 1 year ago

When calculating the version of our Go and Rails deb/rpm packages, take
the 'build' directory into account.

refs #17011

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <>

History

#1 Updated by Peter Amstutz about 1 year ago

  • Status changed from New to In Progress

#2 Updated by Peter Amstutz about 1 year ago

  • Status changed from In Progress to New
  • Tracker changed from Bug to Feature

#3 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#4 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#5 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#6 Updated by Peter Amstutz about 1 year ago

  • Category set to Deployment

#7 Updated by Peter Amstutz about 1 year ago

  • Assigned To set to Javier Bértoli

#8 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#9 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#10 Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2020-11-04 Sprint to 2020-11-18

#11 Updated by Peter Amstutz 12 months ago

  • Description updated (diff)

#12 Updated by Peter Amstutz 12 months ago

  • Blocks Feature #17009: [keep-web] S3 API should accept bucket name as first component of domain name added

#13 Updated by Peter Amstutz 12 months ago

  • Blocks Story #17109: Support keep-web URLs with collection the domain name added

#14 Updated by Ward Vandewege 11 months ago

  • Assigned To changed from Javier Bértoli to Ward Vandewege

#15 Updated by Peter Amstutz 11 months ago

  • Target version changed from 2020-11-18 to 2020-12-02 Sprint

#16 Updated by Ward Vandewege 11 months ago

  • Status changed from New to In Progress

I've gone through all the necessary steps on ce8i5 (see e.g. https://c28440adeb7421c5edc54d8fa31f16ed-114.collections.ce8i5.arvadosapi.com/_/cwl.output.json?disposition=inline), this configuration is now active there. Working on making the salt changes now.

#17 Updated by Ward Vandewege 11 months ago

Ready for review at commit:be3507374c33090fb6023fb2c289df0a314c54de on branch 17011-add-letsencrypt-wildcard-support.

The changes have already been applied for ce8i5. The only thing that is not automated is the creation of the IAM role + policy for the account that does the DNS validation.

#18 Updated by Javier Bértoli 11 months ago

  • Story points set to 8.0

@cure, it LGTM, I think it's ready to merge.

#19 Updated by Ward Vandewege 11 months ago

  • Story points changed from 8.0 to 1.0

merged thanks.

#20 Updated by Ward Vandewege 11 months ago

The terraform piece is now ready for review at commit:38c129609533e85b289c04301a34dfdcf20ac86f on branch 17011-terraform-changes (terraform repo). Applied for ce8i5 only in this commit.

Once this is merged, I'll go around all our clusters and
  • migrate their dns to route53 (if they haven't been yet)
  • switch them to wildcard dns/ssl for keep-web

#21 Updated by Javier Bértoli 11 months ago

@cure, it LGTM.

migrate their dns to route53 (if they haven't been yet)

This was partially done on #16240, it might need an update (to see if any value changed) and finish the migration from corehost.

#22 Updated by Ward Vandewege 11 months ago

Javier Bértoli wrote:

@cure, it LGTM.

migrate their dns to route53 (if they haven't been yet)

This was partially done on #16240, it might need an update (to see if any value changed) and finish the migration from corehost.

Thanks, merged. Indeed, I picked up from there for ce8i5. Will do the rest.

#23 Updated by Ward Vandewege 11 months ago

Converted to Route53:

  • ce8i5
  • 9tee4
  • su92l
  • tb05z
  • bd44f

Already on Route53:

  • jutro
  • lugli
  • pirca
  • tordo

Cleaned up so that terraform applies:

  • jutro
  • lugli
  • pirca
  • lugli

Created IAM role + policy:

  • ce8i5
  • 9tee4
  • su92l

Refactored IAM role + policy:

  • jutro
  • lugli
  • pirca
  • lugli

Enabled *.collections:

  • ce8i5
  • jutro
  • lugli
  • pirca
  • tordo
  • su92l
  • 9tee4

#24 Updated by Peter Amstutz 11 months ago

  • Target version changed from 2020-12-02 Sprint to 2020-12-16 Sprint

#26 Updated by Ward Vandewege 11 months ago

  • Status changed from In Progress to Resolved

#27 Updated by Peter Amstutz 5 months ago

  • Release set to 38

Also available in: Atom PDF