Feature #10684

[Crunch2] crunch-run adding certificates to container

Added by Peter Amstutz almost 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
12/15/2016
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
1.0
Release:
Release relationship:
Auto

Description

Crunch-job includes the following setup logic to make up-to-date SSL certificates available inside the container:

        .q{&& if test -f /etc/ssl/certs/ca-certificates.crt ; then VOLUMES+=("--volume=/etc/ssl/certs/ca-certificates.crt:/etc/arvados/ca-certificates.crt") ; }
        .q{elif test -f /etc/pki/tls/certs/ca-bundle.crt ; then VOLUMES+=("--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/arvados/ca-certificates.crt") ; fi };

We need an equivalent feature in crunch-run. Possibly this is as simple as a -v option that passes through to "runner.Binds".


Subtasks

Task #10744: Review 10684-crunch-run-ca-certsResolvedLucas Di Pentima

Associated revisions

Revision b0f9d8dc
Added by Tom Clegg almost 5 years ago

Merge branch '10684-ssl-ro' refs #10684

Revision ffd47382
Added by Peter Amstutz almost 5 years ago

Merge branch '10684-crunch-run-ca-certs' closes #10684

History

#1 Updated by Peter Amstutz almost 5 years ago

  • Story points set to 1.0

#2 Updated by Peter Amstutz almost 5 years ago

  • Description updated (diff)

#3 Updated by Peter Amstutz almost 5 years ago

  • Description updated (diff)

#4 Updated by Peter Amstutz almost 5 years ago

Alternately, crunch-run could DTRT and propagate the host system certs file into the container by default (which is what crunch-job is doing in the snippet above). At least when API: true

#5 Updated by Peter Amstutz almost 5 years ago

Proposal:

-ca-certs <file>

Will be mounted at /etc/arvados/ca-certificates.crt

If not provided, try

/etc/ssl/certs/ca-certificates.crt

and

/etc/pki/tls/certs/ca-bundle.crt

and mount them at /etc/arvados/ca-certificates.crt

Update Python and Go SDKs to use /etc/arvados/ca-certificates.crt

#6 Updated by Tom Clegg almost 5 years ago

Proposal:
  • In crunch-run, if the container does not already mount anything there, add a read-only bind mount at /etc/arvados/ca-certificates.crt using the first one of these that exists on the worker host:
    • /etc/arvados/ca-certificates.crt
    • /etc/ssl/certs/ca-certificates.crt
    • /etc/pki/tls/certs/ca-bundle.crt
  • In the Python SDK, prepend /etc/arvados/ca-certificates.crt to the array of paths in ca_certs_path() (source:sdk/python/arvados/util.py)
  • In the Go SDK, if /etc/arvados/ca-certificates.crt exists and insecure mode is off, read root CAs from there. Move code from source:sdk/go/crunchrunner/crunchrunner.go (but try /etc/arvados first, don't merge certs from multiple sources, don't silently ignore errors, and don't log a debug message).
This way,
  • if the sysadmin puts certs in /etc/arvados/ca-certificates.crt on worker nodes, those certs (and only those certs) will be used by arvados code running both inside & outside containers
  • on a given worker node, Go and Python programs use the same certificates

#7 Updated by Tom Morris almost 5 years ago

  • Target version set to 2017-01-04 sprint

#8 Updated by Peter Amstutz almost 5 years ago

  • Assigned To set to Peter Amstutz

#9 Updated by Peter Amstutz almost 5 years ago

  • Subject changed from [Crunch2] crunch-run option to add binds to all containers. to [Crunch2] crunch-run adding certificates to container

#10 Updated by Lucas Di Pentima almost 5 years ago

On file sdk/go/arvadosclient/arvadosclient.go:
  • Line 116: If tlsClientconfig.InsecureSkipVerify is true, can the cert file load be skipped? (asking this because of what Tom added on his proposal)
  • Line 124: Maybe it’s a good idea to check if AppendCertFromPEM() returns successfully

Also, got the following install errors when running tests locally:

           ********** Running sdk/go/crunchrunner install **********

# git.curoverse.com/arvados.git/sdk/go/crunchrunner
../../tmp/GOPATH/src/git.curoverse.com/arvados.git/sdk/go/crunchrunner/crunchrunner.go:4: imported and not used: "crypto/x509" 
../../tmp/GOPATH/src/git.curoverse.com/arvados.git/sdk/go/crunchrunner/crunchrunner.go:13: imported and not used: "net/http" 

     ********** !!!!!! sdk/go/crunchrunner install FAILED !!!!!! **********

#11 Updated by Peter Amstutz almost 5 years ago

Lucas Di Pentima wrote:

On file sdk/go/arvadosclient/arvadosclient.go:
  • Line 116: If tlsClientconfig.InsecureSkipVerify is true, can the cert file load be skipped? (asking this because of what Tom added on his proposal)

Done. Also refactored a setting up the TLS config a bit.

  • Line 124: Maybe it’s a good idea to check if AppendCertFromPEM() returns successfully

Done. Logs a warning now.

Also, got the following install errors when running tests locally:

Fixed.

Additional changes:

  • Add -ca-certs command line option to crunch-run
  • crunch-run only bind mounts certificates when API: true in runtime_constraints.

#12 Updated by Lucas Di Pentima almost 5 years ago

Ran sdk/go & services/crunch-run tests locally. LGTM.

#13 Updated by Peter Amstutz almost 5 years ago

  • Status changed from New to Resolved

Applied in changeset arvados|commit:ffd4738242c61fa5acd423f927339f836dfb0ffb.

Also available in: Atom PDF