Bug #17177

[salt][provision] when using SnakeOil certs, Arvados needs a CA or some components won't work correctly

Added by Javier Bértoli 5 months ago. Updated about 16 hours ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Deployment
Target version:
Start date:
12/04/2020
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

As discussed in gitter, the provision installer needs to create a CA which can then be installed by the user, or Arvados' won't work correctly: self-signed certificates are silently discarded by some libraries.


Subtasks

Task #17193: review 17177-use-newly-created-caResolvedJavier Bértoli

Associated revisions

Revision 057cf02d (diff)
Added by Javier Bértoli 5 months ago

fix(provision): Add a CA and sign certificates with it

refs #17177

As discussed [here](https://forum.arvados.org/t/debugging-arvados-deployed-with-salt/58/8)
and [here](https://gitter.im/arvados/community?at=5fc65683496ca3372e3474a3), Arvados needs
certs signed by a known CA to work correctly.

This PR adds a CA and leaves a copy of the certificate in the installer directory.

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 7b009edf (diff)
Added by Javier Bértoli 5 months ago

fix(provision): Document CA certificate purpose and installation

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision e58d099d (diff)
Added by Javier Bértoli 5 months ago

fix(provision): pin formulas' versions

  • Pin formulas versions, to prevent changes upstream breaking the installer
  • Remove verbose/debug flags
  • Remove unused entries in the config pillars

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 05010666 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): add port to workbench2 nginx's stanza

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision ebd40412 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): pin arvados-formula

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision a4345503 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): update arvados-formula's version

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 8fa0ae7e
Added by Javier Bértoli 5 months ago

Merge branch '17177-use-newly-created-ca'

closes #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 7f84cbbc (diff)
Added by Javier Bértoli 5 months ago

fix(provision): Add a CA and sign certificates with it

refs #17177

As discussed [here](https://forum.arvados.org/t/debugging-arvados-deployed-with-salt/58/8)
and [here](https://gitter.im/arvados/community?at=5fc65683496ca3372e3474a3), Arvados needs
certs signed by a known CA to work correctly.

This PR adds a CA and leaves a copy of the certificate in the installer directory.

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 7e4e9c15 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): Document CA certificate purpose and installation

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 799bde4e (diff)
Added by Javier Bértoli 5 months ago

fix(provision): pin formulas' versions

  • Pin formulas versions, to prevent changes upstream breaking the installer
  • Remove verbose/debug flags
  • Remove unused entries in the config pillars

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 08dcbff1 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): add port to workbench2 nginx's stanza

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision d0301d9d (diff)
Added by Javier Bértoli 5 months ago

fix(provision): pin arvados-formula

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

Revision 7919e888 (diff)
Added by Javier Bértoli 5 months ago

fix(provision): update arvados-formula's version

refs #17177

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <>

History

#1 Updated by Peter Amstutz 5 months ago

  • Target version changed from 2020-12-02 Sprint to 2020-12-16 Sprint

#2 Updated by Javier Bértoli 5 months ago

  • % Done changed from 0 to 100

Submitted an arvados-formula's PR and updated the provision.sh script and docs (commit 7b009edfb, branch 17177-use-newly-created-ca). Added a curl test to verify the cert is valid.

#3 Updated by Lucas Di Pentima 5 months ago

  • On the sls files, there’re commented letsencrypt.conf snippets entries, I think we could get rid of them if they aren’t used (following our own code style standards of not keeping commented out code).
  • Got some error when trying to start it with vagrant (as a new instance)
...
    arvados:     Rendering SLS 'base:docker.software.package.repo.install' failed: Jinja variable 'null' is undefined
    arvados: Removing .psql file
    arvados: + '[' xyes = xyes ']'
    arvados: + echo 'Removing .psql file'
    arvados: + rm /root/.psqlrc
    arvados: Copying the Arvados CA certificate to the installer dir, so you can import it
    arvados: + '[' x = xyes ']'
    arvados: + echo 'Copying the Arvados CA certificate to the installer dir, so you can import it'
    arvados: + '[' xyes = xyes ']'
    arvados: + cp /etc/ssl/certs/arvados-snakeoil-ca.pem /vagrant
    arvados: cp:
    arvados: cannot stat '/etc/ssl/certs/arvados-snakeoil-ca.pem'
    arvados: : No such file or directory
    arvados: Adding the vagrant user to the docker group
    arvados: + echo 'Adding the vagrant user to the docker group'
    arvados: + usermod -a -G docker vagrant
    arvados: usermod: group 'docker' does not exist
    arvados: + '[' xyes = xyes ']'
    arvados: + cd /tmp/cluster_tests
    arvados: + ./run-test.sh
    arvados: The Arvados CA was not correctly installed. Although some components will work,
    arvados: others won't. Please verify that the CA cert file was installed correctly and
    arvados: retry running these tests.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

#4 Updated by Javier Bértoli 5 months ago

Added a few more commits, commit ebd40412f@arvados, branch 17177-use-newly-created-ca

Tested locally and both the WB2-UI and CLI work.

#5 Updated by Lucas Di Pentima 5 months ago

a434550 LGTM, thanks!

#6 Updated by Javier Bértoli 5 months ago

  • Status changed from In Progress to Resolved

#7 Updated by Peter Amstutz about 16 hours ago

  • Release set to 38

Also available in: Atom PDF