Story #8815

[Crunch] crunch-job bind mounts crunchrunner & host certs file at well known location inside container

Added by Peter Amstutz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
03/29/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

In order to run arbitrary containers, we need crunchrunner (a compiled Go program) and a TLS certificates file available inside the container. Currently arvados-cwl-runner provides this by assuming a collection with a specific PDH is available on the system. This is brittle, as crunchrunner is an infrastructure that should be managed as part of installation and upgrade.

Solution:

- Create arvados-crunchrunner package which is automatically installed on compute & shell nodes
- crunch-job bind mounts crunchrunner & host system TLS certs file at a well known location
- crunchrunner crunch script uses well know location to run crunchrunner instead of running from collection
- remove code from arvados-cwl-runner for handling crunchrunner collection


Subtasks

Task #8818: Review 8815-crunchrunner-everywhereResolvedWard Vandewege


Related issues

Related to Arvados - Story #8654: [CWL] Incorporate cwl-runner into arvados/jobsResolved03/07/2016

Related to Arvados - Feature #8731: Update arv-box to use arvados/build/* instead of arvados-dev/jenkins/*Resolved03/16/2016

Related to Arvados - Bug #8828: [Crunch] be more resilient when crunchrunner is not available; also don't test for crunchrunner on api serverResolved03/31/2016

Associated revisions

Revision f2f5c6aa (diff)
Added by Ward Vandewege over 3 years ago

Build a package for crunchrunner.

refs #8815

Revision a7d819f6
Added by Peter Amstutz over 3 years ago

Merge branch '8815-crunchrunner-everywhere' closes #8815

Revision dbafab75 (diff)
Added by Peter Amstutz over 3 years ago

Fix sdk/cwl test refs #8815

History

#1 Updated by Peter Amstutz over 3 years ago

  • Description updated (diff)

#2 Updated by Peter Amstutz over 3 years ago

  • Target version set to 2016-03-30 sprint

#3 Updated by Peter Amstutz over 3 years ago

  • Status changed from New to In Progress
  • Story points set to 0.5

#4 Updated by Peter Amstutz over 3 years ago

  • Assigned To set to Peter Amstutz

#5 Updated by Ward Vandewege over 3 years ago

Review comments on dfc93aac9c256d6ebb868aeb6c2107821e9fd041:

  • crunchrunner when installed from packages will live in /usr/bin/crunchrunner
  • as for the question of the ca-certificates file, let's do this:

- the Arvados-specific ca-certificates file will live in /etc/arvados/ca-certificates.crt
- If that file exists, crunch-runner should add it to the list of system certificates (if any) and then connect to the API server. Maybe this logic should end up in the Go SDK?
- we'll do the same for the Python and Ruby SDKs later, as separate stories. In Workbench, we already do something similar but just with the list of system certificates.
- the default behavior for crunch-job should be to load the host certificates file in the container at /etc/arvados/ca-certificates.crt, so that crunch-runner can do its job even when the container doesn't have any certificates installed, and so that there is no special site configuration required in the common case, while allowing for a local setup to override this behavior by means of custom docker options

Other than that, 8815-crunchrunner-everywhere looks good to me, thanks!

#6 Updated by Peter Amstutz over 3 years ago

Ward Vandewege wrote:

Review comments on dfc93aac9c256d6ebb868aeb6c2107821e9fd041:

  • crunchrunner when installed from packages will live in /usr/bin/crunchrunner

Now bind mounts crunchrunner to /usr/local/bin/crunchrunner inside the container. crunch-job uses `which` to find crunchrunner on the PATH.

  • as for the question of the ca-certificates file, let's do this:

- the Arvados-specific ca-certificates file will live in /etc/arvados/ca-certificates.crt
- If that file exists, crunch-runner should add it to the list of system certificates (if any) and then connect to the API server. Maybe this logic should end up in the Go SDK?
- we'll do the same for the Python and Ruby SDKs later, as separate stories. In Workbench, we already do something similar but just with the list of system certificates.
- the default behavior for crunch-job should be to load the host certificates file in the container at /etc/arvados/ca-certificates.crt, so that crunch-runner can do its job even when the container doesn't have any certificates installed, and so that there is no special site configuration required in the common case, while allowing for a local setup to override this behavior by means of custom docker options

So crunchrunner now loads both the system certificates inside the container (/etc/ssl/certs/ca-certificates.crt) and also /etc/arvados/ca-certificates.crt.

#7 Updated by Ward Vandewege over 3 years ago

Reviewing at 94f5be8c86ad975ee7aa9f3df87be23fbc154dec: looks great, please merge. Thank you!

#8 Updated by Peter Amstutz over 3 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:a7d819f660840767df3e393a30a775f445db266e.

Also available in: Atom PDF