Project

General

Profile

Actions

Support #16873

closed

Jenkins job that uses "kitchen test" to test salt formulas against all supported distros

Added by Peter Amstutz almost 4 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Javier Bértoli
Category:
Deployment
Target version:
Due date:
Story points:
-

Description

A new jenkins pipeline job which takes a given salt formula version and a given Arvados version (including dev) and performs installation and correctness checking using Dockerized environments of each supported distro.


Subtasks 2 (0 open2 closed)

Task #16880: Review #16873 - Jenkins job that uses "kitchen test" to test salt formulas against all supported distrosResolvedJavier Bértoli07/01/2021Actions
Task #17844: [jenkins] create an worker image to run arvados-formula testingResolvedJavier Bértoli06/30/2021Actions
Actions #1

Updated by Peter Amstutz almost 4 years ago

  • Description updated (diff)
  • Subject changed from Jenkins job that test-runs salt install in Docker against all supported distros to Jenkins job that uses "kitchen test" to test salt formulas against all supported distros
  • Tracker changed from Bug to Support
Actions #2

Updated by Peter Amstutz almost 4 years ago

  • Assigned To set to Javier Bértoli
Actions #3

Updated by Peter Amstutz almost 4 years ago

  • Target version changed from 2020-10-07 Sprint to 2020-10-21 Sprint
Actions #4

Updated by Peter Amstutz almost 4 years ago

  • Target version changed from 2020-10-21 Sprint to 2020-11-04 Sprint
Actions #5

Updated by Peter Amstutz over 3 years ago

  • Target version changed from 2020-11-04 Sprint to 2020-11-18
Actions #6

Updated by Peter Amstutz over 3 years ago

  • Target version deleted (2020-11-18)
Actions #8

Updated by Javier Bértoli about 3 years ago

There's a WIP, disabled

Actions #9

Updated by Javier Bértoli about 3 years ago

  • Target version set to 2021-07-07 sprint
  • Status changed from New to In Progress
  • Category set to Deployment
Actions #10

Updated by Javier Bértoli about 3 years ago

  • Status changed from In Progress to Feedback

Enabled the jenkins job

It uses the jenkins-image-arvados-formula-tests-{date} image created on #17844, through a cloud worker, using the label saltstack-tests.

It runs kitchen to verify the follow combinations:

$ kitchen list "debian-10-3003|centos-7-3003" 
Instance                        Driver  Provisioner  Verifier  Transport  Last Action    Last Error
api-debian-10-3003-0-py3        Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
api-centos-7-3003-0-py3         Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
workbench-debian-10-3003-0-py3  Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
workbench-centos-7-3003-0-py3   Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
shell-debian-10-3003-0-py3      Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
shell-centos-7-3003-0-py3       Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
keepstore-debian-10-3003-0-py3  Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>
keepstore-centos-7-3003-0-py3   Docker  SaltSolo     Inspec    Ssh        <Not Created>  <None>

(run kitchen list to see all the available)

The naming of the suites follow the kitchen-ci format: suite_name-platform, which are built following the Saltstack-formulas images (for more details see the suites and platforms sections in the kitchen.yml file)

Note that some arvados components are applied and tested on the same image, to speed up testing.

Also, tests are run concurrently for the same reason.

At the moment, centos tests are failing because of #17750, #17787, which is expected.

Didn't enable the ubuntu testing because debian should cover it, but can be enabled if desired. It's just a matter of adding an entry in the regex

Actions #11

Updated by Peter Amstutz about 3 years ago

  • Target version changed from 2021-07-07 sprint to 2021-07-21 sprint
Actions #12

Updated by Nico César about 3 years ago

At the end there is some extra info:

>>>>>>     Converge failed on instance <workbench-centos-7-3003-0-py3>.  Please see .kitchen/logs/workbench-centos-7-3003-0-py3.log for more details
>>>>>>     Converge failed on instance <api-centos-7-3003-0-py3>.  Please see .kitchen/logs/api-centos-7-3003-0-py3.log for more details

are the .kitchen/logs/* files lost in the worker instance that runs it? if that's the case, can we add it to "workspace files" to keep them?

Actions #13

Updated by Nico César about 3 years ago

I also noticed that "Color ANSI Console Output" is enabled for that job, butsomehow the colors are not there and the escape characters are... is there an easy way to disable colors all together?

Actions #15

Updated by Javier Bértoli about 3 years ago

Nico César wrote:

I also noticed that "Color ANSI Console Output" is enabled for that job, butsomehow the colors are not there and the escape characters are... is there an easy way to disable colors all together?

There are ways and I tried a few (with --no-color, txt, yaml)

It seems to be an issue with the ANSIColor plugin, because other parts of the output are correctly colored and when the run ends OK, the output is not garbled even where it fails in the other cases.

I'll enable "yaml", as it seems to be the only one that is correctly managed by the ANSIcolor plugin.

Actions #16

Updated by Javier Bértoli about 3 years ago

Nico César wrote:

I also noticed that "Color ANSI Console Output" is enabled for that job, butsomehow the colors are not there and the escape characters are... is there an easy way to disable colors all together?

Kept the formula code as-is, so running the formula on a local workspace works as usual, but modified the jenkins' job configuration to add a kitchen.local.yml" file with a @yaml outputter, so it's readable in jenkins.

A weird twist is that, adding an invalid output module, produces a yaml output as the one above, but that is parseable by jenkins and, therefore, colored

Actions #17

Updated by Javier Bértoli about 3 years ago

Nico César wrote:

At the end there is some extra info:

[...]

are the .kitchen/logs/* files lost in the worker instance that runs it? if that's the case, can we add it to "workspace files" to keep them?

The .kitchen/logs/* files are a per-suite file with the output already seen in jenkins' output. I've used them in very rare situations, but we can add them, I guess as artifacts?

Actions #18

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-07-21 sprint to 2021-08-04 sprint
Actions #19

Updated by Javier Bértoli almost 3 years ago

Modified the job to save the logs as artifacts as suggested

Actions #20

Updated by Peter Amstutz almost 3 years ago

  • Target version changed from 2021-08-04 sprint to 2021-08-18 sprint
Actions #21

Updated by Javier Bértoli almost 3 years ago

Added a few more changes to fix Nginx/Passenger/Arvados-{api,workbench} issues related to bundler version.

arvados-formula:commit:f3e5a4b branch fix-rvm-deps-in-centos.

Actions #22

Updated by Lucas Di Pentima almost 3 years ago

This LGTM, thanks!

Actions #23

Updated by Javier Bértoli almost 3 years ago

  • Status changed from Feedback to Resolved

Merged

Actions

Also available in: Atom PDF