Project

General

Profile

Running tests » History » Version 8

Ward Vandewege, 12/29/2014 03:48 PM

1 1 Tom Clegg
h1. Running tests
2
3 3 Tom Clegg
{{toc}}
4
5 1 Tom Clegg
The arvados-dev git repository has a @run-tests.sh@ script which tests (nearly) all of the components in the arvados source tree. Jenkins at https://ci.curoverse.com uses this exact script, so running it _before pushing a new master_ is a good way to predict whether Jenkins will fail your build and start pestering you by IRC.
6
7
h2. Background
8
9
You have the arvados source tree at @~/arvados@ and you might have local modifications.
10
11
h2. Download/update the test script
12
13
First time:
14
15
<pre>
16
cd
17
git clone git://git.curoverse.com:arvados-dev.git
18
</pre>
19
20
Next times, to get any updates:
21
22
<pre>
23
cd ~/arvados-dev
24
git pull
25
</pre>
26
27
h2. Install prerequisites
28
29
The test script doesn't (yet) check for all prerequisites up front, so you'll save some time by installing these things before going any attempting to run tests:
30
* [[Go]]
31 6 Ward Vandewege
* "API server dependencies":http://doc.arvados.org/install/install-api-server.html (Install debian packages, ruby, bundler. Create "arvados" postgres user with permission to create databases. You don't need to run "bundle install", though.)
32 1 Tom Clegg
* "Workbench dependencies":http://doc.arvados.org/install/install-workbench-app.html (Currently just graphviz. Skip "bundle install".)
33 4 Tom Clegg
* doc build dependencies: <pre>sudo apt-get install python-epydoc linkchecker</pre>
34 8 Ward Vandewege
* "FUSE driver dependencies":https://arvados.org/projects/arvados/wiki/Hacking_Python_SDK#Prerequisites
35 4 Tom Clegg
* Workbench test suite dependencies (see [[Hacking Workbench]]): <pre>
36
sudo apt-get install xvfb iceweasel
37
(set -e
38
 PJS=phantomjs-1.9.7-linux-x86_64
39
 wget -P /tmp https://bitbucket.org/ariya/phantomjs/downloads/$PJS.tar.bz2
40
 sudo tar -C /usr/local -xjf /tmp/$PJS.tar.bz2
41
 sudo ln -s ../$PJS/bin/phantomjs /usr/local/bin/
42
)
43 2 Tom Clegg
</pre>
44 1 Tom Clegg
45
h2. Run all tests
46
47
Basic usage:
48
49
<pre>
50
~/arvados-dev/jenkins/run-tests.sh WORKSPACE=~/arvados
51
</pre>
52
53
h2. Save time by skipping some tests
54
55
Running all tests takes a while. You don't want to do this after each tiny change. You can use the --skip and --only arguments to select specific components. Some components also accept component-specific arguments to select specific test classes/cases.
56
57
h2. Save more time by skipping install
58
59
Normally, even when you're running only one component's test suite, @run-tests.sh@ still runs the _install_ recipe for all of the components. This addresses dependencies between components: for example, Workbench tests expect the Python and CLI SDKs to be installed.
60
61
When you're not worried about these dependencies going out of sync (e.g., you're only touching Workbench tests) you can install everything once this way:
62
63
<pre>
64
~/arvados-dev/jenkins/run-tests.sh --only install --leave-temp WORKSPACE=~/arvados
65
</pre>
66
67
After running everything, you'll see something like this:
68
69
<pre>
70
Leaving behind temp dirs:  VENVDIR="/tmp/tmp.y3tsTmigio" GOPATH="/tmp/tmp.3r4sSA9F3l"
71
</pre>
72
73
Copy these temp dirs to the end of your run-tests.sh command line, add @--skip-install@, and choose a test suite to run, like this:
74
75
<pre>
76
~/arvados-dev/jenkins/run-tests.sh --only workbench --skip-install WORKSPACE=~/arvados VENVDIR="/tmp/tmp.y3tsTmigio" GOPATH="/tmp/tmp.3r4sSA9F3l"
77 3 Tom Clegg
</pre>
78
79
h2. Other options
80
81
For more usage info, try:
82
83
<pre>
84
~/arvados-dev/jenkins/run-tests.sh --help
85 1 Tom Clegg
</pre>
86 5 Tom Clegg
87
h1. Deficiencies
88
89
There's currently nowhere to add...
90
* Unit tests for Workbench JavaScript assets (see #4132)
91
* Unit/functional tests for crunch-job (see #4156)
92
* Tests for "arv"
93
* Tests for "arv-run-pipeline-instance"
94
* Tests for crunchstat