Project

General

Profile

Hacking prerequisites » History » Version 67

Tom Clegg, 04/07/2020 08:52 PM

1 1 Tom Clegg
{{>toc}}
2
3
h1. Hacking prerequisites
4
5
The Arvados test suite can run in a Docker container, a VM, or your workstation -- provided a few prerequisites are satisfied.
6
7
h2. Host options
8
9
h3. Starting on your workstation
10
11 54 Tom Clegg
If your workstation is a debian stretch system -- and you don't mind installing a bunch of packages on your workstation, some of them without apt -- the easiest way to get running is to run tests on bare metal. Skip to "Dependencies".
12 1 Tom Clegg
13
Other linux distributions should work too with some modifications, but it's probably easier to use a VM.
14
15
h3. Starting on a VM
16
17 54 Tom Clegg
Another option is to create a virtual machine using something like Xen or VirtualBox, and run debian stretch on it. The instructions below assume you have just a few basic requirements:
18 1 Tom Clegg
* SSH server
19
* sudo (@apt-get install sudo@)
20
* A user account with sudo privileges
21
22
h3. Starting in a docker container
23
24 55 Peter Amstutz
_[[Arvbox]] provides a preinstalled Docker-based dev environment.  The following instructions are for creating a dev environment inside Docker from scratch._
25
26 1 Tom Clegg
This can get you started quickly, but (unlike the above options) you'll need to remember to use something like @docker commit@ to save your state before shutting down your container.
27
28
See http://docker.io for more about installing docker. On debian it looks something like this.
29
30
<pre>
31 54 Tom Clegg
sudo apt-get install docker-ce
32 1 Tom Clegg
sudo adduser $USER docker
33
# {log out & log back in}
34
groups
35
# {should include "docker"}
36
</pre>
37
38 53 Tom Clegg
Start up a new container with debian 9 (stretch), make a new user and log in as that user:
39 1 Tom Clegg
40
<pre>
41 53 Tom Clegg
docker run -it --privileged debian:9 bash
42 1 Tom Clegg
apt-get update
43 34 Tom Clegg
apt-get -y install sudo
44 1 Tom Clegg
adduser me
45 33 Tom Clegg
adduser me sudo
46 1 Tom Clegg
sudo -u me -i
47
</pre>
48 12 Joshua Randall
49 15 Tom Clegg
The &quot;--privileged&quot; is required in order for /dev/fuse to be accessible (without it, no tests that require FUSE will work).
50 1 Tom Clegg
51
h2. Install dev environment
52
53 67 Tom Clegg
Run the following commands as root.
54 36 Tom Clegg
55 67 Tom Clegg
Note that the last command here ("arvados-server install -type test") installs additional debian packages to your system, along with additional software in /var/lib/arvados/ (such as suitable versions of Ruby and Go) that do not interfere with system packages. It also creates a postgresql database user named "arvados" with an insecure password. Don't expose this postgresql server to the internet or to untrusted users!
56 66 Peter Amstutz
57 43 Tom Clegg
<pre>
58 67 Tom Clegg
apt-get install --no-install-recommends build-essential ca-certificates git golang
59 1 Tom Clegg
cd
60 67 Tom Clegg
git clone https://git.arvados.org/arvados.git
61
cd arvados
62
go mod download
63
go install
64
~/go/bin/arvados-server install -type test
65 1 Tom Clegg
</pre>
66
67
h2. Start Postgres
68
69
_If you're running in a docker container_ you'll need to start Postgres manually:
70 9 Joshua Randall
71 47 Tom Clegg
<pre>
72 11 Joshua Randall
sudo /etc/init.d/postgresql start
73 60 Peter Amstutz
</pre>
74
75
(If you're on a regular workstation/server/VM, startup scripts have already taken care of that for you.)
76
77
h2. Setup groups
78
79
Make sure the fuse and docker groups exist (create them if necessary) and that the user who will run the tests is a member of them.
80 1 Tom Clegg
81
h2. Run tests
82
83
<pre>
84 67 Tom Clegg
time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados
85 1 Tom Clegg
</pre>
86
87 14 Tom Clegg
During development, you'll probably want something more like this. It reuses the given temp directory, which avoids a lot of repetitive downloading of dependencies, and allows you to save time with @--skip-install@ or @--only-install sdk/ruby@ and so on.
88 1 Tom Clegg
89
<pre>
90 23 Tom Clegg
mkdir -p ~/.cache/arvados-build
91 67 Tom Clegg
time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados --temp ~/.cache/arvados-build
92 1 Tom Clegg
</pre>