Project

General

Profile

Hacking prerequisites » History » Version 70

Ward Vandewege, 06/30/2020 01:52 PM

1 1 Tom Clegg
{{>toc}}
2
3
h1. Hacking prerequisites
4
5
The Arvados test suite can run in a Docker container, a VM, or your workstation -- provided a few prerequisites are satisfied.
6
7
h2. Host options
8
9
h3. Starting on your workstation
10
11 70 Ward Vandewege
If your workstation is a debian buster system -- and you don't mind installing a bunch of packages on your workstation, some of them without apt -- the easiest way to get running is to run tests on bare metal. Skip to "Dependencies".
12 1 Tom Clegg
13
Other linux distributions should work too with some modifications, but it's probably easier to use a VM.
14
15
h3. Starting on a VM
16
17 54 Tom Clegg
Another option is to create a virtual machine using something like Xen or VirtualBox, and run debian stretch on it. The instructions below assume you have just a few basic requirements:
18 1 Tom Clegg
* SSH server
19
* sudo (@apt-get install sudo@)
20
* A user account with sudo privileges
21
22
h3. Starting in a docker container
23
24 55 Peter Amstutz
_[[Arvbox]] provides a preinstalled Docker-based dev environment.  The following instructions are for creating a dev environment inside Docker from scratch._
25
26 1 Tom Clegg
This can get you started quickly, but (unlike the above options) you'll need to remember to use something like @docker commit@ to save your state before shutting down your container.
27
28
See http://docker.io for more about installing docker. On debian it looks something like this.
29
30
<pre>
31 54 Tom Clegg
sudo apt-get install docker-ce
32 1 Tom Clegg
sudo adduser $USER docker
33
# {log out & log back in}
34
groups
35
# {should include "docker"}
36
</pre>
37
38 69 Ward Vandewege
Start up a new container with debian 10 (buster), make a new user and log in as that user:
39 1 Tom Clegg
40
<pre>
41 69 Ward Vandewege
docker run -it --privileged debian:10 bash
42 1 Tom Clegg
apt-get update
43 34 Tom Clegg
apt-get -y install sudo
44 1 Tom Clegg
adduser me
45 33 Tom Clegg
adduser me sudo
46 1 Tom Clegg
sudo -u me -i
47
</pre>
48 12 Joshua Randall
49 15 Tom Clegg
The &quot;--privileged&quot; is required in order for /dev/fuse to be accessible (without it, no tests that require FUSE will work).
50 1 Tom Clegg
51
h2. Install dev environment
52
53 67 Tom Clegg
Run the following commands as root.
54 36 Tom Clegg
55 67 Tom Clegg
Note that the last command here ("arvados-server install -type test") installs additional debian packages to your system, along with additional software in /var/lib/arvados/ (such as suitable versions of Ruby and Go) that do not interfere with system packages. It also creates a postgresql database user named "arvados" with an insecure password. Don't expose this postgresql server to the internet or to untrusted users!
56 66 Peter Amstutz
57 43 Tom Clegg
<pre>
58 68 Ward Vandewege
apt-get install --no-install-recommends build-essential ca-certificates git golang libpam0g-dev
59 1 Tom Clegg
cd
60 67 Tom Clegg
git clone https://git.arvados.org/arvados.git
61
cd arvados
62 1 Tom Clegg
go mod download
63 68 Ward Vandewege
cd cmd/arvados-server
64 67 Tom Clegg
go install
65
~/go/bin/arvados-server install -type test
66 1 Tom Clegg
</pre>
67
68
h2. Start Postgres
69
70
_If you're running in a docker container_ you'll need to start Postgres manually:
71 9 Joshua Randall
72 47 Tom Clegg
<pre>
73 11 Joshua Randall
sudo /etc/init.d/postgresql start
74 60 Peter Amstutz
</pre>
75
76
(If you're on a regular workstation/server/VM, startup scripts have already taken care of that for you.)
77
78
h2. Setup groups
79
80
Make sure the fuse and docker groups exist (create them if necessary) and that the user who will run the tests is a member of them.
81 1 Tom Clegg
82
h2. Run tests
83
84
<pre>
85 67 Tom Clegg
time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados
86 1 Tom Clegg
</pre>
87
88 14 Tom Clegg
During development, you'll probably want something more like this. It reuses the given temp directory, which avoids a lot of repetitive downloading of dependencies, and allows you to save time with @--skip-install@ or @--only-install sdk/ruby@ and so on.
89 1 Tom Clegg
90
<pre>
91 23 Tom Clegg
mkdir -p ~/.cache/arvados-build
92 67 Tom Clegg
time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados --temp ~/.cache/arvados-build
93 1 Tom Clegg
</pre>