Hacking prerequisites » History » Version 1
Tom Clegg, 06/30/2015 09:08 PM
1 | 1 | Tom Clegg | {{>toc}} |
---|---|---|---|
2 | |||
3 | h1. Hacking prerequisites |
||
4 | |||
5 | The Arvados test suite can run in a Docker container, a VM, or your workstation -- provided a few prerequisites are satisfied. |
||
6 | |||
7 | h2. Host options |
||
8 | |||
9 | h3. Starting on your workstation |
||
10 | |||
11 | If your workstation is a debian wheezy or jessie system -- and you don't mind installing a bunch of packages on your workstation, some of them without apt -- the easiest way to get running is to run tests on bare metal. Skip to "Dependencies". |
||
12 | |||
13 | Other linux distributions should work too with some modifications, but it's probably easier to use a VM. |
||
14 | |||
15 | h3. Starting on a VM |
||
16 | |||
17 | Another option is to create a virtual machine using something like Xen or VirtualBox, and run debian jessie on it. The instructions below assume you have just a few basic requirements: |
||
18 | * SSH server |
||
19 | * sudo (@apt-get install sudo@) |
||
20 | * A user account with sudo privileges |
||
21 | |||
22 | h3. Starting in a docker container |
||
23 | |||
24 | This can get you started quickly, but (unlike the above options) you'll need to remember to use something like @docker commit@ to save your state before shutting down your container. |
||
25 | |||
26 | See http://docker.io for more about installing docker. On debian it looks something like this. |
||
27 | |||
28 | <pre> |
||
29 | echo 'deb http://http.debian.net/debian jessie-backports main' | sudo tee -a /etc/apt/sources.list.d/jessie-backports.list |
||
30 | sudo apt-get install docker.io |
||
31 | sudo adduser $USER docker |
||
32 | # {log out & log back in} |
||
33 | groups |
||
34 | # {should include "docker"} |
||
35 | </pre> |
||
36 | |||
37 | Start up a new container with jessie, make a new user and log in as that user: |
||
38 | |||
39 | <pre> |
||
40 | docker run -it debian:jessie bash |
||
41 | apt-get update |
||
42 | apt-get install sudo |
||
43 | adduser me |
||
44 | adduser me sudo |
||
45 | sudo -u me -i |
||
46 | </pre> |
||
47 | |||
48 | h2. Install dev environment |
||
49 | |||
50 | <pre> |
||
51 | apt-get install bison build-essential fuse gettext git graphviz \ |
||
52 | iceweasel libattr1-dev libfuse-dev libcurl3 libcurl3-gnutls \ |
||
53 | libcurl4-openssl-dev libpcre3-dev libpq-dev libpython2.7-dev \ |
||
54 | libreadline-dev libssl-dev libxslt1.1 nginx postgresql python \ |
||
55 | pkg-config sudo virtualenv wget xvfb zlib1g-dev |
||
56 | |||
57 | # ruby 2.1: |
||
58 | sudo apt-get install ruby2.1 || \ |
||
59 | ( |
||
60 | set -e |
||
61 | mkdir -p ~/src |
||
62 | cd ~/src |
||
63 | wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.6.tar.gz |
||
64 | tar xzf ruby-2.1.6.tar.gz |
||
65 | cd ruby-2.1.6 |
||
66 | ./configure --disable-install-doc |
||
67 | make |
||
68 | sudo make install |
||
69 | sudo gem install bundler |
||
70 | ) |
||
71 | |||
72 | # go >= 1.3 |
||
73 | sudo apt-get install golang=2:1.3.3-1 || \ |
||
74 | ( |
||
75 | set -e |
||
76 | wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz |
||
77 | sudo tar -C /usr/local -xzf go1.4.2.linux-amd64.tar.gz |
||
78 | cd /usr/local/bin |
||
79 | sudo ln -s ../go/bin/* . |
||
80 | ) |
||
81 | |||
82 | # phantomjs 1.9.8 |
||
83 | ( |
||
84 | set -e |
||
85 | PJS=phantomjs-1.9.8-linux-x86_64 |
||
86 | wget -P /tmp https://bitbucket.org/ariya/phantomjs/downloads/$PJS.tar.bz2 |
||
87 | sudo tar -C /usr/local -xjf /tmp/$PJS.tar.bz2 |
||
88 | sudo ln -s ../$PJS/bin/phantomjs /usr/local/bin/ |
||
89 | ) |
||
90 | </pre> |
||
91 | |||
92 | h2. Get the arvados source tree and test scripts |
||
93 | |||
94 | <pre> |
||
95 | cd |
||
96 | git clone https://github.com/curoverse/arvados.git |
||
97 | git clone https://github.com/curoverse/arvados-dev.git |
||
98 | </pre> |
||
99 | |||
100 | ...or, if you're a committer with your public key on our git server: |
||
101 | |||
102 | <pre> |
||
103 | cd |
||
104 | git clone git@git.curoverse.com:arvados.git |
||
105 | git clone git@git.curoverse.com:arvados-dev.git |
||
106 | </pre> |
||
107 | |||
108 | h2. Create a Postgres user |
||
109 | |||
110 | This creates an "arvados" user with "create database" privileges. The test suite will create and drop the arvados_test database as needed. |
||
111 | |||
112 | <pre> |
||
113 | newpw=`tr -cd a-zA-Z </dev/urandom |head -c32` |
||
114 | sudo -u postgres psql -c "create user arvados with createdb encrypted password '$newpw'" |
||
115 | cp -i ~/arvados/services/api/config/database.yml{.sample,} |
||
116 | newpw="$newpw" perl -pi~ -e 's/xxxxxxxx/$ENV{newpw}/' ~/arvados/services/api/config/database.yml |
||
117 | </pre> |
||
118 | |||
119 | h2. Run tests |
||
120 | |||
121 | <pre> |
||
122 | time ~/arvados-dev/jenkins/run-tests.sh WORKSPACE=~/arvados |
||
123 | </pre> |
||
124 | |||
125 | During development, you'll probably want something more like this. It leaves its temp dirs in place after running tests, which allows you to save time with @--skip-install@ or @--only-install sdk/ruby@ and so on. |
||
126 | |||
127 | <pre> |
||
128 | time ~/arvados-dev/jenkins/run-tests.sh WORKSPACE=~/arvados \ |
||
129 | VENVDIR=~/.cache/arvados-venv \ |
||
130 | VENV3DIR=~/.cache/arvados-venv3 \ |
||
131 | GOPATH=~/.cache/arvados-gopath \ |
||
132 | GEMHOME=~/.cache/arvados-gemhome |
||
133 | </pre> |