Hacking prerequisites » History » Revision 6
« Previous |
Revision 6/77
(diff)
| Next »
Tom Clegg, 06/30/2015 10:43 PM
- Table of contents
- Hacking prerequisites
Hacking prerequisites¶
The Arvados test suite can run in a Docker container, a VM, or your workstation -- provided a few prerequisites are satisfied.
Host options¶
Starting on your workstation¶
If your workstation is a debian wheezy or jessie system -- and you don't mind installing a bunch of packages on your workstation, some of them without apt -- the easiest way to get running is to run tests on bare metal. Skip to "Dependencies".
Other linux distributions should work too with some modifications, but it's probably easier to use a VM.
Starting on a VM¶
Another option is to create a virtual machine using something like Xen or VirtualBox, and run debian jessie on it. The instructions below assume you have just a few basic requirements:- SSH server
- sudo (
apt-get install sudo
) - A user account with sudo privileges
Starting in a docker container¶
This can get you started quickly, but (unlike the above options) you'll need to remember to use something like docker commit
to save your state before shutting down your container.
See http://docker.io for more about installing docker. On debian it looks something like this.
echo 'deb http://http.debian.net/debian jessie-backports main' | sudo tee -a /etc/apt/sources.list.d/jessie-backports.list sudo apt-get install docker.io sudo adduser $USER docker # {log out & log back in} groups # {should include "docker"}
Start up a new container with jessie, make a new user and log in as that user:
docker run -it debian:jessie bash apt-get update apt-get install sudo adduser me adduser me sudo sudo -u me -i
Install dev environment¶
sudo apt-get install bison build-essential fuse gettext git graphviz \ iceweasel libattr1-dev libfuse-dev libcurl3 libcurl3-gnutls \ libcurl4-openssl-dev libpcre3-dev libpq-dev libpython2.7-dev \ libreadline-dev libssl-dev libxslt1.1 linkchecker nginx \ postgresql python python-epydoc pkg-config sudo virtualenv \ wget xvfb zlib1g-dev # ruby 2.1: sudo apt-get install ruby2.1 ruby2.1-dev || \ ( set -e mkdir -p ~/src cd ~/src wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.6.tar.gz tar xzf ruby-2.1.6.tar.gz cd ruby-2.1.6 ./configure --disable-install-doc make sudo make install sudo gem install bundler ) # go >= 1.3 sudo apt-get install golang=2:1.3.3-1 || \ ( set -e wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz sudo tar -C /usr/local -xzf go1.4.2.linux-amd64.tar.gz cd /usr/local/bin sudo ln -s ../go/bin/* . ) # phantomjs 1.9.8 ( set -e PJS=phantomjs-1.9.8-linux-x86_64 wget -P /tmp https://bitbucket.org/ariya/phantomjs/downloads/$PJS.tar.bz2 sudo tar -C /usr/local -xjf /tmp/$PJS.tar.bz2 sudo ln -s ../$PJS/bin/phantomjs /usr/local/bin/ )
Get the arvados source tree and test scripts¶
cd git clone https://github.com/curoverse/arvados.git git clone https://github.com/curoverse/arvados-dev.git
...or, if you're a committer with your public key on our git server:
cd git clone git@git.curoverse.com:arvados.git git clone git@git.curoverse.com:arvados-dev.git
Start Postgres¶
If you're running in a docker container you'll need to start Postgres manually:
sudo /etc/init.d/postgresql start
(If you're on a regular workstation/server/VM, startup scripts have already taken care of that for you.)
Create a Postgres user¶
Create an "arvados" user with "create database" privileges. The test suite will create and drop the arvados_test database as needed.
newpw=`tr -cd a-zA-Z </dev/urandom |head -c32` sudo -u postgres psql -c "create user arvados with createdb encrypted password '$newpw'" cp -i ~/arvados/services/api/config/database.yml{.sample,} newpw="$newpw" perl -pi~ -e 's/xxxxxxxx/$ENV{newpw}/' ~/arvados/services/api/config/database.yml
Run tests¶
time ~/arvados-dev/jenkins/run-tests.sh WORKSPACE=~/arvados
During development, you'll probably want something more like this. It leaves its temp dirs in place after running tests, which allows you to save time with --skip-install
or --only-install sdk/ruby
and so on.
time ~/arvados-dev/jenkins/run-tests.sh WORKSPACE=~/arvados \ VENVDIR=~/.cache/arvados-venv \ VENV3DIR=~/.cache/arvados-venv3 \ GOPATH=~/.cache/arvados-gopath \ GEMHOME=~/.cache/arvados-gemhome
Updated by Tom Clegg about 9 years ago · 6 revisions