Project

General

Profile

Development and testing with Docker » History » Version 14

Tom Clegg, 07/31/2014 06:45 PM

1 12 Tom Clegg
h1. Development and testing with Docker
2 1 Tim Pierce
3 12 Tom Clegg
Arvados is primarily intended as a system to be run on large server clusters.  For development and testing purposes, we have provided a process to build and deploy the Arvados system in "Docker":https://www.docker.io/ containers on a single machine.
4 1 Tim Pierce
5 12 Tom Clegg
http://doc.arvados.org/install/install-docker.html shows how to install and update a development system.
6 1 Tim Pierce
7 12 Tom Clegg
h2. Additional sources of information
8
9
The "docker/README.md":https://github.com/curoverse/arvados/blob/master/docker/README.md file in the source distribution has more notes.
10
11
h2. Installing the client libraries
12
13
@./install_sdk.sh@ (in the docker directory) will install the CLI tools like arv and arv-put. If that does not work for you, try installing these by hand:
14
15
* *RVM* -- if you do not already have RVM installed on your workstation, install it now.
16
<pre>
17
sudo apt-get install curl libcurl3 libcurl3-gnutls libcurl4-openssl-dev python-pip
18
curl -sSL https://get.rvm.io | bash -s stable
19
source ~/.rvm/scripts/rvm
20
rvm install 2.1.0
21
</pre>
22
23
* The arvados-cli gem:
24
<pre>
25
gem install arvados-cli
26
sudo pip install --upgrade httplib2
27
</pre>
28
29 13 Tom Clegg
h2. Debugging containers
30
31 14 Tom Clegg
Once it's running, you can get run a shell inside a container using nsenter.
32 13 Tom Clegg
33
Install nsenter:
34
35
<pre>
36
docker run --rm jpetazzo/nsenter cat /nsenter > /tmp/nsenter
37
sudo install -m 0755 /tmp/nsenter /usr/local/bin/
38
</pre>
39
40 14 Tom Clegg
Install this shortcut:
41 1 Tim Pierce
42
<pre>
43 14 Tom Clegg
sudo tee /usr/local/bin/nse <<'EOF'
44
#!/bin/sh
45
PID=$(docker inspect --format {{.State.Pid}} "$*")
46
exec nsenter --target $PID --mount --uts --ipc --net --pid
47
EOF
48
49
chmod +x /usr/local/bin/nse
50 1 Tim Pierce
</pre>
51 14 Tom Clegg
52
Then you can run a shell in a container like this:
53
54
<pre>
55
nse keep_server_0
56
</pre>
57
58
h3. Console
59
60
@./arvdock@ starts containers in detached mode. You can attach your terminal to them in order to see stdout/stderr:
61
62
<pre>
63
docker attach --sig-proxy=false keep_server_0
64
</pre>
65
66
Press control-C to detach. (If you didn't use --sig-proxy=false, control-C will send SIGINT into the container.)
67
68 13 Tom Clegg
69 12 Tom Clegg
h2. Tutorials
70
71
Once you have installed the servers and started them with @arvdock start@, and installed the client libraries, you should be able to work through the "user tutorials":http://doc.arvados.org/user/ (using your host instead of a shell VM) to upload data into Arvados and run Crunch jobs in local development mode.