Development and testing with Docker » History » Revision 13

« Previous | Revision 13/14 (diff) | Next »
Tom Clegg, 07/31/2014 04:12 PM

Development and testing with Docker

Arvados is primarily intended as a system to be run on large server clusters. For development and testing purposes, we have provided a process to build and deploy the Arvados system in Docker containers on a single machine. shows how to install and update a development system.

Additional sources of information

The docker/ file in the source distribution has more notes.

Installing the client libraries

./ (in the docker directory) will install the CLI tools like arv and arv-put. If that does not work for you, try installing these by hand:

  • RVM -- if you do not already have RVM installed on your workstation, install it now.
    sudo apt-get install curl libcurl3 libcurl3-gnutls libcurl4-openssl-dev python-pip
    curl -sSL | bash -s stable
    source ~/.rvm/scripts/rvm
    rvm install 2.1.0
  • The arvados-cli gem:
    gem install arvados-cli
    sudo pip install --upgrade httplib2

Debugging containers

Once it's running, you can get into a container using nsenter.

Install nsenter:

docker run --rm jpetazzo/nsenter cat /nsenter > /tmp/nsenter
sudo install -m 0755 /tmp/nsenter /usr/local/bin/

Use it to run a shell in your keep_server_0 container:

PID=$(docker inspect --format {{.State.Pid}} keep_server_0)
sudo nsenter --target $PID --mount --uts --ipc --net --pid


Once you have installed the servers and started them with arvdock start, and installed the client libraries, you should be able to work through the user tutorials (using your host instead of a shell VM) to upload data into Arvados and run Crunch jobs in local development mode.

Updated by Tom Clegg about 9 years ago · 13 revisions