• Table of contents
• Hacking prerequisites
  • Host options
    • Supported distributions
    • Base configuration
    • Virtual machine requirements
  • Install development environment with Ansible
    • Install Ansible
    • Write an Arvados database configuration
    • Write an Ansible inventory
    • Run the playbook
    • Troubleshooting
    • Notes

Hacking prerequisites¶

This page describes how to install all the software necessary to develop Arvados and run tests.

Host options¶

You must have a system running a supported distribution. That system can be installed directly on hardware; running on a cloud instance; or in a virtual machine.

Supported distributions¶

As of February 2025/Arvados 3.0, these instructions and the entire test suite are known to work on Debian 11 "bullseye." They mostly work on Debian 12 "bookworm," but a small handful of tests fail because they haven't been adapted to newer software yet.

You may try to run these instructions and tests on Ubuntu 20.04 "focal"/22.04 "jammy"/24.04 "noble," but they have not been tested and you are likely to hit some bugs throughout.

These instructions are not suitable for any Red Hat-based distribution and many tests would fail on them.

Base configuration¶

On your development system, you should have a user account with full permission to use sudo.

You can run the Ansible playbook to install your development system on a different system. To do this, you must have permission to SSH into your user account from the system running Ansible (the "control node") to the development system you're installing (the "target node").

Virtual machine requirements¶

If you run your development system in a virtual machine, it needs some permissions. Many environments will allow these operations by default, but they could be limited by your virtual machine setup.

• It must be able to create and manage FUSE mounts (/dev/fuse)
• It must be able to create and run Docker containers
• It must be able to create and run Singularity containers—this requires creating and managing block loopback devices (/dev/block-loop)
• It must have the fs.inotify.max_user_watches sysctl set to at least 524288. Our Ansible playbook will try to set this on the managed host, but if it is unable to do so, you may need to set it on the parent host instead.

Install development environment with Ansible¶

Install Ansible¶

The simplest thing to do is pip install Ansible inside a virtualenv:

sudo apt install python3-venv
python3 -m venv ~/arvados-ansible
~/arvados-ansible/bin/pip install "ansible~=8.7" 

If that works, you're done and you can go on to the next section. There are other installation options if you can't do this for some reason.

For background, the Arvados Ansible playbooks are tested with Ansible 8, which we use for its Python version compatibility. Older versions are known not to work. You're welcome to try newer versions if you want to for any reason, but they haven't been tested.

Write an Arvados database configuration¶

Next, write an Arvados configuration file that defines how you want to set up the PostgreSQL database. Write a file ~/arvados-ansible/zzzzz.yml like this:

Clusters:
  zzzzz:
    PostgreSQL:
      Connection:
        user: arvados
        password: GoodPasswordHere
        dbname: arvados_test
        host: localhost
        port: "5432" 

The cluster ID must be zzzzz. You can change the user, password, and dbname settings freely. Our Ansible playbook will configure PostgreSQL so your settings here work.

The playbook will always install the postgresql server package. It will not change any PostgreSQL configuration except to add pg_hba.conf entries for this user. You should only change host and port if you need to use a PostgreSQL server that is already installed and running somewhere else.

Write an Ansible inventory¶

An inventory file tells Ansible what host(s) to manage, how to connect to them, and what settings they use. Write an inventory file to ~/arvados-ansible/inventory.ini like this:

[arvados-test]
# This is the list of host(s) where we're installing the test environment.
# The line below installs on the same system running Ansible.
# If you want to manage remote hosts, you can write your own host list:
# <https://docs.ansible.com/ansible/latest/getting_started/get_started_inventory.html>
localhost ansible_connection=local

[arvados-test:vars]
# The path to the Arvados cluster configuration you wrote in the previous section.
arvados_config_file={{ lookup('env', 'HOME') }}/arvados-ansible/zzzzz.yml

# The primary user doing Arvados development and tests.
# This user will be added to the `docker` group.
# It defaults to the name of the user running `ansible-playbook`.
# If you want to configure a different user, set that here:
#arvados_dev_user=USERNAME

# The authentication mechanism to allow in `pg_hba.conf`.
# The default is `scram-sha-256`, which is the most secure method on the most
# recent versions of PostgreSQL.
# If your development system is running Debian 11, set this to `md5` here.
#arvados_postgresql_hba_method=md5

Run the playbook¶

The playbook is available from the tools/ansible directory of the Arvados source tree. If you don't already have a checkout of the Arvados source, get it by running:

git clone https://git.arvados.org/arvados.git ~/arvados

The basic command to run the playbook is:

cd arvados/tools/ansible
~/arvados-ansible/bin/ansible-playbook -K -i ~/arvados-ansible/inventory.ini install-test-env.yml

When you are prompted for the BECOME password:, enter the password for your user account on the development host that lets you run sudo commands.

ansible-playbook has many options to control how it runs that you can add if you like. Refer to the ansible-playbook documentation for more information.

After the playbook runs successfully, you should be able to run the Arvados tests from a source checkout on your development host. e.g.,

cd arvados
WORKSPACE="$PWD" build/run-tests.sh --temp ~/arvados-test --interactive

Refer to Running tests for details.

Troubleshooting¶

The playbook writes your database configuration at ~/.config/arvados/config.yml and sets up a hook /etc/profile.d/arvados-test.sh to set your CONFIGSRC environment variable to that directory. If most tests fail with a database connection error, check that this variable is set:

$ echo "${CONFIGSRC:-UNSET}" 
/home/you/.config/arvados

If that reports UNSET, add a line to set CONFIGSRC="$HOME/.config/arvados" to your shell configuration, or set it manually when you run run-tests.sh:

WORKSPACE="$PWD" CONFIGSRC="$HOME/.config/arvados" build/run-tests.sh ...

Notes¶

The playbook will install symlinks for Go, Node, Singularity, and Yarn under /usr/local/bin. The actual tools are installed under /opt. If you need different versions of these tools for other work on this system, you'll need to customize your PATH environment variable so the Arvados versions are found first when you're doing Arvados work.