Arvados

{{>toc}}

h1. Hacking prerequisites

This page describes how to install all the software necessary to develop Arvados and run tests.

h2. Host options

You must have a system running a supported distribution. That system can be installed directly on hardware; running on a cloud instance; or in a virtual machine.

h3. Supported distributions

As of February 2025/Arvados 3.0, these instructions and the entire test suite are known to work on Debian 11 "bullseye." They mostly work on Debian 12 "bookworm," but a small handful of tests fail because they haven't been adapted to newer software yet.

You may try to run these instructions and tests on Ubuntu 20.04 "focal"/22.04 "jammy"/24.04 "noble," but they have not been tested and you are likely to hit some bugs throughout.

These instructions are not suitable for any Red Hat-based distribution and many tests would fail on them.

h3. Base configuration

On your development system, you should have a user account with full permission to use sudo.

You can run the Ansible playbook to install your development system on a different system. To do this, you must have permission to SSH into your user account from the system running Ansible (the "control node") to the development system you're installing (the "target node").

h3. Virtual machine requirements

If you run your development system in a virtual machine, it needs some permissions. Many environments will allow these operations by default, but they could be limited by your virtual machine setup.

* It must be able to create and manage FUSE mounts (@/dev/fuse@)

* It must be able to create and run Docker containers

* It must be able to create and run Singularity containers—this requires creating and managing block loopback devices (@/dev/block-loop@)

* It must have the @fs.inotify.max_user_watches@ sysctl set to at least 524288. Our Ansible playbook will try to set this on the managed host, but if it is unable to do so, you may need to set it on the parent host instead.

h2. Install development environment with Ansible

h3. Install Ansible

The simplest thing to do is @pip install@ Ansible inside a virtualenv:

<pre><code class="shell">sudo apt install python3-venv

python3 -m venv ~/arvados-ansible

~/arvados-ansible/bin/pip install "ansible~=8.7"

</code></pre>

If that works, you're done and you can go on to the next section. There are "other installation options":https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html if you can't do this for some reason.

For background, the Arvados Ansible playbooks are tested with Ansible 8, which we use for its Python version compatibility. Older versions are known not to work. You're welcome to try newer versions if you want to for any reason, but they haven't been tested.

h3. Write an Arvados database configuration

Next, write an Arvados configuration file that defines how you want to set up the PostgreSQL database. Write a file @~/arvados-ansible/zzzzz.yml@ like this:

<pre><code class="yaml">Clusters:

  zzzzz:

    PostgreSQL:

      Connection:

        user: arvados

        password: GoodPasswordHere

        dbname: arvados_test

        host: localhost

        port: "5432"

</code></pre>

The cluster ID *must* be @zzzzz@. You can change the @user@, @password@, and @dbname@ settings freely. Our Ansible playbook will configure PostgreSQL so your settings here work.

The playbook will always install the @postgresql@ server package. It will *not* change any PostgreSQL configuration except to add @pg_hba.conf@ entries for this user. You should only change @host@ and @port@ if you need to use a PostgreSQL server that is already installed and running somewhere else.

h3. Write an Ansible inventory

An inventory file tells Ansible what host(s) to manage, how to connect to them, and what settings they use. Write an inventory file to @~/arvados-ansible/inventory.ini@ like this:

<pre><code class="ini">[arvados-test]

# This is the list of host(s) where we're installing the test environment.

# The line below installs on the same system running Ansible.

# If you want to manage remote hosts, you can write your own host list:

# <https://docs.ansible.com/ansible/latest/getting_started/get_started_inventory.html>

localhost ansible_connection=local

[arvados-test:vars]

# The path to the Arvados cluster configuration you wrote in the previous section.

arvados_config_file={{ lookup('env', 'HOME') }}/arvados-ansible/zzzzz.yml

# The primary user doing Arvados development and tests.

# This user will be added to the `docker` group.

# It defaults to the name of the user running `ansible-playbook`.

# If you want to configure a different user, set that here:

#arvados_dev_user=USERNAME

# The authentication mechanism to allow in `pg_hba.conf`.

# The default is `scram-sha-256`, which is the most secure method on the most

# recent versions of PostgreSQL.

# If your development system is running Debian 11, set this to `md5` here.

#arvados_postgresql_hba_method=md5

</code></pre>

h3. Run the playbook

The playbook is available from the @tools/ansible@ directory of the Arvados source tree. If you don't already have a checkout of the Arvados source, get it by running:

<pre><code class="sh">git clone https://git.arvados.org/arvados.git ~/arvados</code></pre>

The basic command to run the playbook is:

<pre><code class="sh">cd arvados/tools/ansible

~/arvados-ansible/bin/ansible-playbook -K -i ~/arvados-ansible/inventory.ini install-test-env.yml</code></pre>

@ansible-playbook@ has many options to control how it runs that you can add if you like. Refer to "the @ansible-playbook@ documentation":https://docs.ansible.com/ansible/latest/cli/ansible-playbook.html for more information.

After the playbook runs successfully, you should be able to run the Arvados tests from a source checkout on your development host. e.g.,

<pre><code class="sh">cd arvados

WORKSPACE="$PWD" build/run-tests.sh --temp ~/arvados-test --interactive

</code></pre>

Refer to [[Running tests]] for details.

h3. Troubleshooting

The playbook writes your database configuration at @~/.config/arvados/config.yml@ and sets up a hook @/etc/profile.d/arvados-test.sh@ to set your @CONFIGSRC@ environment variable to that directory. If most tests fail with a database connection error, check that this variable is set:

<pre>$ echo "${CONFIGSRC:-UNSET}"

/home/you/.config/arvados

</pre>

If that reports @UNSET@, add a line to set @CONFIGSRC="$HOME/.config/arvados"@ to your shell configuration, or set it manually when you run @run-tests.sh@:

<pre><code class="sh">WORKSPACE="$PWD" CONFIGSRC="$HOME/.config/arvados" build/run-tests.sh ...

</code></pre>

h3. Notes

The playbook will install symlinks for Go, Node, Singularity, and Yarn under @/usr/local/bin@. The actual tools are installed under @/opt@. If you need different versions of these tools for other work on this system, you'll need to customize your @PATH@ environment variable so the Arvados versions are found first when you're doing Arvados work.