Build an Arvados development environment with OpenSSL 1.1¶
HISTORICAL¶
This page was written to address a deployment need for Arvados 2.5/2.6. Arvados 2.7 introduced support for Ruby 3.0+ which resolves this whole issue: you can deploy Arvados 2.7 on a distribution with OpenSSL 3.x and Ruby 3.x.
Background¶
The Arvados 2.x series requires Rails 5.2,
which requires Ruby 2.7,
which requires OpenSSL 1.1.
But Ubuntu 22.04+, Debian 12+, and RHEL 9+ all ship with OpenSSL 3. To get Arvados working on these distributions, you need to build OpenSSL 1.1 yourself, then build a complete Ruby stack linked against it, including the openssl
and pg
gems. This page documents how to do that.
This is expert-level stuff. This page does not walk you through downloading, extracting, and navigating source. If you're not comfortable figuring out those steps yourself, then you probably won't be comfortable working with this stack once it's set up. Consider another development solution like arvbox.
Environment assumptions¶
We're gonna install the whole stack under /opt/arvados
. You can adjust to taste if you like.
I have /var/lib/arvados
symlinked to /opt/arvados
for compatibility with arvados-server install
.
I have write permission to /opt/arvados
, so I can install stuff without sudo
or any other elevated permissions.
I have /opt/arvados/bin
near the front of my $PATH
. This is important, even if the directory starts empty, because some of these build steps will install tools here that later build steps need.
Notes¶
I have documented the exact source code I used for all steps, but there should be a little slack in specific versions if needed.
It might be possible to clean up some redundancy from the configure
lines, especially in LDFLAGS
settings. If you know better, feel free to make those edits.
Build steps¶
Install build dependencies¶
These are mostly for Ruby and should be enough to cover everything else.
sudo apt install bison build-essential file libffi-dev libgdbm-compat-dev libgdbm-dev libgmp-dev libncurses-dev libedit-dev libyaml-dev netbase pkg-config procps zlib1g-dev
OpenSSL 1.1¶
https://www.openssl.org/source/openssl-1.1.1v.tar.gz
LDFLAGS="-Wl,-rpath=/opt/arvados/lib" ./config --prefix=/opt/arvados --openssldir=/opt/arvados/etc/ssl
make -j6 test
make install
Ruby 2.7 (including openssl gem)¶
https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.7.tar.gz
LDFLAGS="-Wl,-rpath=/opt/arvados/lib" ./configure --disable-install-static-library --enable-shared --disable-install-doc --prefix=/opt/arvados --with-openssl-dir=/opt/arvados
make -j6
make install
gem install bundler --no-document
libpq/PostgreSQL¶
https://ftp.postgresql.org/pub/source/v15.4/postgresql-15.4.tar.bz2
What you should probably do: Complete install¶
arvados-server boot
will call pg_config --bindir
and expect to be able to find initdb
and postgres
under it. Because of that, you should probably just install the whole thing.
LDFLAGS="-Wl,-rpath=/opt/arvados/lib" ./configure --prefix=/opt/arvados --with-ssl=openssl --with-includes=/opt/arvados/include --with-libraries=/opt/arvados/lib
make -j6
make install
What I actually did: Fragile partial install¶
You can download the entire PostgreSQL source, configure it, then build and install just libpq and friends. Specific make tasks based on this SO answer.
pg_config --bindir
will return /opt/arvados/bin
. To make arvados-server boot
happy, we symlink the binaries it needs from Debian's package.
LDFLAGS="-Wl,-rpath=/opt/arvados/lib" ./configure --prefix=/opt/arvados --with-ssl=openssl --with-includes=/opt/arvados/include --with-libraries=/opt/arvados/lib
make -j6 -C src/backend generated-headers
make -j6 -C src/include install
make -j6 -C src/interfaces/libpq install
make -j6 -C src/bin/pg_config install
ln -st /opt/arvados/bin /usr/lib/postgresql/15/bin/initdb /usr/lib/postgresql/15/bin/postgres
pg gem 1.1.4¶
Run these steps in the arvados/services/api
directory. bundle install
will install all of the API server's dependencies, not just the pg
gem, but you were gonna do that sooner or later anyway so might as well now.
cd .../arvados/services/api
bundle config build.pg --with-opt-dir=/opt/arvados --with-pg-config=/opt/arvados/bin/pg_config
bundle install
Rest of the Arvados install¶
From here, you should be able to run arvados-server install
as described on the hacking prerequisites dev environment. You can test that everything worked by running the API server tests (e.g., run-tests.sh
→ test services/api
).
Updated by Brett Smith 4 months ago · 7 revisions