Support #21700
closedRemove RVM support from our build process and packages
Description
Broken out of #21524#note-19
In general I believe our attempt to "standardize" on Bundler 2.2.19 is a farce. We install it a lot but it's only setting a minimum version. Running the bundler
command will pick up the latest version, which is >2.2.19 on every distro after debian11. And when it's >=2.3.0, Bundler will standardize itself on the version in Gemfile.lock, which is currently 2.4.19. And all this is true both when we build packages and when they get installed. And note when we build packages what matters is the version of Bundler that came with the Ruby we installed with RVM, not the distro Ruby.
I would like to expand the scope of this ticket to be a bunch of Ruby build cleanup:
- Stop using RVM in our package build Dockerfiles, just use the distro Ruby to better match the install environment.
- Make our "standard" Bundler install
gem install --conservative --version '>=2.4.0' bundler
. As long as we have some version matching that then I think we can rely on Bundler's own lock behavior to take us the rest of the way. - Add an API server test that Bundler is locked to version 2.4.X, because 2.5.0 drops support for Ruby 2.7.
- Standardize API server on Bundler 2.4.22, because that's the last 2.4.X release so why not.
I'll be honest, I have no idea whether or not this will actually fix anything for test-provision-ubuntu2004, but I still want to do it because:
- At this point I've read enough to be confident this is the right approach.
- All the RVM stuff in our package builds is historical cruft that isn't doing anything for us anymore. Removing it would be easier to maintain long-term and also speed up package builds.
- It would make this stuff easier to debug because then instead of trying to "standardize" our Bundler version in twenty different places we can trust that the version in Gemfile.lock is the one that actually matters.
Updated by Brett Smith 8 months ago
21700-remove-rvm @ 24e3397d88f64e7f71b4378ea93cb3e60a1b7f5c
developer-run-tests: #4181 (this ran on the previous commit but the latest commit is in the "obvious trivial fix" category so I hope this is good enough)
build-packages-multijob: #4174
The actual removing RVM part of this was relatively simple. What got complicated was coaxing Bundler to work with a wider variety of Ruby versions and setups (e.g., different sets of gems already installed across different distros). On the whole I think these were good bugs to tackle: I think the arvados-api-server package is more likely to work on more systems now. And now I think our build+deploy process is better established to write into a Docker image later. But admittedly I didn't plan to have to tackle so much when I started work on the ticket.
- All agreed upon points are implemented / addressed.
- Yes
- Anything not implemented (discovered or discussed during work) has a follow-up story.
- N/A
- Code is tested and passing, both automated and manual, what manual testing was done is described
- See above
- Documentation has been updated.
- All references to RVM have been removed. Not sure if this warrants an upgrade note? If it does, it would basically say: "If you previously installed with RVM, you'll need to make sure the Ruby you want is the first
ruby
on the system$PATH
. Our package won't find it automatically anymore." I am skeptical anybody actually installed that way but this would cover our bases I guess.
- All references to RVM have been removed. Not sure if this warrants an upgrade note? If it does, it would basically say: "If you previously installed with RVM, you'll need to make sure the Ruby you want is the first
- Behaves appropriately at the intended scale (describe intended scale).
- No change in scale
- Considered backwards and forwards compatibility issues between client and server.
- No change
- Follows our coding standards and GUI style guidelines.
- Mostly N/A since we don't have a shell style guide, but yes to the extent it makes sense
Updated by Peter Amstutz 8 months ago
- Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Updated by Tom Clegg 8 months ago
Now that we don't use $bundler
anywhere, I suppose this gem🙄 can also be deleted from run-tests.sh:
if test -d /var/lib/arvados-arvbox/ ; then # Inside arvbox, use bundler-installed binstubs. The # system bundler and rail's own bin/bundle refuse to work. # I don't know why. bundle=binstubs/bundle fi
The bit where we use awk to implement "bundle package --no-install" ourselves is so awful that I tried to research what we might be doing wrong, and spent some time trying to trick bundler into doing the thing the docs say it does, and -- success! -- now I hate bundler more than I hate the awk script.
So, LGTM
Updated by Brett Smith 8 months ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|fb19cf980838b541a50af8a27ccf8058409b88e4.
Updated by Brett Smith 8 months ago
Tom Clegg wrote in #note-3:
The bit where we use awk to implement "bundle package --no-install" ourselves is so awful that I tried to research what we might be doing wrong, and spent some time trying to trick bundler into doing the thing the docs say it does, and -- success! -- now I hate bundler more than I hate the awk script.
Yup, I also went through these same seven stages of grief. I didn't want to write it. I tried everything you did. I eventually decided that I hate the awk script, and it's the least worst option.
Updated by Brett Smith 7 months ago
- Related to Idea #21522: Stop using RVM in package-build-dockerfiles added
Updated by Brett Smith 7 months ago
- Related to Bug #21906: Seemingly duplicate Gem directories after installing arvados-api-server added