Project

General

Profile

Actions

Support #21700

closed

Remove RVM support from our build process and packages

Added by Brett Smith 13 days ago. Updated about 6 hours ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Deployment
Due date:
Story points:
-
Release:
Release relationship:
Auto

Description

Broken out of #21524#note-19

In general I believe our attempt to "standardize" on Bundler 2.2.19 is a farce. We install it a lot but it's only setting a minimum version. Running the bundler command will pick up the latest version, which is >2.2.19 on every distro after debian11. And when it's >=2.3.0, Bundler will standardize itself on the version in Gemfile.lock, which is currently 2.4.19. And all this is true both when we build packages and when they get installed. And note when we build packages what matters is the version of Bundler that came with the Ruby we installed with RVM, not the distro Ruby.

I would like to expand the scope of this ticket to be a bunch of Ruby build cleanup:

  • Stop using RVM in our package build Dockerfiles, just use the distro Ruby to better match the install environment.
  • Make our "standard" Bundler install gem install --conservative --version '>=2.4.0' bundler. As long as we have some version matching that then I think we can rely on Bundler's own lock behavior to take us the rest of the way.
  • Add an API server test that Bundler is locked to version 2.4.X, because 2.5.0 drops support for Ruby 2.7.
  • Standardize API server on Bundler 2.4.22, because that's the last 2.4.X release so why not.

I'll be honest, I have no idea whether or not this will actually fix anything for test-provision-ubuntu2004, but I still want to do it because:

  • At this point I've read enough to be confident this is the right approach.
  • All the RVM stuff in our package builds is historical cruft that isn't doing anything for us anymore. Removing it would be easier to maintain long-term and also speed up package builds.
  • It would make this stuff easier to debug because then instead of trying to "standardize" our Bundler version in twenty different places we can trust that the version in Gemfile.lock is the one that actually matters.

Subtasks 1 (0 open1 closed)

Task #21715: Review 21700-remove-rvmResolvedBrett Smith04/26/2024Actions
Actions #1

Updated by Brett Smith 9 days ago

21700-remove-rvm @ 24e3397d88f64e7f71b4378ea93cb3e60a1b7f5c

developer-run-tests: #4181 (this ran on the previous commit but the latest commit is in the "obvious trivial fix" category so I hope this is good enough)

build-packages-multijob: #4174

The actual removing RVM part of this was relatively simple. What got complicated was coaxing Bundler to work with a wider variety of Ruby versions and setups (e.g., different sets of gems already installed across different distros). On the whole I think these were good bugs to tackle: I think the arvados-api-server package is more likely to work on more systems now. And now I think our build+deploy process is better established to write into a Docker image later. But admittedly I didn't plan to have to tackle so much when I started work on the ticket.

  • All agreed upon points are implemented / addressed.
    • Yes
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • N/A
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • See above
  • Documentation has been updated.
    • All references to RVM have been removed. Not sure if this warrants an upgrade note? If it does, it would basically say: "If you previously installed with RVM, you'll need to make sure the Ruby you want is the first ruby on the system $PATH. Our package won't find it automatically anymore." I am skeptical anybody actually installed that way but this would cover our bases I guess.
  • Behaves appropriately at the intended scale (describe intended scale).
    • No change in scale
  • Considered backwards and forwards compatibility issues between client and server.
    • No change
  • Follows our coding standards and GUI style guidelines.
    • Mostly N/A since we don't have a shell style guide, but yes to the extent it makes sense
Actions #2

Updated by Peter Amstutz 8 days ago

  • Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Actions #3

Updated by Tom Clegg 7 days ago

Now that we don't use $bundler anywhere, I suppose this gem🙄 can also be deleted from run-tests.sh:

        if test -d /var/lib/arvados-arvbox/ ; then
            # Inside arvbox, use bundler-installed binstubs.  The                                                                                                         
            # system bundler and rail's own bin/bundle refuse to work.                                                                                                    
            # I don't know why.                                                                                                                                           
            bundle=binstubs/bundle
        fi

The bit where we use awk to implement "bundle package --no-install" ourselves is so awful that I tried to research what we might be doing wrong, and spent some time trying to trick bundler into doing the thing the docs say it does, and -- success! -- now I hate bundler more than I hate the awk script.

So, LGTM

Actions #4

Updated by Brett Smith 6 days ago

  • Status changed from In Progress to Resolved
Actions #5

Updated by Brett Smith 6 days ago

Tom Clegg wrote in #note-3:

The bit where we use awk to implement "bundle package --no-install" ourselves is so awful that I tried to research what we might be doing wrong, and spent some time trying to trick bundler into doing the thing the docs say it does, and -- success! -- now I hate bundler more than I hate the awk script.

Yup, I also went through these same seven stages of grief. I didn't want to write it. I tried everything you did. I eventually decided that I hate the awk script, and it's the least worst option.

Actions #6

Updated by Peter Amstutz about 6 hours ago

  • Release set to 70
Actions

Also available in: Atom PDF