Arvados

Here's bundler's explanation on why dependabot suggested to upgrade everything:

$ bundle add passenger --version 6.0.26
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Could not find compatible versions

    Because passenger >= 6.0.24 depends on rackup >= 2.0.0
      and rackup >= 2.0.0 depends on rack >= 3,
      passenger >= 6.0.24 requires rack >= 3.
(1) So, because actionpack >= 7.0.5, < 7.1.0.beta1 depends on rack >= 2.2.4, < 3.A,
      passenger >= 6.0.24 is incompatible with actionpack >= 7.0.5, < 7.1.0.beta1.

    Because rails >= 7.0.4.3, < 7.0.5 depends on actionmailer = 7.0.4.3
      and rails >= 7.0.4.2, < 7.0.4.3 depends on actionmailer = 7.0.4.2,
      rails >= 7.0.4.2, < 7.0.5 requires actionmailer = 7.0.4.2 OR = 7.0.4.3.
    And because rails >= 7.0.4.1, < 7.0.4.2 depends on actionmailer = 7.0.4.1
      and rails >= 7.0.4, < 7.0.4.1 depends on actionmailer = 7.0.4,
      rails >= 7.0.4, < 7.0.5 requires actionmailer = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR = 7.0.4.3.
    And because rails >= 7.0.3.1, < 7.0.4 depends on actionmailer = 7.0.3.1
      and rails >= 7.0.3, < 7.0.3.1 depends on actionmailer = 7.0.3,
rails >= 7.0.3, < 7.0.5 requires actionmailer = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3.
    And because rails >= 7.0.2.4, < 7.0.3 depends on actionmailer = 7.0.2.4
      and rails >= 7.0.2.3, < 7.0.2.4 depends on actionmailer = 7.0.2.3,
rails >= 7.0.2.3, < 7.0.5 requires actionmailer = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR
= 7.0.4.1 OR = 7.0.4.2 OR = 7.0.4.3.
    And because rails >= 7.0.2.2, < 7.0.2.3 depends on actionmailer = 7.0.2.2
      and rails >= 7.0.2.1, < 7.0.2.2 depends on actionmailer = 7.0.2.1,
rails >= 7.0.2.1, < 7.0.5 requires actionmailer = 7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3
OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR = 7.0.4.3.
    And because rails >= 7.0.2, < 7.0.2.1 depends on actionmailer = 7.0.2
      and rails >= 7.0.1, < 7.0.2 depends on actionmailer = 7.0.1,
rails >= 7.0.1, < 7.0.5 requires actionmailer = 7.0.1 OR = 7.0.2 OR = 7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR =
7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR = 7.0.4.3.
    And because rails >= 7.0.0, < 7.0.1 depends on actionmailer = 7.0.0
      and rails >= 7.0.8.7, < 7.1.0.beta1 depends on actionpack = 7.0.8.7,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.8.7, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.8.7.
    And because rails >= 7.0.8.6, < 7.0.8.7 depends on actionpack = 7.0.8.6
      and rails >= 7.0.8.5, < 7.0.8.6 depends on actionpack = 7.0.8.5,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.8.5, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.8.4, < 7.0.8.5 depends on actionpack = 7.0.8.4
      and rails >= 7.0.8.3, < 7.0.8.4 depends on actionpack = 7.0.8.3,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.8.3, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.8.3 OR = 7.0.8.4 OR = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.8.2, < 7.0.8.3 depends on actionpack = 7.0.8.2
      and rails >= 7.0.8.1, < 7.0.8.2 depends on actionpack = 7.0.8.1,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.8.1, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.8.1 OR = 7.0.8.2 OR = 7.0.8.3 OR = 7.0.8.4 OR = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.8, < 7.0.8.1 depends on actionpack = 7.0.8
      and rails >= 7.0.7.2, < 7.0.8 depends on actionpack = 7.0.7.2,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.7.2, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.7.2 OR = 7.0.8 OR = 7.0.8.1 OR = 7.0.8.2 OR = 7.0.8.3 OR = 7.0.8.4 OR = 7.0.8.5 OR =
7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.7.1, < 7.0.7.2 depends on actionpack = 7.0.7.1
      and rails >= 7.0.7, < 7.0.7.1 depends on actionpack = 7.0.7,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.7, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.7 OR = 7.0.7.1 OR = 7.0.7.2 OR = 7.0.8 OR = 7.0.8.1 OR = 7.0.8.2 OR = 7.0.8.3 OR =
7.0.8.4 OR = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.6, < 7.0.7 depends on actionpack = 7.0.6
      and rails >= 7.0.5.1, < 7.0.6 depends on actionpack = 7.0.5.1,
rails >= 7.0.0, < 7.0.5 OR >= 7.0.5.1, < 7.1.0.beta1 requires actionmailer = 7.0.0 OR = 7.0.1 OR = 7.0.2 OR =
7.0.2.1 OR = 7.0.2.2 OR = 7.0.2.3 OR = 7.0.2.4 OR = 7.0.3 OR = 7.0.3.1 OR = 7.0.4 OR = 7.0.4.1 OR = 7.0.4.2 OR =
7.0.4.3 or actionpack = 7.0.5.1 OR = 7.0.6 OR = 7.0.7 OR = 7.0.7.1 OR = 7.0.7.2 OR = 7.0.8 OR = 7.0.8.1 OR =
7.0.8.2 OR = 7.0.8.3 OR = 7.0.8.4 OR = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because rails >= 7.0.5, < 7.0.5.1 depends on actionpack = 7.0.5
      and every version of actionmailer depends on actionpack = 7.0.8.7,
rails >= 7.0.0, < 7.1.0.beta1 requires actionpack = 7.0.5 OR = 7.0.5.1 OR = 7.0.6 OR = 7.0.7 OR = 7.0.7.1 OR
= 7.0.7.2 OR = 7.0.8 OR = 7.0.8.1 OR = 7.0.8.2 OR = 7.0.8.3 OR = 7.0.8.4 OR = 7.0.8.5 OR = 7.0.8.6 OR = 7.0.8.7.
    And because passenger >= 6.0.24 is incompatible with actionpack >= 7.0.5, < 7.1.0.beta1 (1),
      passenger >= 6.0.24 is incompatible with rails >= 7.0.0, < 7.1.0.beta1.
    So, because Gemfile depends on rails ~> 7.0.0
      and Gemfile depends on passenger = 6.0.26,
      version solving has failed.

There's not much description about the CVE, but given that RailsAPI only talks with arvados-controller, this might not be a big concern in our case and we could plan the passenger upgrade with time along with the rest of the dependent components.

Peter Amstutz wrote:

Ignore the dependabot request and only upgrade the passenger gem.

I don't know why we thought this would work when the linked report specifically says, "These dependencies needed to be updated together."

The definition of find_or_create_by in Rails 7.0 is:

find_by(attributes) || create(attributes, &block)

The definition of find_or_create_by in Rails 7.1 is:

find_by(attributes) || create_or_find_by(attributes, &block)

In other words, it tries to do some basic race condition handling for you. This is noted in the new paragraph in the docs:

If creation failed because of a unique constraint, this method will assume it encountered a race condition and will try finding the record once more. If somehow the second find still does not find a record because a concurrent DELETE happened, it will then raise an ActiveRecord::RecordNotFound exception.

This is causing the ApiClientAuthorization.find_or_create_by near the bottom of ApiClientAuthorization::validate to not work as expected, so our own race condition handling doesn't work, so RemoteUsersTest#test_authenticate_with_remote_token_with_secret_part_identical_to_previously_cached_token fails.

I have tried doing the seemingly-obvious thing of replacing the call with an inlined version of the old definition. That doesn't get the test passing, but I might be doing something wrong.

Related reading: implementing pull request, bug report filed after

Our upgrade to Rails 7.2 is affected by issue 52643. groups contents calls can generate a query like this:

Group Load (0.7ms)  SELECT "groups".* FROM "groups" WHERE (NOT (((groups.owner_uuid IN (SELECT group_uuid FROM trashed_groups WHERE trash_at <= statement_timestamp())) OR groups.trash_at <= statement_timestamp() IS TRUE))) AND (groups.uuid is $1) ORDER BY "groups"."id" ASC LIMIT $2 OFFSET $3  [[nil, nil], ["LIMIT", 1], ["OFFSET", 0]]

Querying (groups.uuid is $1) with the nil value no longer works.

lol we can't do any of this

Because rails >= 7.2.0.beta1, < 8.0.0.beta1 depends on Ruby >= 3.1.0
  and Gemfile depends on rails ~> 7.2.0,
  Ruby >= 3.1.0 is required.
So, because current Ruby version is = 2.7.4,
  version solving has failed.

LGTM, thanks.

Rails 7.1 introduced a dependency on the psych gem, which wants to build against libyaml. On Red Hat-based distributions, libyaml-devel is available from the "PowerTools" repository, which is official but not enabled by default. We need to enable this in places like our package test Dockerfile and probably add an upgrade note.

22608-enable-powertools @ 40d48732e52689a425adad966b5291f34d6b3b46

This branch should fix the rocky8 package build that has been failing since the initial branch was merged.

• All agreed upon points are implemented / addressed.
  • Fixes the broken rocky8 package build by enabling the necessary repository in the test image. Adds a corresponding upgrade note.
• Anything not implemented (discovered or discussed during work) has a follow-up story.
  • N/A
• Code is tested and passing, both automated and manual, what manual testing was done is described
  • Manually built the new rocky8 test image and confirmed you could run microdnf --assumeyes install libyaml-devel in it. That's the most testing I can actually do besides just going ahead and building new packages, which is always dicey to do from a branch.
• Documentation has been updated.
  • Yes
• Behaves appropriately at the intended scale (describe intended scale).
  • No change in scale
• Considered backwards and forwards compatibility issues between client and server.
  • No change
• Follows our coding standards and GUI style guidelines.
  • N/A

22608-enable-powertools LGTM, thanks.