Arvados

So, this is mostly good news. Version 570 of the CUDA drivers switches from this static modules-load file to a udev rule. On a fresh install:

$ apt list --installed 'nvidia*' | cut -d/ -f1 | xargs dpkg -L | grep -e '^/etc/' -e /udev/
/etc/modprobe.d
/etc/modprobe.d/nvidia-modeset.conf
/etc/modprobe.d/nvidia.conf
/usr/lib/udev/rules.d
/usr/lib/udev/rules.d/60-nvidia.rules
/etc/OpenCL
/etc/OpenCL/vendors
/etc/OpenCL/vendors/nvidia.icd
/etc/X11

This means that out of the box, it has the dynamic behavior we want. We can upgrade our pin to version 570, rip out all our code to deal with this dynamism, and still have all the functionality we need. Great!

Except NVIDIA has stopped updating the driver for Debian 11. We need to continue supporting the layout of version 560 as long as we want to continue supporting Debian 11.

In principle we have agreed to drop Debian 11 for the next release. In practice, with all of our own clusters upgraded to Debian 12, I am not aware of any users currently on Debian 11. So, I think it would be okay to go ahead with the upgrade plan described above. But this would pretty officially commit us to dropping Debian 11, so I want to make sure we're on the same page about that before I go ahead.

22792-cuda-570 @ be1277c6a8b468c25d91f07681d320569d3578ba - packer-build-compute-image: #329

• All agreed upon points are implemented / addressed.
  • Yes
• Anything not implemented (discovered or discussed during work) has a follow-up story.
  • N/A
• Code is tested and passing, both automated and manual, what manual testing was done is described
  • Tested the new image at tordo-xvhdp-3o7bnol1rzguqj6 - You can tell this used the new image because it's using crunch-run 3.2.0~dev20250416134717 - note the very recent dev timestamp - and you can tell it successfully used CUDA in the subprocess logs (if it hadn't it wouldn't have run so fast)
• Documentation has been updated.
  • N/A
• Behaves appropriately at the intended scale (describe intended scale).
  • No change
• Considered backwards and forwards compatibility issues between client and server.
  • Peter agreed at kickoff that it's okay to take this approach and begin abandoning Debian 11.
• Follows our coding standards and GUI style guidelines.
  • N/A (no Ansible guide)

As Peter use to say, "the best patch is the red patch"!

LGTM, thanks.