Support crunchstat tracking and memory limits with singularity
Singularity has capability to put the container in a new cgroup and set resource usage limits. Even without applying any limits, this also enables resource usage tracking by crunchstat.
The docs say "the
--apply-cgroups option can only be used with root privileges" but these tests worked as a non-root user:
$ singularity version 3.10.4-dirty $ singularity exec --apply-cgroups /dev/null docker://debian:12 sleep 600 &  60133 $ pstree -up | grep sleep | | `-starter-suid(60133)-+-sleep(60151) $ cat /proc/60133/cgroup 0::/user.slice/user-1000.slice/session-5424.scope $ cat /proc/60151/cgroup 0::/firstname.lastname@example.org/user.slice/singularity-60151.scope $ cat /email@example.com/user.slice/singularity-60151.scope/memory.current 2465792
$ singularity exec --apply-cgroups <(printf '[memory]\n limit = 5000000\n') docker://debian:12 echo ok ok $ singularity exec --apply-cgroups <(printf '[memory]\n limit = 5000\n') docker://debian:12 echo ok Killed
As of #17244 crunch-run does not correctly identify the pid of a process inside the container when telling crunchstat which process/cgroup to monitor (it returns the pid of the singularity executor wrapper instead). This will also need to be fixed in order for crunchstat to work correctly.
Updated by Brett Smith 5 months ago
Tom Clegg wrote:
The docs say "the
--apply-cgroupsoption can only be used with root privileges" but these tests worked as a non-root user:
- It's possible that line was written before user namespaces were widely available/enabled, and has become obsolete since. The timeline kinda works: Singularity 3.0.0 was released October 2018, and Debian got user namespaces in 11, released August 2021.
- But also, if you're going through
starter-suid, don't you have root privileges at some level?