Project

General

Profile

Actions

Feature #5540

closed

[Crunch] Contain all job/task containers in a memory-limiting cgroup

Added by Tom Clegg about 9 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
-
Story points:
1.0

Description

Even with #5425, it could improve worker node stability -- and allow some memory oversubscription for task processes -- to have all docker containers run in one memory cgroup with an RSS limit (and run in their own cgroups, of course). The docker daemon itself could be in this outer group too. Presumably the outer cgroup doesn't need to be a full-blown docker container, though -- maybe something like this:

cgcreate -g memory:docker
cgset -r memory.limit_in_bytes=4G docker
cgexec -g memory:docker docker.io --daemon

I tried this, and it affected the docker daemon itself, but not the containers it created. Perhaps there is a way to make docker inherit memory limits from the outer container? My impression is that cgroups are hierarchical, which should make this sort of thing possible.


Related issues

Related to Arvados - Bug #5425: [Crunch] SLURM Node failureResolvedPeter Amstutz03/23/2015Actions
Actions #1

Updated by Tom Clegg over 6 years ago

  • Status changed from New to Closed

obsolete. crunch2.

Actions #2

Updated by Tom Clegg over 6 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF