Feature #5540

[Crunch] Contain all job/task containers in a memory-limiting cgroup

Added by Tom Clegg almost 6 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
-
Start date:
03/23/2015
Due date:
% Done:

0%

Estimated time:
Story points:
1.0

Description

Even with #5425, it could improve worker node stability -- and allow some memory oversubscription for task processes -- to have all docker containers run in one memory cgroup with an RSS limit (and run in their own cgroups, of course). The docker daemon itself could be in this outer group too. Presumably the outer cgroup doesn't need to be a full-blown docker container, though -- maybe something like this:

cgcreate -g memory:docker
cgset -r memory.limit_in_bytes=4G docker
cgexec -g memory:docker docker.io --daemon

I tried this, and it affected the docker daemon itself, but not the containers it created. Perhaps there is a way to make docker inherit memory limits from the outer container? My impression is that cgroups are hierarchical, which should make this sort of thing possible.


Related issues

Related to Arvados - Bug #5425: [Crunch] SLURM Node failureResolved03/23/2015

History

#1 Updated by Tom Clegg almost 3 years ago

  • Status changed from New to Closed

obsolete. crunch2.

#2 Updated by Tom Clegg almost 3 years ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF