QuantyKube

ADR-020 cgroups follow the cgroupfs driver

All pages

ADR-020, from docs/DECISIONS.md in the QuantyKube repository.

Status In force

The kubelet creates a cgroup per pod and sends its path as the pod's cgroup parent; the runtime makes one cgroup per container inside it, named by the container's id, and writes the limits the kubelet sent: memory.max, cpu.max, cpu.weight converted from shares the way the kubelet converts them, cpusets, huge pages, and the unified files as they are. The child is in its cgroup before it runs anything (ADR-013), gets a cgroup namespace rooted there, and sees it read only at /sys/fs/cgroup, where Go and the JVM look for their limits. A container the kernel killed for memory is reported as OOMKilled, read from memory.events. ContainerStats and PodSandboxStats come from the same files, and UpdateContainerResources rewrites them in place, which is how the kubelet resizes a running pod.

Only the cgroupfs driver is supported. The systemd driver names slices instead of paths and needs a D-Bus client to ask systemd for them; a slice name is refused. On a host without cgroup v2, a container with limits is refused rather than run without them.

RuntimeConfig tells the kubelet so, and a kubelet that asks, as current ones do, needs no flag.

The cost: a kubelet that does not ask must run with --cgroup-driver=cgroupfs, which most distributions no longer default to. A pod's own cgroup and its pids limit are the kubelet's, and stats for a pod are the sum of its containers, without network.