QuantyKube

ADR-036 The runtime runs what a kubeadm node needs: host namespaces, added capabilities, privileged

All pages

ADR-036, from docs/DECISIONS.md in the QuantyKube repository.

Status In force

On a kubeadm node kube-proxy and the CNI's daemon run as DaemonSets on the host's network, kube-proxy as a privileged container, the CNI's with NET_ADMIN and NET_RAW added. k3s runs both inside its own process, which is why nothing asked for them before. Without them w1 has no Service network, and a pod there cannot resolve a name.

The runtime now does what containerd does. A pod on the host's network gets no network or UTS namespace of its own and no CNI call; one in the host's IPC or PID namespace gets none of those. Capabilities a container adds are added. A privileged container gets every capability the runtime itself holds, no seccomp filter, no no_new_privs, /sys and its cgroup writable, nothing of /proc masked or made read only, and every character and block device of the host in its /dev, with no device filter in its cgroup. Ambient capabilities stay refused.

The cost: a pod that asks for any of this is confined as little as under containerd, which is to say a privileged pod owns the node. What keeps ordinary workloads from asking is the cluster's admission, Pod Security Admission at baseline or stricter for every namespace but kube-system, as on any cluster with containerd. The runtime's own default for pods that ask for nothing is unchanged.

Added on 26 October: as containerd 2 does by default, every pod with a network namespace of its own may bind ports below 1024 and send pings as any user (net.ipv4.ip_unprivileged_port_start=0, ping_group_range open): CoreDNS runs as a user other than root and binds port 53. The host's own settings are not touched. A pod on the host's network keeps the host's. Separately, the sandbox's status now names its namespaces: without that, the kubelet took every pod on the host's network for a changed one and replaced it as soon as it started.