QuantyKube

ADR-039 kubectl apply reaches a node through a privileged pod that starts a systemd unit

All pages

ADR-039, from docs/DECISIONS.md in the QuantyKube repository.

Status In force

What kubectl apply can start is a pod. To run install store or install runtime (ADR-035, ADR-038) on a node, the installer starts a pod there from the public image (ADR-037) with the host's PID namespace, privileged, and the host's / mounted. quantykube host in it copies the binary to the host's /usr/local/bin, enters the namespaces of the host's PID 1 and runs the command with systemd-run as a unit of its own, waits for it, then shows its journal and passes on its result. The unit writes to the journal, not to the pod: given the pod's output pipe, it held the pod open, and containerd, asked by the unit to stop that pod, waited for the pipe until the unit gave up.

The unit is what makes this work: moving a node's runtime stops every pod of the old runtime, the installer's own pod among them, and a command running inside that pod would die halfway. As a unit it runs to its end, or to its own undo, whatever happens to the pod.

The cost: the installer's pod is root on the node, which is what ADR-009 says it has to be, and it is visible as such in the manifest. It runs only while a step runs, not as a DaemonSet that stays.