QuantyKube

What is built

All pages

The sections Read this first and Status come from README.md in the QuantyKube repository.

On this page

QuantyKube is early, and says so in the first section of its README. Here are that section and the status table, as they stand on main. Everything marked built is exercised in CI against the real tools it names; nothing has run in production yet. The benchmarks have the numbers, including where QuantyKube is slower than etcd.

Read this first

QuantyKube is not a general etcd replacement. It implements the part of the etcd v3 API that the Kubernetes API server uses, on one node or on several joined by Raft with a membership fixed at start. Other etcd clients may call things that are not there (ADR-001); kine makes the same choice.

It is early. Everything marked built below is exercised in CI against the real etcdctl, an unmodified kube-apiserver, a control plane set up by kubeadm and one by k3s (scripts/e2e.sh, scripts/kubeadm.sh), and against SIGKILL under load (tests/crash.rs). Nothing has run in production yet.

The second half is a node runtime, the program the kubelet starts containers through. In CI a k3s node runs on both halves at once, its API server on the store and its kubelet on the runtime, and an unchanged kubelet starts a pod with a ConfigMap volume, a service account token, CPU and memory limits, a pod network, an exec liveness probe, a user other than root and dropped capabilities (scripts/cluster.sh).

Status

PartState
etcd v3 subset used by kube-apiserver, Unix socket (docs/RPCS.md)built
Watch, with replay from a past revision and progress notificationsbuilt
Leases, with expirybuilt
Group commit: concurrent writes share fsyncs (ADR-012)built
History beyond compaction: history, get, restorebuilt
kubectl quanty pluginbuilt
Benchmarks against etcd (docs/BENCHMARKS.md)measured
Retention: long history in the log, short lived snapshots (ADR-014)built, soak tested
The store over a network with TLS and client certificates (ADR-031)built, unreviewed TLS
Node runtime: layer unpacking and a hardened sandbox (docs/RUNTIME.md)built, measured against runc
A k3s node on the store and the runtime together (scripts/cluster.sh)runs in CI
Node runtime: the CRI service, pods and containers with crictlbuilt with volumes, limits, stats, exec probes, CNI, pulls over HTTPS; pod start measured against containerd
Several store nodes (Raft): a simulator that finds planted bugs, three nodes under kube-apiserver with the leader killed, log compaction and snapshots (ADR-032 to 034)built, runs in CI
kubeadm with the store as its etcd (scripts/kubeadm.sh)runs in CI
Moving a kubeadm cluster from etcd and back: migrate, install store, undo store, every object at its resourceVersion (ADR-035)runs in CI
A kubeadm cluster with every pod on the runtime, kube-proxy and the control plane included (ADR-036)runs in CI
One kubectl apply (deploy/quantykube.yaml): an operator moves the store and labelled nodes onto QuantyKube and back (ADR-038 to 040)runs in CI on one node