QuantyKube

Getting started

All pages
On this page

QuantyKube is early. Run it where losing the cluster costs nothing: a test machine, a throwaway k3s, a laptop. Everything below comes from the repository's README and the commands its CI runs.

What you need

  • Rust 1.89 or newer, to build it.
  • For the store: the machine the Kubernetes API server runs on. The two talk over a Unix socket (ADR-004), or over TLS with client certificates when the store runs elsewhere (ADR-031).
  • For the node runtime: Linux with cgroup v2 and the standard CNI plugins.

Build it

git clone https://github.com/QuantyRoot/QuantyKube
cd QuantyKube
cargo build --release

The build fetches no third-party crates, only QuantyRoot's own: QuantyCore for protobuf, HTTP/2, gRPC and TLS, and QuantyDB's engine for storage (ADR-003). One binary, target/release/quantykube, holds every command; run it without arguments for the full usage.

Start the store

target/release/quantykube serve --data /var/lib/quantykube/db --socket /run/quantykube.sock

--data is the store's file, --socket where it listens. History is kept for an hour by default; --retain takes another window, written as 90s, 15m, 2h or 7d (ADR-011).

Put an API server on it

An unmodified kube-apiserver takes the socket as its etcd:

kube-apiserver --etcd-servers=unix:///run/quantykube.sock ...

With k3s, the same socket goes to --datastore-endpoint=unix:///run/quantykube.sock. Both run in CI. To look at the raw keys, etcdctl works against the socket too:

etcdctl --endpoints=unix:///run/quantykube.sock get /registry/namespaces --prefix --keys-only

Delete something and bring it back

The binary is also the kubectl plugin. Put it on the PATH as kubectl-quanty:

ln -s "$PWD/target/release/quantykube" /usr/local/bin/kubectl-quanty

It finds the store at $QUANTYKUBE_SOCKET, or /run/quantykube.sock when that is not set. Then make a ConfigMap, change it, and lose it:

kubectl create configmap important --from-literal=password=hunter2
kubectl patch configmap important -p '{"data":{"password":"correct horse"}}'
kubectl delete configmap important
kubectl quanty history configmap/default/important

The history lists the create, the update and the delete, each with its revision and time. Put the object back as it was after the update:

kubectl quanty restore --rev <rev> configmap/default/important
kubectl get configmap important -o jsonpath='{.data.password}'

--ago 5m instead of --rev picks the state five minutes ago. A restore is an ordinary write with a new revision, so controllers and watches see the object return. Leases are not restored: a key that had one comes back without it. A revision older than the retention window is refused with OUT_OF_RANGE.

From a laptop that has only a kubeconfig, the same history is served through the Kubernetes API, with RBAC deciding who may restore. The History API page says how to register it.

Start the node runtime

On a node with cgroup v2 and the CNI plugins:

target/release/quantykube runtime --root /var/lib/quantykube-runtime \
  --socket /run/quantykube-cri.sock

Start the kubelet with --container-runtime-endpoint=unix:///run/quantykube-cri.sock and --cgroup-driver=cgroupfs, the only driver the runtime supports (ADR-020). One process serves every pod: there is no shim and no pause container.

Images are pulled from their registries, or put there with quantykube import. The TLS under those pulls is QuantyCore's own and has not been reviewed by anyone who did not write it; until it has, treat a pull as no safer than the network it crosses.

Where to go next

  • Move a cluster onto QuantyKube with one kubectl apply.
  • Demo commands walk through all of the above with k3s, including a SIGKILL of the store in the middle.
  • Status says what is built and what is not.