QuantyKube

ADR-014 History lives in the change log, snapshots only briefly

All pages

ADR-014, from docs/DECISIONS.md in the QuantyKube repository.

Status In force

ADR-011 kept every QuantyDB commit for the whole history window. A soak run with a k3s control plane under steady writes showed what that costs with copy on write: about 30 KB of disk per revision, because every commit keeps every page it replaced, and nothing came back until the API server's compaction, five to ten minutes later, let go. Memory grew alongside.

The change log already stores the state before every change (ADR-010). So any revision can be rebuilt from any later snapshot by undoing the changes in between, the same mechanism ADR-012 uses inside a group. That makes old snapshots a speed-up, not the history itself:

  • --snapshots, one minute by default, is how long QuantyDB keeps commits. Reading a revision inside it is a snapshot read.
  • --retain, one hour by default, is how long the change log is kept. Reading a revision inside it but outside the snapshot window starts from the oldest snapshot left and undoes the log back to it.
  • Neither drops anything an etcd client may still read: the log keeps everything after the compaction revision, whatever the windows say.

The commit index now records each commit's time too, so --ago and history work for the whole window after the snapshots are gone. A test writes from eight threads, then reads a scan and every key at every revision against a model, once with all snapshots and once after all but one are collected.

The cost: an old read is no longer one snapshot lookup. It costs a scan of the log from that revision to the oldest snapshot and memory for every key changed in between. The API server reads at old revisions rarely, mostly to resume a paginated list a few seconds old, which is inside the snapshot window. And "the revision is a QuantyDB commit" holds only for the last minute; beyond it, the history is QuantyKube's own log on top of QuantyDB.