Why speculative decoding
Single-stream (batch-1) decode is memory-bandwidth-bound: each token streams the
model's active weights from memory once, so throughput is bounded by
bandwidth ÷ bytes-per-token. The software levers are to move fewer bytes per
token (quantization, sparsity) or to emit more tokens per weight-read
(speculative decoding). Praecise Engine focuses on the latter — techniques that
are largely backend-independent, which is why they belong in a layer rather than
a fork.
For what moves the ceiling itself rather than working within it, see docs/what-actually-helps.md. The short version: KV-cache quantization is worth +48% and is off by default.