What the layer provides
Speculation policy
praecise_runtime::spec_policy decides whether to speculate given model shape
and load, and returns a reason with every plan — a policy that cannot explain
itself gets switched off. It refuses where speculation does not pay: MoE under
concurrency, a loaded engine, a request too short to amortise a draft.
Self-speculation with no draft model
praecise_runtime::ngram drafts from token history alone — no second model, no
GPU memory. It is deliberately not tied to a request, so an agent loop or a
code-editing session reuses patterns across turns, which no single-request
runtime is positioned to do.
This is the technique behind the 3.5x above.
Drafter catalogue with licence enforcement
praecise_runtime::drafters records published drafter checkpoints with their
licences, which are frequently not the licence of the model they draft for.
Qwen3.8-27B-DFlash2 is Apache-2.0; GLM-5.3-Flash-DFlash2 is CC-BY-NC-ND. Both
sit on the same hub and a checkpoint path is just a string.
Only permissive checkpoints are offered. The rest are kept in the catalogue so that naming one explains why it was refused, and a checkpoint whose card states no licence at all is refused too — absent terms grant nothing.
It also preflights four documented ways a drafter deployment silently underperforms: a block size the checkpoint was not trained at, a model runner that quietly downgrades a newer drafter to an older one, grammar conflicts, and partial-vocabulary sampling. Each produces a smaller speedup rather than an error, which is why they are worth checking for.
Request shaping
praecise_runtime::shaping makes a batch cheap before it reaches a backend.
The sharpest item: all_greedy and no_penalties are batch-wide booleans, so
one request using a penalty drags the entire batch onto the expensive sampling
path — including every request that asked for none. Penalties cost 6–47 ms/step
at batch 4; the sampler itself costs ~43 µs.