Praecise Engine v0.1.0 Read the docs

Documentation · Using it

What the layer provides

Speculation policy

praecise_runtime::spec_policy decides whether to speculate given model shape and load, and returns a reason with every plan — a policy that cannot explain itself gets switched off. It refuses where speculation does not pay: MoE under concurrency, a loaded engine, a request too short to amortise a draft.

Self-speculation with no draft model

praecise_runtime::ngram drafts from token history alone — no second model, no GPU memory. It is deliberately not tied to a request, so an agent loop or a code-editing session reuses patterns across turns, which no single-request runtime is positioned to do.

This is the technique behind the 3.5x above.

Drafter catalogue with licence enforcement

praecise_runtime::drafters records published drafter checkpoints with their licences, which are frequently not the licence of the model they draft for. Qwen3.8-27B-DFlash2 is Apache-2.0; GLM-5.3-Flash-DFlash2 is CC-BY-NC-ND. Both sit on the same hub and a checkpoint path is just a string.

Only permissive checkpoints are offered. The rest are kept in the catalogue so that naming one explains why it was refused, and a checkpoint whose card states no licence at all is refused too — absent terms grant nothing.

It also preflights four documented ways a drafter deployment silently underperforms: a block size the checkpoint was not trained at, a model runner that quietly downgrades a newer drafter to an older one, grammar conflicts, and partial-vocabulary sampling. Each produces a smaller speedup rather than an error, which is why they are worth checking for.

Request shaping

praecise_runtime::shaping makes a batch cheap before it reaches a backend. The sharpest item: all_greedy and no_penalties are batch-wide booleans, so one request using a penalty drags the entire batch onto the expensive sampling path — including every request that asked for none. Penalties cost 6–47 ms/step at batch 4; the sampler itself costs ~43 µs.

See docs/request-shaping.md.

Edit this page on GitHub · Read it as Markdown

    Type to search. Arrow keys move, Enter opens.