Praecise Engine v0.1.0 Read the docs

Documentation · Overview

What this layer is for, measured

Decode is memory-bandwidth-bound. On a DGX Spark (GB10) it already runs at ~80% of the 273 GB/s peak — the highest fraction of peak of any comparable machine (M4 Max 58%, M3 Ultra 44%). See docs/gb10-bandwidth-ceiling.md.

That constrains what a layer above a backend can honestly claim: everything host-side shares the ~20% that is not already saturated. So this engine concentrates on decisions the backend cannot make for itself, and on avoiding losses rather than manufacturing wins.

Measured on an RTX 5070 Ti with qwen3.5-0.8b and n-gram self-speculation:

prompt tok/s drafted vs baseline
SQL 226.3 171 3.5x
code 94.5 73 1.5x
number list 63.8 0 —
JSON 65.4 0 —
prose 62–65 0 —

Speculation is worth 1.5–3.5x on structured text and nothing on prose. The decision of whether to speculate is where the value is; see docs/measured-acceptance.md.

Edit this page on GitHub · Read it as Markdown

    Type to search. Arrow keys move, Enter opens.