Praecise Engine · v0.1.0
Documentation
Praecise Engine is a backend-agnostic inference-acceleration layer for large language models, written in Rust: speculative decoding, hardware-aware tuning and one serving surface over any backend. Agents can read all of it as Markdown: llms.txt, llms-full.txt, or any page with .md in place of the last slash.
Overview
Introduction
A backend-agnostic inference-acceleration layer for large language models.
Design
Praecise Engine is the acceleration layer — not a model server on its own, and not a fork of any single engine. Each backend is an adapter. Optimizations that can be expressed...
What this layer is for, measured
Decode is memory-bandwidth-bound. On a DGX Spark (GB10) it already runs at ~80% of the 273 GB/s peak — the highest fraction of peak of any comparable machine (M4 Max 58%, M3...
Why speculative decoding
Single-stream (batch-1) decode is memory-bandwidth-bound: each token streams the model's active weights from memory once, so throughput is bounded by bandwidth ÷...
What was tried and does not work
praecise_runtime::prax sized the draft block per position. Measured on hardware, block size does not change throughput — a 16x range moves it ~5% non-monotonically, and...
Using it
Depending on the engine
Depend on the crate you need (today, llama-cpp-2 / praecise-runtime). The backend is pulled as a dependency; a host that already builds the same backend shares it rather than...
Selecting a backend
praecise_runtime::Backend names the runtimes the engine knows about and reports, per backend, what is actually possible:
Any vendor's silicon
A backend is not a vendor. llama.cpp reaches every accelerator a ggml backend is built for; vLLM, SGLang and transformers reach whichever one the packages of their environment...
Running several engines
An application can serve several models at once, each on the engine that suits it: llama.cpp in its own process, and any hosted backend in a runtime host....
Perception
Video in, per-frame object tracks out, as an open replacement for a Metropolis/DeepStream perception pipeline. praecise-codec decodes the video in process, an ONNX detector...
What the layer provides
praecise_runtime::spec_policy decides whether to speculate given model shape and load, and returns a reason with every plan — a policy that cannot explain itself gets switched...
Crates
Testing
Runs fmt, clippy and the test suite on the hardware layer and the backend-agnostic runtime, plus a compile check of the backend-gated code. It validates correctness on x86 —...
Guides
Hardware support
Praecise Engine runs on any vendor's silicon, not one. This page is the record of what is supported on each, by which route, and how far each claim has been proved. The plan...
Silicon coverage
Praecise Engine is described as backend-agnostic. Until November 2026 that was true of its backends and not of its hardware: every route that touched a device assumed NVIDIA,...
Admission and queueing
A serving layer that admits on predicted end-to-end time, against a fixed deadline, from a cost model that never saw the output length, will refuse almost everything on a slow...
Prefix reuse
BatchEngine serves every running request from one context and a fixed pool of sequence slots. When a request finishes, its slot keeps what it computed, and the next request...
Request shaping
Decode on GB10 is bandwidth-bound at ~80% of peak (see gb10-bandwidth-ceiling.md), so neither speculative allocation nor host-side scheduling has much room. What remains...
Adding a diffusion family
Status: the first family is implemented as the praecise-diffusion crate: FLUX.2 [klein] (rectified-flow transformer, Qwen3 prompt encoder, KL autoencoder), text-to-image and...
Method and measurements
Foundations
Chip vendors are the visible layer. Under them, a small number of foundries, packaging lines, memory makers and interconnect standards decide what silicon can exist, how much...
Measured draft acceptance
The first non-simulated measurement in this work. Run 28 Aug 2026 on an RTX 5070 Ti (16 GB, discrete), Qwen3-0.6B Q8_0, llama.cpp b10667, n-gram self-speculation (--spec-type...
What actually helps
Decode on this hardware runs at ~80% of its 273 GB/s memory-bandwidth ceiling (see gb10-bandwidth-ceiling.md), so the only things that help are the ones that move fewer bytes...
The GB10 decode ceiling
Decode on a DGX Spark is bandwidth-bound and already running at ~80% of peak. There is no software win waiting. This is the governing constraint on anything Praecise Engine...
Acceleration research
Surveyed August 2026 against llama.cpp, vLLM, SGLang and unsloth, with a November 2026 addendum at the end. Every claim below was checked against source, release notes or a...
qwen4exp provenance
qwen4exp is upstream llama.cpp's work, not ours. This note exists because a modified copy of it sat uncommitted in the vendored fork, and committing that to Praecise/llama.cpp...