Open source · Apache-2.0 · Rust · v0.1.0
Make any model faster, on any backend.
Praecise Engine sits between an application and an inference backend and speeds up token generation, through speculative decoding, hardware-aware tuning and a uniform serving surface. The application does not need to know which backend is underneath. llama.cpp is the first supported backend.
cargo add praecise-runtime --git https://github.com/Praecise/praecise-engineOne interface, and it never substitutes silently.
A backend is named, checked and asked what it can do. An unknown name is an error, and a backend that is not available refuses instead of falling back to another.
use praecise_runtime::Backend;
// unknown names error, never default
let backend = Backend::parse("vllm")?;
// refuses rather than silently substituting
backend.ensure_available()?;
let caps = backend.supports();application / serving layer
│
Praecise Engine
speculation policy, drafter selection,
request shaping, sampling, KV strategy
│ pluggable backends
┌───────┴────────────────┐
linked hosted remote
llama.cpp vLLM, SGLang an inference
and others API (opt-in)What the layer provides
It concentrates on the decisions a backend cannot make for itself, and on avoiding losses instead of manufacturing wins.
Speculation policy
Decides whether to speculate from the model's shape and the load, and returns a reason with every plan. It refuses where speculation does not pay.
Self-speculation, no draft model
Drafts from token history alone: no second model and no GPU memory. Patterns are reused across the turns of an agent loop.
Drafters with licence enforcement
Published drafter checkpoints are recorded with their licences. Only permissive ones are offered, and a refusal says why.
Request shaping
Makes a batch cheap before it reaches a backend, so one request with a penalty does not put every other request on the slow path.
Any vendor's silicon
NVIDIA, AMD, Intel, Apple, Qualcomm, Arm and Huawei are each reached by a named route, and none is the default.
Several engines, one budget
praecise-host runs confined, supervised engines, one per model, within one memory budget.
Measured, including where it does nothing
On an RTX 5070 Ti with qwen3.5-0.8b and n-gram self-speculation. Speculation is worth 1.5 to 3.5 times on structured text and nothing on prose, so the decision whether to speculate is where the value is.
| Prompt | Tokens per second | Drafted | Against baseline |
|---|---|---|---|
| SQL | 226.3 | 171 | 3.5x |
| Code | 94.5 | 73 | 1.5x |
| Number list | 63.8 | 0 | no gain |
| JSON | 65.4 | 0 | no gain |
| Prose | 62 to 65 | 0 | no gain |
Decode is bound by memory bandwidth, which limits what any layer above a backend can honestly claim. How acceptance was measured
The documentation
Every page is written in the project's own repository and built from it, so the site and the code say the same thing.
Overview
Using it
Guides
Around the engine
The engine makes a model faster. These work at other levels and none of them is needed for it to be useful.
Praecise Harness
The framework for AI agents: models, escalation, grounding, memory and resumable workflows.
Hardware support
What is supported on each vendor's silicon, and how far each claim has been proved.
Praecise Cloud
Cloud services, email, hosting and domains from the same independent provider.