Praecise Engine v0.1.0 Read the docs

Open source · Apache-2.0 · Rust · v0.1.0

Make any model faster, on any backend.

Praecise Engine sits between an application and an inference backend and speeds up token generation, through speculative decoding, hardware-aware tuning and a uniform serving surface. The application does not need to know which backend is underneath. llama.cpp is the first supported backend.

cargo add praecise-runtime --git https://github.com/Praecise/praecise-engine

One interface, and it never substitutes silently.

A backend is named, checked and asked what it can do. An unknown name is an error, and a backend that is not available refuses instead of falling back to another.

Selecting a backend
rust
use praecise_runtime::Backend;

// unknown names error, never default
let backend = Backend::parse("vllm")?;

// refuses rather than silently substituting
backend.ensure_available()?;

let caps = backend.supports();
Where it sits
application / serving layer
          │
  Praecise Engine
  speculation policy, drafter selection,
  request shaping, sampling, KV strategy
          │  pluggable backends
  ┌───────┴────────────────┐
  linked     hosted        remote
  llama.cpp  vLLM, SGLang  an inference
             and others    API (opt-in)

What the layer provides

It concentrates on the decisions a backend cannot make for itself, and on avoiding losses instead of manufacturing wins.

Measured, including where it does nothing

On an RTX 5070 Ti with qwen3.5-0.8b and n-gram self-speculation. Speculation is worth 1.5 to 3.5 times on structured text and nothing on prose, so the decision whether to speculate is where the value is.

PromptTokens per secondDraftedAgainst baseline
SQL226.31713.5x
Code94.5731.5x
Number list63.80no gain
JSON65.40no gain
Prose62 to 650no gain

Decode is bound by memory bandwidth, which limits what any layer above a backend can honestly claim. How acceptance was measured

The documentation

Every page is written in the project's own repository and built from it, so the site and the code say the same thing.

Around the engine

The engine makes a model faster. These work at other levels and none of them is needed for it to be useful.

    Type to search. Arrow keys move, Enter opens.