Selecting a backend
praecise_runtime::Backend names the runtimes the engine knows about and
reports, per backend, what is actually possible:
use praecise_runtime::Backend;
let backend = Backend::parse("vllm")?; // unknown names error, never default
backend.ensure_available()?; // refuses rather than silently substituting
let caps = backend.supports();The distinction that matters is how a backend is reached, because it decides what acceleration is available at all:
| Backend | Integration | Engine-side speculation | Structured output |
|---|---|---|---|
| llama.cpp | linked (FFI) | yes — the engine drives the decode loop | yes |
| vLLM | hosted | no — the runtime owns its decode loop | no |
| SGLang | hosted | no | no |
| TensorRT-LLM | hosted | no | no |
| transformers | hosted (CPU or GPU) | no | no |
| OpenVINO GenAI | hosted (Intel CPU, GPU or NPU) | no | no |
| an inference API | remote (opt-in) | no — the provider owns the decode loop | no |
A linked backend is compiled in, so the engine can propose a draft block, verify it, and read per-position logits. A hosted backend runs in a runtime host: the engine chooses the model and the sampling parameters, but there is no seam at which to interpose on token-by-token decoding. Speculation against a hosted backend is therefore not merely unimplemented — it is impossible by construction, and those runtimes do their own instead.
A remote backend runs nowhere on this machine: praecise-remote asks an
OpenAI-compatible inference API (Groq, Cerebras, SambaNova or any other) and
answers in the same types. It is the one route to silicon sold as a service,
and the one backend whose prompts leave the machine, so it is opt-in twice:
the application has to depend on the crate, and the operator has to name each
host in PRAECISE_REMOTE_ALLOW. Runtime hosts still have no network.
use praecise_remote::{Allow, Provider, Remote};
let groq = Remote::open(Provider::Groq.endpoint(model), &Allow::from_env())?;
let reply = groq.generate(&messages, &config, |text| { print!("{text}"); true })?;An unknown backend name is an error listing the valid ones, never a quiet fallback: a typo that silently ran on a different runtime would invalidate any measurement taken against it, with nothing in the output to show it.