Adding a diffusion family to Praecise Engine
Status: the first family is implemented as the praecise-diffusion
crate: FLUX.2 [klein] (rectified-flow transformer, Qwen3 prompt encoder,
KL autoencoder), text-to-image and reference-image conditioning, bf16 or
load-time 8-bit weights, on the same ggml build, device and backend as the
language-model family. On a host with GPU hardware it refuses to run on the
CPU. It runs in the process: an application claims the model in
praecise_host::Engines with Flux2Klein::resident_bytes() before loading
it, so one model is served by exactly one engine within one GPU memory
budget. examples/flux2 generates and reports per-stage timings;
praecise-diffusion/tests/parity checks every stage against the reference
implementation on a small random checkpoint.
The design notes below were written after serving LTX-2.5 (22B video DiT) alongside Praecise Engine-accelerated LLMs on a GB10.
Naming. Praecise is the family. Praecise Engine is the acceleration layer — it makes a model faster. praecise-harness is the agentic framework — it makes an agent work. They sit at different levels and are used independently. This document is about Praecise Engine.
The problem
Praecise Engine is backend-agnostic acceleration on top of an inference backend, llama.cpp first (linked), with vLLM, SGLang, TensorRT-LLM and transformers reached as hosted backends. That is agnosticism across backends. Every backend on that roadmap is an autoregressive token generator.
A diffusion model is not a different backend for the same task. It is a different task, and the public surface says so:
pub struct GenerationConfig {
pub temperature: f64, pub top_p: f64,
pub max_tokens: u32, pub repeat_penalty: f32,
pub repeat_last_n: usize, pub seed: u64,
pub top_k: Option<u32>, pub min_p: Option<f64>,
pub frequency_penalty: f32, pub presence_penalty: f32,
pub stop: Vec<String>,
pub draft_n: Option<u8>, pub commitment_k: Option<u8>,
}
pub struct InferenceResult {
pub text: String, pub thinking: Option<String>,
pub input_tokens: u32, pub output_tokens: u32,
pub tokens_per_second: f64, pub stop_reason: StopReason,
pub commitment: Option<InferenceCommitment>,
}One field — seed — means anything to a denoiser. There is no token to sample,
no distribution to shape, no stop string, no KV cache, no draft model to verify
against. StopReason has no referent: a diffusion run ends because the schedule
ended.
The question is whether Praecise Engine takes a second task family. That is a larger commitment than a second backend.
What is shared
- Residency and admission. The binding constraint is not decode speed, it is which weights may occupy memory. On a 121.7 GiB unified-memory GB10 a 22B video DiT needs a 66.5 GiB serving set against an LLM holding 41.6 GiB under an 88 GiB ceiling. That arithmetic is identical in kind for both families, and today neither Praecise Engine nor its consumer owns it — it leaks into node config.
- The hardware-tuning harness. The machinery for measuring and selecting per device. Not the tunings themselves.
- Load-time numeric casts.
fp8-cast-style downcasting is the same plumbing for a decode loop or a denoiser. - Determinism. Seeded, reproducible runs matter more for media than text.
- Verifiable inference (
toploc), if it stays — see open questions.
What is not shared
Sampling, speculative decode, KV cache, sequence batching, stop-string
streaming. batching.rs batches sequences; diffusion batches latents across
a fixed step count — a different scheduling problem with a different cost model.
Acceleration conclusions invert between the families
This is the finding that shapes the design.
| technique | autoregressive decode | video diffusion |
|---|---|---|
| weight-only NVFP4 | wins — bandwidth-bound | loses — ~33% smaller, ~6% slower |
| speculative decode (DFlash/MTP) | 1.85–2.13x measured | n/a — nothing to draft |
| step caching (TeaCache) | n/a | wins |
| attention sparsification (STA) | limited | wins — ~3x reported |
LLM decode is memory-bandwidth-bound, so shrinking weights buys time. Video diffusion is compute-bound, so shrinking weights buys an unpack penalty. NVIDIA's published "20% faster, 40% memory savings" for LTX-2.5 is NVFP4 plus FastVideo kernels, STA and TeaCache. The kernels are the speedup, not the format.
The shared layer is therefore the harness — measure, select, record — never a
table of techniques presumed portable. A praecise-core exporting "use NVFP4
on Blackwell" is wrong half the time.
Shape
praecise-core error, seed/determinism, device + memory budget,
tuning harness, telemetry, (toploc?)
praecise-runtime autoregressive LM family — today's crate, API unchanged
praecise-diffusion iterative-denoiser family — newAdditive. praecise-runtime keeps its public API verbatim. The families share
praecise-core and nothing else, each owning its config and result types:
pub struct DenoiseConfig {
pub steps: u32,
pub guidance_scale: f32,
pub schedule: SigmaSchedule, // distilled checkpoints ship fixed ones
pub seed: u64,
pub cast: Option<NumericCast>, // fp8-cast et al, load-time
}
pub struct DenoiseResult {
pub steps_run: u32,
pub time_ms: u64,
pub steps_per_second: f64,
}No StopReason, no token counts, no sampler. Unifying these with
GenerationConfig behind one trait produces a struct where most fields are
None for any given caller.
What this has to earn
A diffusion path already exists outside Praecise Engine: a Python/diffusers
worker with a FamilyAdapter seam, a job marketplace, and per-family conversion
fixes. A second implementation is justified by one of:
- In-process serving — no Python worker, no IPC, for a node already linking Praecise Engine.
- One memory budget across families, so a DiT and an LLM contend through the same admission control rather than two schedulers that cannot see each other. This is the strongest case, and the exact failure observed: 66.5 + 41.6 GiB against an 88 GiB ceiling, neither side aware of the other.
- Verifiable media inference, if commitments extend to latents.
Staging
- Extract
praecise-corefrompraecise-runtimewith no behaviour change — error, seed, device/memory budget, tuning harness.praecise-runtimedepends on it and keeps its API. - Land the memory/admission model in
praecise-core; the LM family uses it. This carries its own value independent of a second family. praecise-diffusionbehind a feature flag, one model (LTX-2.5 dev bf16), no kernel work — correctness first,fp8-castand CPU offload as the levers.- Kernels (NVFP4-aware linears, STA, TeaCache) last, against measured baselines, given the inversion above.
Step 3 is the commitment point, and it is reversible.
Open questions
toplocplacement.InferenceResult.commitmentandtoploc.rsare the largest non-batching module, and verifiable-inference is slated to leave Praecise Engine as a consumer concern. Settle this before a second family inherits the ambiguity.- Does
batchinggeneralise? Latent batching across a fixed step count may reuse the slot accounting. - ARM. FastVideo ships no prebuilt aarch64 wheel; kernel work on GB10 builds from source.
Serving a video world model: what is hosted today and what a service would need
Inventory (2026-10-29, checked against the code). The Nebius study discussed in admission-and-queueing.md served a video model as a service; this repository hosts video models in two narrower ways:
- In process:
praecise-diffusioncontains aCosmos3implementation (src/cosmos3.rs, withsrc/cosmos3/), exposed throughexamples/cosmos3, which takes a checkpoint directory, a prompt, optional first frame, frame count and run count and reports per-stage timings.wan-world,ltx2and amatrix_gamemodule sit beside it. These are examples and library code: nothing here is a long-running, multi-request video server, and the examples run one generation at a time. - As a job:
praecise_host::jobrunscosmos.generate(Cosmos 3 Edge or Nano through the Pythondiffuserspipeline in a pinned environment, with the Cosmos guardrail forced on). One run per process, confined, to completion. - Not present: the Cosmos 3 Super model the study used, a request queue for video, multi-GPU parallelism, or any measurement of any of it against this study. No claim is made here that the engine reproduces the study's throughput or latency.
If the engine were to serve a video world model as a service, the study says what the serving path must have:
- One in-flight generation per resident copy unless a measurement shows
otherwise, with the queue outside the model; the second request should wait,
not share. The
Enginesclaim already guarantees one copy. - A memory ledger that sees the whole serving set.
diffusion-family.mdrecords a 66.5 GiB video DiT serving set beside a 41.6 GiB LLM (108.1 GiB against an 88 GiB ceiling on one GB10, neither side aware of the other); theBudgetreservation must include text encoder, VAE and activations, not weights alone. - A video gate: the output decodes fully, has the expected frame count,
width and height, and is neither blank nor frozen.
Gatetakes any checker; the video one would usepraecise-codec. It does not exist yet. - Latency as time to a complete file, and a throughput unit of valid
video-seconds per hour, which is
unitsinAttemptwithunits = seconds of valid video. - Throughput accounted per whole run: warmup, model load and encode in or out of the timed boundary stated in the pin, as the study did (it excluded startup and load, included routing, generation and MP4 encoding).
- Guardrail state in the pin. The study ran with guardrails disabled;
the
cosmos.generatejob forces them on. A comparison across the two is not like for like. - A topology decision only on hardware with more than one device. On a single GB10 the question does not arise.