Praecise Engine v0.1.0 Read the docs

Documentation · Guides

Adding a diffusion family to Praecise Engine

Status: the first family is implemented as the praecise-diffusion crate: FLUX.2 [klein] (rectified-flow transformer, Qwen3 prompt encoder, KL autoencoder), text-to-image and reference-image conditioning, bf16 or load-time 8-bit weights, on the same ggml build, device and backend as the language-model family. On a host with GPU hardware it refuses to run on the CPU. It runs in the process: an application claims the model in praecise_host::Engines with Flux2Klein::resident_bytes() before loading it, so one model is served by exactly one engine within one GPU memory budget. examples/flux2 generates and reports per-stage timings; praecise-diffusion/tests/parity checks every stage against the reference implementation on a small random checkpoint.

The design notes below were written after serving LTX-2.5 (22B video DiT) alongside Praecise Engine-accelerated LLMs on a GB10.

Naming. Praecise is the family. Praecise Engine is the acceleration layer — it makes a model faster. praecise-harness is the agentic framework — it makes an agent work. They sit at different levels and are used independently. This document is about Praecise Engine.

The problem

Praecise Engine is backend-agnostic acceleration on top of an inference backend, llama.cpp first (linked), with vLLM, SGLang, TensorRT-LLM and transformers reached as hosted backends. That is agnosticism across backends. Every backend on that roadmap is an autoregressive token generator.

A diffusion model is not a different backend for the same task. It is a different task, and the public surface says so:

rust
pub struct GenerationConfig {
    pub temperature: f64,      pub top_p: f64,
    pub max_tokens: u32,       pub repeat_penalty: f32,
    pub repeat_last_n: usize,  pub seed: u64,
    pub top_k: Option<u32>,    pub min_p: Option<f64>,
    pub frequency_penalty: f32, pub presence_penalty: f32,
    pub stop: Vec<String>,
    pub draft_n: Option<u8>,   pub commitment_k: Option<u8>,
}

pub struct InferenceResult {
    pub text: String,          pub thinking: Option<String>,
    pub input_tokens: u32,     pub output_tokens: u32,
    pub tokens_per_second: f64, pub stop_reason: StopReason,
    pub commitment: Option<InferenceCommitment>,
}

One field — seed — means anything to a denoiser. There is no token to sample, no distribution to shape, no stop string, no KV cache, no draft model to verify against. StopReason has no referent: a diffusion run ends because the schedule ended.

The question is whether Praecise Engine takes a second task family. That is a larger commitment than a second backend.

What is shared

  1. Residency and admission. The binding constraint is not decode speed, it is which weights may occupy memory. On a 121.7 GiB unified-memory GB10 a 22B video DiT needs a 66.5 GiB serving set against an LLM holding 41.6 GiB under an 88 GiB ceiling. That arithmetic is identical in kind for both families, and today neither Praecise Engine nor its consumer owns it — it leaks into node config.
  2. The hardware-tuning harness. The machinery for measuring and selecting per device. Not the tunings themselves.
  3. Load-time numeric casts. fp8-cast-style downcasting is the same plumbing for a decode loop or a denoiser.
  4. Determinism. Seeded, reproducible runs matter more for media than text.
  5. Verifiable inference (toploc), if it stays — see open questions.

What is not shared

Sampling, speculative decode, KV cache, sequence batching, stop-string streaming. batching.rs batches sequences; diffusion batches latents across a fixed step count — a different scheduling problem with a different cost model.

Acceleration conclusions invert between the families

This is the finding that shapes the design.

technique autoregressive decode video diffusion
weight-only NVFP4 wins — bandwidth-bound loses — ~33% smaller, ~6% slower
speculative decode (DFlash/MTP) 1.85–2.13x measured n/a — nothing to draft
step caching (TeaCache) n/a wins
attention sparsification (STA) limited wins — ~3x reported

LLM decode is memory-bandwidth-bound, so shrinking weights buys time. Video diffusion is compute-bound, so shrinking weights buys an unpack penalty. NVIDIA's published "20% faster, 40% memory savings" for LTX-2.5 is NVFP4 plus FastVideo kernels, STA and TeaCache. The kernels are the speedup, not the format.

The shared layer is therefore the harness — measure, select, record — never a table of techniques presumed portable. A praecise-core exporting "use NVFP4 on Blackwell" is wrong half the time.

Shape

praecise-core        error, seed/determinism, device + memory budget,
                     tuning harness, telemetry, (toploc?)
praecise-runtime     autoregressive LM family — today's crate, API unchanged
praecise-diffusion   iterative-denoiser family — new

Additive. praecise-runtime keeps its public API verbatim. The families share praecise-core and nothing else, each owning its config and result types:

rust
pub struct DenoiseConfig {
    pub steps: u32,
    pub guidance_scale: f32,
    pub schedule: SigmaSchedule,   // distilled checkpoints ship fixed ones
    pub seed: u64,
    pub cast: Option<NumericCast>, // fp8-cast et al, load-time
}

pub struct DenoiseResult {
    pub steps_run: u32,
    pub time_ms: u64,
    pub steps_per_second: f64,
}

No StopReason, no token counts, no sampler. Unifying these with GenerationConfig behind one trait produces a struct where most fields are None for any given caller.

What this has to earn

A diffusion path already exists outside Praecise Engine: a Python/diffusers worker with a FamilyAdapter seam, a job marketplace, and per-family conversion fixes. A second implementation is justified by one of:

  • In-process serving — no Python worker, no IPC, for a node already linking Praecise Engine.
  • One memory budget across families, so a DiT and an LLM contend through the same admission control rather than two schedulers that cannot see each other. This is the strongest case, and the exact failure observed: 66.5 + 41.6 GiB against an 88 GiB ceiling, neither side aware of the other.
  • Verifiable media inference, if commitments extend to latents.

Staging

  1. Extract praecise-core from praecise-runtime with no behaviour change — error, seed, device/memory budget, tuning harness. praecise-runtime depends on it and keeps its API.
  2. Land the memory/admission model in praecise-core; the LM family uses it. This carries its own value independent of a second family.
  3. praecise-diffusion behind a feature flag, one model (LTX-2.5 dev bf16), no kernel work — correctness first, fp8-cast and CPU offload as the levers.
  4. Kernels (NVFP4-aware linears, STA, TeaCache) last, against measured baselines, given the inversion above.

Step 3 is the commitment point, and it is reversible.

Open questions

  • toploc placement. InferenceResult.commitment and toploc.rs are the largest non-batching module, and verifiable-inference is slated to leave Praecise Engine as a consumer concern. Settle this before a second family inherits the ambiguity.
  • Does batching generalise? Latent batching across a fixed step count may reuse the slot accounting.
  • ARM. FastVideo ships no prebuilt aarch64 wheel; kernel work on GB10 builds from source.

Serving a video world model: what is hosted today and what a service would need

Inventory (2026-10-29, checked against the code). The Nebius study discussed in admission-and-queueing.md served a video model as a service; this repository hosts video models in two narrower ways:

  • In process: praecise-diffusion contains a Cosmos3 implementation (src/cosmos3.rs, with src/cosmos3/), exposed through examples/cosmos3, which takes a checkpoint directory, a prompt, optional first frame, frame count and run count and reports per-stage timings. wan-world, ltx2 and a matrix_game module sit beside it. These are examples and library code: nothing here is a long-running, multi-request video server, and the examples run one generation at a time.
  • As a job: praecise_host::job runs cosmos.generate (Cosmos 3 Edge or Nano through the Python diffusers pipeline in a pinned environment, with the Cosmos guardrail forced on). One run per process, confined, to completion.
  • Not present: the Cosmos 3 Super model the study used, a request queue for video, multi-GPU parallelism, or any measurement of any of it against this study. No claim is made here that the engine reproduces the study's throughput or latency.

If the engine were to serve a video world model as a service, the study says what the serving path must have:

  1. One in-flight generation per resident copy unless a measurement shows otherwise, with the queue outside the model; the second request should wait, not share. The Engines claim already guarantees one copy.
  2. A memory ledger that sees the whole serving set. diffusion-family.md records a 66.5 GiB video DiT serving set beside a 41.6 GiB LLM (108.1 GiB against an 88 GiB ceiling on one GB10, neither side aware of the other); the Budget reservation must include text encoder, VAE and activations, not weights alone.
  3. A video gate: the output decodes fully, has the expected frame count, width and height, and is neither blank nor frozen. Gate takes any checker; the video one would use praecise-codec. It does not exist yet.
  4. Latency as time to a complete file, and a throughput unit of valid video-seconds per hour, which is units in Attempt with units = seconds of valid video.
  5. Throughput accounted per whole run: warmup, model load and encode in or out of the timed boundary stated in the pin, as the study did (it excluded startup and load, included routing, generation and MP4 encoding).
  6. Guardrail state in the pin. The study ran with guardrails disabled; the cosmos.generate job forces them on. A comparison across the two is not like for like.
  7. A topology decision only on hardware with more than one device. On a single GB10 the question does not arise.

Edit this page on GitHub · Read it as Markdown

    Type to search. Arrow keys move, Enter opens.