Praecise Engine v0.1.0 Read the docs

Documentation · Using it

Any vendor's silicon

A backend is not a vendor. llama.cpp reaches every accelerator a ggml backend is built for; vLLM, SGLang and transformers reach whichever one the packages of their environment drive; TensorRT-LLM is NVIDIA's and OpenVINO is Intel's. The engine asks which is which before it assigns a model:

rust
use praecise_runtime::hardware::Machine;
use praecise_runtime::Backend;

let machine = Machine::from_env_or_detect()?;   // every vendor, read from the OS
Backend::Vllm.ensure_reaches(&machine)?;        // refuses, naming the backends that do
Silicon Linked (ggml feature) ONNX provider Hosted
NVIDIA GPUs cuda, vulkan TensorRT, CUDA vLLM, SGLang, TensorRT-LLM, transformers
AMD Radeon, Instinct, Ryzen rocm, vulkan; zendnn on Zen CPUs MIGraphX; Vitis AI on the NPU vLLM, SGLang, transformers
Intel Arc, Xe, Core Ultra NPU, Gaudi sycl, openvino, vulkan OpenVINO vLLM, SGLang, transformers, OpenVINO GenAI
Apple M-series metal CoreML none (hosts need Linux)
Qualcomm Adreno, Hexagon opencl, hexagon QNN none
Arm CPUs, Mali kleidiai, vulkan WebGPU none
Huawei Ascend cann CANN vLLM, SGLang, transformers
Google TPU, AWS Trainium, Moore Threads, Cambricon and others musa for Moore Threads none vLLM through the vendor's plugin
Groq, Cerebras, SambaNova none none none; remote, opt-in (praecise-remote)

praecise-hw prints the accelerators of the machine it runs on, the build features that machine needs, and the route to each device. Only the NVIDIA rows have been run on their silicon; the rest are compiled or wired and wait for a run on the device. The full table, with what each word means, is in docs/hardware.md.

Edit this page on GitHub · Read it as Markdown

    Type to search. Arrow keys move, Enter opens.