Any vendor's silicon
A backend is not a vendor. llama.cpp reaches every accelerator a ggml backend is built for; vLLM, SGLang and transformers reach whichever one the packages of their environment drive; TensorRT-LLM is NVIDIA's and OpenVINO is Intel's. The engine asks which is which before it assigns a model:
rust
use praecise_runtime::hardware::Machine;
use praecise_runtime::Backend;
let machine = Machine::from_env_or_detect()?; // every vendor, read from the OS
Backend::Vllm.ensure_reaches(&machine)?; // refuses, naming the backends that do| Silicon | Linked (ggml feature) | ONNX provider | Hosted |
|---|---|---|---|
| NVIDIA GPUs | cuda, vulkan |
TensorRT, CUDA | vLLM, SGLang, TensorRT-LLM, transformers |
| AMD Radeon, Instinct, Ryzen | rocm, vulkan; zendnn on Zen CPUs |
MIGraphX; Vitis AI on the NPU | vLLM, SGLang, transformers |
| Intel Arc, Xe, Core Ultra NPU, Gaudi | sycl, openvino, vulkan |
OpenVINO | vLLM, SGLang, transformers, OpenVINO GenAI |
| Apple M-series | metal |
CoreML | none (hosts need Linux) |
| Qualcomm Adreno, Hexagon | opencl, hexagon |
QNN | none |
| Arm CPUs, Mali | kleidiai, vulkan |
WebGPU | none |
| Huawei Ascend | cann |
CANN | vLLM, SGLang, transformers |
| Google TPU, AWS Trainium, Moore Threads, Cambricon and others | musa for Moore Threads |
none | vLLM through the vendor's plugin |
| Groq, Cerebras, SambaNova | none | none | none; remote, opt-in (praecise-remote) |
praecise-hw prints the accelerators of the machine it runs on, the build
features that machine needs, and the route to each device. Only the NVIDIA
rows have been run on their silicon; the rest are compiled or wired and wait
for a run on the device. The full table, with what each word means, is
in docs/hardware.md.