Platform · Metrale Engine
Not a generic runtime. A compiled inference stack.
Metrale Engine is a pure Rust and CUDA inference engine. It picks a workload, model plus quantization plus hardware target, compiles the kernel set for it, ships a tested build, and serves it behind the APIs your applications already speak.
Who this is forPlatform teams that own GPUs and are judged on tokens per GPU, and the developers who have to keep those GPUs busy at 128 concurrent agents rather than one chat window.
What it does
Selected workload
Model, quantization and hardware target chosen together. NVFP4 and FP8 with per target kernels, not one generic path with flags.
Optimized for the silicon
Hand tuned attention, MoE, Gated DeltaNet and Mamba class kernels per hardware target, register level work with no generic fallback on the hot path.
Speculative decoding
MTP draft heads and DFlash block diffusion, with a resolver that picks the verify width itself. The scheduler decides, not a flag.
Prefix cache and KV
Radix tree prefix caching so a shared system prompt is prefilled once, not per agent. Paged KV with tiered offload across host RAM, NVMe and RDMA peers.
Expert parallel across nodes
EP=2 across two DGX Sparks ships as recipes today. Three node topology is being wired up. The same binary, the same recipes.
Familiar APIs
OpenAI compatible chat and completions, the Anthropic Messages API and the Responses API from one binary, so agents, SDKs and gateways point at it unchanged.
the matched vLLM configuration at C=128, same GB10
aggregate at C=128, unsloth/Qwen3.8-27B-NVFP4
one binary, no Python, no PyTorch, no runtime compilation
model recipes, every one verified before it is listed
“Night and day compared to the 10 minute torch.compile cycle. Startup in about 15 seconds and it just stays coherent in an agentic loop.”ronald_15496, Discord· source
Questions
The questions we actually get asked.
Short answers. Each one is backed by something on this site or in the repository.
What hardware does it run on?
NVIDIA DGX Spark (GB10) is verified today, and AMD Strix Halo (gfx1151) runs the same CUDA source compiled through SCALE, with both submitted to MLPerf Inference v6.1. Hopper and Blackwell datacenter targets are in active bring up with receipts in the changelog. Expert parallelism across two nodes ships as recipes and a three node topology is being wired up.
Which models can I run?
Every model on this site maps to a recipe in the atlas-recipes repository, which is the single source of truth, so the site cannot list a model without one. Qwen leads with the most recipes, alongside Gemma, Nemotron, Mistral, MiniMax and DeepSeek. Bring your own weights and we scope the bring up.
Ask the rest in a working session, or read the deployment guide ↗.
Next step
See it against your own workload.
A side by side ladder on your hardware in week one. Your models, your criteria, your receipt.