Platform · Metrale Control
Deploys, governs and repairs the fleet. Never on the inference path.
Metrale Control is the governance and control plane. It turns a signed recipe into a running, routed, observed workload on the GPUs you own, then keeps it that way without a human in the loop. A control plane outage never stops inference that is already running.
Who this is forInfrastructure and platform engineering teams responsible for a GPU fleet, and the security and compliance owners who have to sign off on what runs on it.
Metrale Console. Demo data, recorded from the product mockup.
What it does
Signed recipes and release channels
Every model is a recipe. Every recipe is signed. Edge, beta, stable and LTS channels, promoted by digest and never rebuilt between environments.
Canary rollouts that roll back on telemetry
Five percent, twenty five, one hundred. A canary that regresses TTFT, throughput or error rate fails the release gate and rolls back on its own.
GPU aware routing
Workers are chosen on projected prefill and decode cost, queue depth, VRAM pressure and KV cache reuse. A request stays on the worker that started it.
Autoscaling and node repair
Pods scale on queue wait, TTFT and active sequences, nodes scale on demand, and a failed GPU is cordoned, drained and replaced, not patched by hand.
Fleet policy
Data residency, model allowlists, redaction and tenant quotas as policy objects, enforced at the router and recorded in the audit log.
Your account, your metal, or air gapped
Runs in your AWS, Azure or GCP account through Terraform or Helm, on premises, or on an isolated network. Regulated buyers can use the customer pull GitOps mode where Metrale never holds credentials.
The single operator fleet manager is live at /control and in active development. Multi tenant control plane, RBAC and SLA enforcement are the next milestones and this page will say so until they ship.
How it is built
One control plane. Three places to run it. Never in the request path.
Proposed. From the platform architecture brief, September 2026. What is live is named in the status line below. The rest is the shape the platform is being built to.
Operator and agent, outbound only
Your cluster runs an operator that reconciles desired state. An agent calls out over mTLS on port 443 and opens no inbound port. No SSH, and in the customer pull mode Metrale holds no credentials.
Certified combinations, or it does not schedule
Runtime release, model revision, quantization, GPU class and topology are certified together, with the benchmark receipt attached. Production scheduling refuses a combination that is not on the registry.
Signed releases, staged rollouts
An immutable release manifest names the image digest, the kernel set, the model hashes and the recipe digest. Rings from development to canary to production, and rollback on its own when TTFT, error rate or throughput regress.
A kernel registry, under license
Hardware specific kernel sets are versioned in a registry. A licensed node fetches the set for its silicon at boot, and nothing serves through the gateway without an entitlement.
The engine keeps its scheduler
The platform chooses the cluster, the pool and the replica. Batching, prefill and decode, KV cache and speculative decoding stay inside the engine, where the numbers come from.
Prompts stay where they run
The control plane stores configuration, policy and usage records. Prompts, weights, KV contents and responses stay in the data plane unless you switch on a debugging feature yourself.
release channels, promoted by digest
inference requests that pass through the control plane
percent, the canary ladder every release climbs
Questions
The questions we actually get asked.
Short answers. Each one is backed by something on this site or in the repository.
Do I have to replace vLLM, SGLang or llama.cpp to use it?
No. Metrale deploys beside your current engine and takes traffic one model family at a time. Many teams keep the old engine for a family we do not ship a recipe for yet. You decide what moves, on your own side by side numbers.
How long does a deployment take?
One binary per node and one signed recipe per model. A single box runs in minutes from one install command. A fleet pilot has a side by side ladder against your current engine in week one. Production cutover is workload by workload over the following weeks, at your pace.
Can it run air gapped?
Yes. The engine is one binary with no runtime download and no Python environment to resolve. Recipes, models and kernels are delivered as signed artifacts and installed from local media. Telemetry can stay entirely inside the network and export on your schedule, or never.
Does it run in my cloud account?
Yes. Bring your own cloud deploys the engine and router into your AWS, Azure or GCP account, on your GPU node pools, through Terraform or Helm. The control plane sees configuration, licensing, versions and aggregate telemetry, and nothing else. Regulated buyers can run the customer pull GitOps mode, where Metrale never holds credentials to your account.
Which APIs does it expose?
OpenAI compatible chat and completions, the Anthropic Messages API and the Responses API, from the same binary, so existing SDKs, agents and gateways point at Metrale without code changes.
What support comes with it?
Community support in Discord for the open source engine. Enterprise includes a named engineer, a response SLA and a shared channel. Forward deployed engineering for the pilot and the cutover is scoped per engagement and credited against the first year on conversion.
Ask the rest in a working session, or read the deployment guide ↗.
Next step
See it against your own workload.
A side by side ladder on your hardware in week one. Your models, your criteria, your receipt.