Stacks/Agent platform / infrastructure

NVIDIA Agent Substrate

NVIDIA's Agent Substrate is a production-oriented Kubernetes runtime, donated to the CNCF Sandbox, for the idle-heavy shape of agent workloads. Many sessions share a pool of warm Pods; idle ones are snapshotted (memory and filesystem) and restored later. It pairs with NVIDIA Dynamo for inference routing and KAI Scheduler for cluster placement, and targets coding assistants, RL environments, and GPU sandboxes.

Use case

A security-hardened, zero-trust Kubernetes runtime for running sandboxed agents and tools at high density — coding assistants, RL post-training environments, and short-lived GPU sandboxes — now donated to the CNCF Sandbox

Category

Agent platform / infrastructure

Team size

not stated

Orchestrator / framework

Agent Substrate multiplexes many agent sessions over a small pool of warm Kubernetes worker Pods. A separate control plane maps actors (agents and their environments) onto that pool. NVIDIA Dynamo coordinates inference across GPU workers; Dynamo's router uses session identity, KV-cache location, and tool-call boundaries to keep related model calls on workers that already hold reusable context. KAI Scheduler places Substrate worker Pods alongside GPU trainers and inference servers on the NVIDIA DSX AI Factory Platform.

Memory

Idle sessions are checkpointed, memory and filesystem included, and restored onto whichever worker is free when the next request arrives. Dynamo exposes inference signals (projected time to first token, GPU-pool saturation) so Substrate can keep an actor resident, suspend it during long inference waits, or resume it as the response nears completion.

Evals

not stated

Observability

not stated

Monthly cost

not stated

Models

not stated

Observability tools

not stated

Other tools

gVisor or microVM sandboxesKubernetes Dynamic Resource Allocation (for pausing and restoring GPU state)CNCF Sandbox

Source

https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale/

Ekin Karabulut and Omer Yahud · Nov 9, 2026