NVIDIA brings enforced GPU memory sharing to Kubernetes

NVIDIA’s October 27 announcement addresses AI jobs that need only part of a GPU. CUDA 13.4 supplies memory enforcement, while GPU Fractioning Operator applies workload limits on Kubernetes nodes. KAI Scheduler selects the GPU and records allocations for the operator to enforce. This separates placement from enforcement; the scheduler alone does not provide the new runtime boundary.

The operator documentation requires a compatible software stack, including Kubernetes 1.28 or later and an NRI-capable container runtime. NRI is the runtime integration used to inject memory limits before the container process starts. GPU nodes need NVIDIA driver r615 or newer; GPU Operator 26.7.1 does not select that driver by default. Its version-check bypass only disables verification: it cannot make an unsupported stack enforce memory limits.

The NvFractions guide adds an explicit readiness prerequisite: administrators install this sharing mode, and both its default configuration and target GPU node must report ready before workloads are submitted. A request reserves the scheduling allocation; a higher limit permits spare-memory use, but the runtime can reclaim that extra memory under contention. Compute sharing is separate: time-slicing takes turns, while sm-sharing runs workloads concurrently and requires MPS. Different compute modes cannot share the same device.