Kubernetes v1.36 Alpha: Pod-Level Resource Managers for Smarter NUMA Allocation

Introduction: A New Era for Resource Management

Kubernetes v1.36 ships an alpha feature called Pod-Level Resource Managers that fundamentally changes how the kubelet allocates CPUs, memory, and topology-aligned resources. Instead of treating each container independently, the kubelet now understands .spec.resources at the pod level, enabling hybrid allocation models that were previously impossible. This brings much-needed flexibility to performance-sensitive workloads without sacrificing NUMA (Non-Uniform Memory Access) alignment.

Why Pod-Level Resource Managers?

Consider a typical pod running a latency-critical database alongside sidecar containers for logging, monitoring, or a service mesh. In earlier Kubernetes versions, to achieve NUMA alignment and Guaranteed QoS class, every container in the pod had to request integer CPU and memory limits. Lightweight sidecars—which might only need a fraction of a core—were forced to reserve entire dedicated cores, wasting precious resources. If you tried to avoid that waste by using non-integer requests, the pod would drop to Burstable QoS, losing the deterministic performance benefits.

Pod-Level Resource Managers eliminate this trade-off. They allow the main application container to own exclusive, NUMA-aligned resources while sidecars share a separately managed pool from the same pod budget. The result: predictable performance for the primary workload and efficient resource usage for auxiliary containers—all without leaving the Guaranteed QoS class.

How Pod-Level Resource Managers Work

This feature is controlled by two feature gates: PodLevelResourceManagers and PodLevelResources. When enabled, the kubelet's Topology Manager, CPU Manager, and Memory Manager all respect pod-level resource specifications. The core idea is simple: the pod's total resource budget (requests and limits defined in spec.resources) becomes the unit of NUMA alignment. The kubelet then carves out exclusive slices for containers that need dedicated resources, while the remaining budget forms a “pod shared pool.” Sidecars or secondary containers can run in that shared pool, sharing resources among themselves but staying isolated from the exclusive slices and from other pods.

This architecture supports multiple Topology Manager scopes (e.g., pod scope, container scope). With pod scope, the entire pod's resource is aligned to one NUMA node; with container scope, each container is independently aligned. Pod-Level Resource Managers work with any scope, but the most powerful use case is pod scope, where the hybrid model truly shines.

Real-World Use Cases

Tightly-Coupled Database with Sidecars (Pod Scope)

Imagine a database pod that includes a main database container, a local metrics exporter, and a backup agent. With pod-scoped Topology Manager and pod-level resources enabled, the kubelet aligns the entire pod budget to a single NUMA node. The database container receives exclusive CPU and memory slices from that node. The remaining resources become a shared pool. The metrics exporter and backup agent run inside that pool, sharing resources with each other but not interfering with the database. This setup guarantees NUMA locality for the database while using only fractional resources for the sidecars.

Example YAML (simplified):

apiVersion: v1
kind: Pod
metadata:
  name: tightly-coupled-database
spec:
  resources:
    requests:
      cpu: "8"
      memory: "16Gi"
    limits:
      cpu: "8"
      memory: "16Gi"
  initContainers:
  - name: metrics-exporter
    image: metrics-exporter:v1
    restartPolicy: Always
  - name: backup-agent
    image: backup-agent:v1
    restartPolicy: Always
  containers:
  - name: database
    image: mydb:latest
    resources:
      requests:
        cpu: "6"
        memory: "12Gi"
      limits:
        cpu: "6"
        memory: "12Gi"

In this example, the pod defines an overall budget of 8 CPUs and 16GB memory. The database container requests 6 CPUs and 12GB memory—these are exclusive and NUMA-aligned. The remaining 2 CPUs and 4GB memory form the shared pool for the init containers (metrics exporter and backup agent). The init containers do not need dedicated resources; they can share the pool.

Other Topology Manager Scopes

While the pod scope example is most impactful, the feature also works with container scope or device scope. For instance, in a machine learning training pod with a GPU and a data loader sidecar, you could align the GPU container to one NUMA node while the sidecar uses resources from a different node, all managed within the pod budget. The flexibility allows cluster operators to design exactly the resource isolation their workloads need.

Enabling and Testing Pod-Level Resource Managers

As of Kubernetes v1.36, Pod-Level Resource Managers are alpha and disabled by default. To try them, you must enable both feature gates on the kubelet:

--feature-gates=PodLevelResourceManagers=true,PodLevelResources=true

Also ensure your Topology Manager is configured with a scope (e.g., --topology-manager-scope=pod). The feature works with static CPU manager policy and any memory manager policy. Note that the pod-level spec.resources field is separate from container-level resources; containers can still define their own requests/limits, and the kubelet will reconcile them with the pod budget.

Conclusion

Pod-Level Resource Managers address a long-standing pain point for performance-critical workloads in Kubernetes. By treating the pod as the fundamental resource allocation unit, this alpha feature eliminates the wasteful trade-off between NUMA alignment and sidecar efficiency. It enables hybrid allocation models that keep primary containers isolated and sidecars resource-light, all while maintaining Guaranteed QoS. For teams running databases, ML training, HFT applications, or any latency-sensitive service, this is a game-changer—and it's available to test in v1.36.