Device allocation¶

Goal: each Pod receives its NPUs and RoCE NICs together from one NUMA node. The reference configuration uses 4 NPUs and 2 NICs per Pod. In Dynamic Resource Allocation (DRA) mode, the ResourceClaimTemplate (RCT) that you apply sets the NPU count and the NIC count.

The RCT is a namespaced resource, so first create the namespace the stack uses. You also create the Secrets and the model store of Cluster foundation in this namespace.

1

kubectl create namespace <ns>

The reference configuration allocates the NPUs and RoCE NICs through DRA, because an RCT can require the NPUs and NICs to come from the same NUMA node. For the comparison with a device plugin, see Coding Assistant on Rebel100™ › Design alternatives.

For the general DRA concepts and API, see the Kubernetes documentation on Dynamic Resource Allocation. This page covers only the DRA-related Kubernetes resources this stack uses and who creates each.

The scheduler reads the following resources to allocate devices to a Pod. If the DeviceClass, the ResourceSlice, or the RCT is missing, the Pod stays Pending. The chart writes the Pod's spec.resourceClaims at install time. Kubernetes creates a ResourceClaim for each Pod from the RCT, and the scheduler records the allocated devices in that ResourceClaim.

The reference configuration allocates NPUs and RoCE NICs from the same NUMA node to each Pod, so that the KV transfer does not cross NUMA nodes. The matchAttribute: resource.kubernetes.io/numaNode constraint of the numa-aligned RCT implements this same-NUMA allocation. A server node in the reference architecture has two NUMA nodes with 4 NPUs each. Therefore, at most two Pods that use 4 NPUs each run on one server node.

ResourceClaimTemplate for npu-rdma¶

You create this resource. The chart does not create the RCT. The chart references the RCT only by name (devices.claimTemplateName).

The reference configuration uses the chart's examples/manifests/rct/numa-aligned.yaml. numa-aligned.yaml requests 4 NPUs and 2 NICs per Pod from the same NUMA node. numa-aligned.yaml also requires the two NICs to be on different physical NICs, that is, to have different resource.kubernetes.io/pcieRoot values. Both the RBLN NPU DRA driver and dranet (v1.5.0 or later) must advertise resource.kubernetes.io/numaNode. dranet must also advertise resource.kubernetes.io/pcieRoot. On Kubernetes 1.34 and 1.35, enable the DRAConsumableCapacity feature gate on kube-apiserver, kube-scheduler, and kubelet.

1

kubectl -n <ns> apply -f examples/manifests/rct/numa-aligned.yaml

On a cluster that differs from the reference configuration, apply or edit the RCT as follows.

• A driver does not advertise resource.kubernetes.io/numaNode, or dranet does not advertise resource.kubernetes.io/pcieRoot: examples/manifests/rct/numa-unaligned.yaml.
• An AGG install without NICs: examples/manifests/rct/npu-only.yaml. Also set devices.rdma.enabled=false and devices.claimTemplateName=npu-only.
• A model that has a recipe in the chart: the recipe's examples/models/<model>/rct-*.yaml.
• Backend NICs other than the Broadcom NICs of the reference node: change the pciSubsystem == "9140" term in the RCT's NIC selector to match your NICs, or remove the term.

Device allocation checks¶

Check that the scheduler can allocate the devices that the RCT requests. Check item (5) before installing (see Item (5) attribute cross-check).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# (1) both DeviceClasses
kubectl get deviceclasses

# (2) per node, a ResourceSlice from each driver (NPUs and NICs)
kubectl get resourceslices
kubectl get resourceslices -o wide | head

# (3) the RCT exists
kubectl -n <ns> get resourceclaimtemplate npu-rdma

# (4) actual allocation, after the stack is installed
kubectl -n <ns> get resourceclaims

The reference RCT (numa-aligned.yaml) requests only NICs that can use RDMA (NICs whose dra.net/rdma is true). dranet sets dra.net/rdma to true when the NIC vendor's RDMA driver is loaded. The Broadcom NICs of the reference node use bnxt_re as their RDMA driver. On each node, check that the RDMA driver recognizes the NICs.

1

ls /sys/class/infiniband/    # one entry per NIC that uses RDMA; on the reference node, bnxt_re0, bnxt_re1, and so on

Run the same ls /sys/class/infiniband/ check inside the modelserver container. Inside a Pod, the list shows only the NICs allocated to that Pod.

1

kubectl -n <ns> exec <modelserver-pod> -c modelserver -- sh -c 'ls /sys/class/infiniband/'

1
2
3
4
5
6
7

# (a) attributes the RCT requires: the CEL expressions in selectors, and matchAttribute and distinctAttribute in constraints
kubectl -n <ns> get resourceclaimtemplate npu-rdma -o yaml \
  | grep -nE 'device\.attributes|matchAttribute|distinctAttribute'

# (b) attributes the cluster advertises, per driver
kubectl get resourceslices -o json | jq '.items[]
  | {driver: .spec.driver, attrs: (.spec.devices[0].attributes | keys)}'

Key device settings¶

Set the following chart values to match the RCT.

For the RDS node label, see Node preparation › RDS setup.

Device-plugin mode¶

On a cluster without DRA, install the chart in device-plugin mode with the following values. Device-plugin mode uses no RCT and does not guarantee NUMA alignment. Device-plugin mode is Unverified.

1
2
3
4
5
6
7
8

devices:
  npu:
    mode: devicePlugin
    count: 4
  rdma:
    mode: devicePlugin
    resourceName: <rdma-resource-name>  # the cluster's RDMA extended resource name, for example rdma/hca
    count: 2