NPU Driver Installation¶

The RBLN NPU Operator supports two ways to install the NPU kernel driver on each node. Pick one before deploying the operator chart:

Container driver mode (driver.enabled=true): The operator deploys a DaemonSet based on the RBLNDriver CR spec and installs and manages the kernel driver. Use this mode to manage driver installation, upgrades, and rollbacks through Kubernetes.
Host driver mode (driver.enabled=false): You install the kernel driver directly on each node's host OS (for example, with the runfile installer, a distribution package, or DKMS) before deploying the operator. The operator detects the host driver and does not install, upgrade, or roll back the driver itself.

By default, the chart sets driver.enabled=false, so host driver mode is used unless you explicitly enable container driver mode.

This page covers both installation methods and how to verify the active mode for each node. For chart installation, see Installing the NPU Operator.

Container driver mode¶

Set driver.enabled=true when running helm install or helm upgrade. The operator creates the RBLNDriver CR, deploys the driver DaemonSet, and manages the kernel driver on each node.

driver:
  enabled: true

For the full lifecycle, including driver image selection, upgrade rollout policy, manual upgrades, and upgrade exclusions for each node, see NPU Driver Upgrades.

Per-family driver images¶

Prepare driver images for each NPU family, OS, and kernel combination. To support several NPU families in one cluster, the operator resolves the driver image per family and builds each node pool's pull path from the node's rebellions.ai/npu.family label.

1	`<registry>/<org>/<family>/<name>:<version>-<kernel>-<os>`

Segment	Comes from	Example
`<registry>/<org>`	`driver.image.registry` and the org part of `driver.image.repository`; do not include the family in either	`repo.rebellions.ai/rebellions`
`<family>`	The node's `rebellions.ai/npu.family` label, inserted by the operator	`atom`
`<name>`	The last segment of `driver.image.repository`	`rbln-driver`
`<version>-<kernel>-<os>`	`driver.image.tag` plus the node's kernel and OS labels	`3.4.1-6.8.0-138-generic-ubuntu22.04`

The operator inserts the family segment itself, so rebellions/rbln-driver becomes rebellions/atom/rbln-driver on an ATOM™ node. When configuring an RBLNDriver directly, also set spec.image to rebellions/rbln-driver without the family segment.

Check the following when preparing images:

Every driver version your cluster runs must be published under the path of each family that uses it. A single RBLNDriver covering more than one family needs the same version available for all of them; use multiple driver versions when the versions differ.
The family label comes from a NodeFeatureRule named rbln-npu-family that the operator manages itself. The label carries the node's NPU product family in lowercase, such as atom. No manual node labeling is needed, but Node Feature Discovery must be running for the rule to take effect.

Before rolling out a pool, the operator checks that its image exists in the registry. A missing image is reported as DriverImageNotFound with the full image path. See the Helm chart's operator.driverImageCheck setting for when to disable this check.

Host driver mode¶

Install the kernel driver on every NPU node first (see the RBLN Driver Installation Guide) and confirm that rbln-smi is in the host's PATH. Then install the operator with driver.enabled=false:

Host driver detection requires rbln-smi

The driver manager initContainer detects the host driver by running rbln-smi --version through chroot. If rbln-smi is missing from the host's PATH, the node is treated as having no host driver. Confirm the binary is available before installing:

which rbln-smi && rbln-smi --version

driver:
  enabled: false

After installation, confirm the mode on each node as described in Verifying which mode is active per node.

With driver.enabled=false, the chart does not create the RBLNDriver CR or deploy the driver DaemonSet. This setting does not change how the other components are deployed. They require the kernel module to be loaded, regardless of how the driver was installed.

Operational responsibilities¶

Kernel updates: if the host kernel is upgraded, you must reinstall the driver.
Driver upgrades: use existing host management tools such as apt, yum, or DKMS. After the upgrade, reboot the node so the new kernel module is applied.

Mixed Clusters¶

Nodes in container driver mode and nodes in host driver mode can run together under a single RBLNClusterPolicy. Even with driver.enabled=true, the operator detects each node's state and uses the appropriate mode for that node.

Each time a driver pod starts, it checks for a host driver. The installation mode depends only on the node's state.

---
config:
  htmlLabels: false
  flowchart:
    curve: linear
---
flowchart TD
    A["Driver DaemonSet<br/>pod starts"] --> B{"Host driver<br/>detected?"}
    B -->|"Yes"| C["Keep the host driver<br/>Skip container installation"]
    B -->|"No"| D["Install and manage<br/>the container driver"]
    C --> E["Other components use<br/>the node's kernel module"]
    D --> E

The Driver Manager initContainer detects the host driver with chroot /host rbln-smi --version. If it detects a host driver, it sets the node's rebellions.ai/npu.deploy.driver and rebellions.ai/npu.deploy.rbln-smd labels to pre-installed.

Automatic detection supports these two common setups:

Some nodes can use host-installed drivers while others use container drivers. The operator detects the installation mode on each node, so you do not need to configure the mode separately for each node.
To switch a node to host driver mode, install the kernel driver on the host and restart the driver pod. The driver pod detects the host driver and automatically stops the container-based driver installation.

If every node uses a host-installed driver, set driver.enabled=false.

Switching a node back to container driver mode

A node where a host-installed driver was detected keeps its pre-installed labels, and the operator does not revert them automatically. To let the container driver take over, remove the host driver first, then delete both labels:

kubectl label node <NODE_NAME> rebellions.ai/npu.deploy.driver- rebellions.ai/npu.deploy.rbln-smd-

Once the labels are removed, the operator re-applies rebellions.ai/npu.deploy.driver=true, the driver DaemonSet is scheduled on the node, and the container driver installs and manages the kernel module.

Verifying which mode is active per node¶

After installation, check which mode each node is using:

kubectl get nodes -L rebellions.ai/npu.deploy.driver -L rebellions.ai/npu.deploy.rbln-smd

true: the operator-managed container driver is active on the node.
pre-installed: the operator detected a host driver and skipped container driver installation. rebellions.ai/npu.deploy.rbln-smd is also pre-installed, which indicates that RSMD is not deployed on this node.