Skip to content

Upgrading the RBLN NPU Operator

This document explains how to upgrade the operator chart, which manages all RBLN components, to a newer version. For driver version rollout (the RBLNDriver lifecycle), see NPU Driver Upgrade Workflow.

Performing an Upgrade

The chart synchronizes the operator CRDs automatically, so upgrading requires only a single helm upgrade command. Run it with the same release name and namespace you used during installation, and pin the new version explicitly so the upgrade is reproducible:

1
2
3
4
$ helm upgrade rbln-npu-operator \
     oci://docker.io/rebellions/rbln-npu-operator-chart \
     --version 0.4.5 \
     --namespace rbln-system

To keep the custom Helm values used during installation, pass -f my-values.yaml or reapply the same --set overrides.

Available version tags are listed on the chart page on Docker Hub.

Synchronizing CRDs Automatically

Helm applies the CRDs bundled with the chart only on the first helm install and never updates them on helm upgrade. Because the operator's CRD schema evolves across releases, the chart closes this gap with a pre-install/pre-upgrade hook Job that applies the current CRDs server-side on every install and upgrade. The hook runs the operator image's crd-apply binary against the CRDs baked into the image at /opt/rbln/crds, so it needs no separate kubectl image and no manual kubectl apply step.

The hook is enabled by default (crds.upgrade.enabled: true) and needs no configuration. Because the operator image runs as non-root without a fixed user ID, the hook works on both vanilla Kubernetes and OpenShift. On OpenShift, Security Context Constraints assign an in-range UID.

The chart never removes CRDs, including on helm uninstall, so existing custom resources survive upgrades.

Applying CRDs Manually

If a GitOps tool such as Argo CD or Flux already reconciles the CRDs, disable the hook so the two do not conflict:

1
2
3
crds:
  upgrade:
    enabled: false

With the hook disabled, apply the updated CRDs yourself before running helm upgrade. Pin the release tag and apply the two CRDs managed by the chart: RBLNClusterPolicy and RBLNDriver. --server-side is required because of CRD size and field ownership constraints:

1
2
3
4
5
$ kubectl apply --server-side --force-conflicts -f \
  https://raw.githubusercontent.com/RBLN-SW/rbln-npu-operator/v0.4.5/config/crd/bases/rebellions.ai_rblnclusterpolicies.yaml

$ kubectl apply --server-side --force-conflicts -f \
  https://raw.githubusercontent.com/RBLN-SW/rbln-npu-operator/v0.4.5/config/crd/bases/rebellions.ai_rblndrivers.yaml

Use the same tag for both CRDs and the --version flag in the helm upgrade command.

Apply CRDs first when schema changes

When the hook is disabled and the new release includes CRD schema changes, apply the updated CRDs before running helm upgrade. Otherwise, schema validation can fail or new fields can be omitted.

Verifying the upgrade

Wait for the cluster policy and driver custom resources to reconverge to ready:

1
2
3
4
5
$ kubectl wait --for=jsonpath='{.status.state}'=ready \
    rblnclusterpolicy --all --timeout=10m

$ kubectl wait --for=jsonpath='{.status.state}'=ready \
    rblndriver --all --timeout=10m

For workload-type progress during the upgrade, query status.workloads[]. See Verifying the Installation for the full command.