Installing the RBLN NPU Operator in a Disconnected or Air-gapped Environment¶

Overview¶

Install the RBLN NPU Operator on an OpenShift cluster without direct internet access. Mirror the required catalogs and images to an internal registry before installing the Operator.

Topology A — Mirroring through controlled egress¶

The mirror host uses an approved external connection to download images and push them to the internal registry.

Topology B — Air gap (removable media)¶

Download images on an external network, transfer them on removable media, and import them into the internal registry.

Prerequisites¶

An OpenShift Container Platform cluster covered by the support matrix and cluster-admin privileges
An internal registry supporting Docker v2-2, reachable from every cluster node, and a TLS certificate trusted by the cluster
A mirror host with oc and oc-mirror, and credentials for the source and internal registries
A Red Hat pull secret and an RBLN Portal account
Worker nodes equipped with ATOM™ or Rebel100™ (PCI vendor ID 1eff)

Mirroring Container Images¶

Mirror the NPU Operator and NFD catalogs, plus the driver and RSMD images. Match the catalog tag (v4.21) to the cluster's OpenShift version.

ImageSetConfiguration¶

Save the example for your configuration as imageset-config.yaml. Select driver versions from the support matrix, use the node's kernel and RHEL versions, and confirm image availability through RBLN Portal.

Include atom or rebel100 in the driver image paths under additionalImages. Use rebellions/rbln-driver for spec.image in each RBLNDriver instance. See Driver image selection for the path rules.

ATOMRebel100Mixed configuration

imageset-config.yaml
apiVersion: mirror.openshift.io/v2alpha1
kind: ImageSetConfiguration
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/certified-operator-index:v4.21
      packages:
        - name: rbln-npu-operator
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.21
      packages:
        - name: nfd
  additionalImages:
    - name: repo.rebellions.ai/rebellions/atom/rbln-driver:<ATOM_DRIVER_VERSION>-<ATOM_KERNEL_VERSION>-rhel<ATOM_RHEL_VERSION>
    - name: repo.rebellions.ai/rebellions/rbln-smd:<ATOM_DRIVER_VERSION>

imageset-config.yaml
apiVersion: mirror.openshift.io/v2alpha1
kind: ImageSetConfiguration
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/certified-operator-index:v4.21
      packages:
        - name: rbln-npu-operator
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.21
      packages:
        - name: nfd
  additionalImages:
    - name: repo.rebellions.ai/rebellions/rebel100/rbln-driver:<REBEL100_DRIVER_VERSION>-<REBEL100_KERNEL_VERSION>-rhel<REBEL100_RHEL_VERSION>
    - name: repo.rebellions.ai/rebellions/rbln-smd:<REBEL100_DRIVER_VERSION>

imageset-config.yaml
apiVersion: mirror.openshift.io/v2alpha1
kind: ImageSetConfiguration
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/certified-operator-index:v4.21
      packages:
        - name: rbln-npu-operator
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.21
      packages:
        - name: nfd
  additionalImages:
    - name: repo.rebellions.ai/rebellions/atom/rbln-driver:<ATOM_DRIVER_VERSION>-<ATOM_KERNEL_VERSION>-rhel<ATOM_RHEL_VERSION>
    - name: repo.rebellions.ai/rebellions/rbln-smd:<ATOM_DRIVER_VERSION>
    - name: repo.rebellions.ai/rebellions/rebel100/rbln-driver:<REBEL100_DRIVER_VERSION>-<REBEL100_KERNEL_VERSION>-rhel<REBEL100_RHEL_VERSION>
    - name: repo.rebellions.ai/rebellions/rbln-smd:<REBEL100_DRIVER_VERSION>

Match each RSMD image tag to its driver version, listing each tag only once. Include driver images for every required combination of NPU family, version, OS, and kernel.

Node kernel version match

Driver images must match the node kernel version. Mirror the matching image whenever the kernel changes.

Checking the kernel and RHEL versions¶

Check the kernel version on each NPU worker node.

Rebel100™ATOM™

$ oc get nodes -o custom-columns='NAME:.metadata.name,KERNEL-VERSION:.status.nodeInfo.kernelVersion'
NAME            KERNEL-VERSION
node-rebel100   5.14.0-570.107.1.el9_6.x86_64

$ oc get nodes -o custom-columns='NAME:.metadata.name,KERNEL-VERSION:.status.nodeInfo.kernelVersion'
NAME        KERNEL-VERSION
node-atom   5.14.0-570.107.1.el9_6.x86_64

In this example, el9_6 indicates RHEL 9.6. Use the queried values in each family's image tags.

When the cluster is inaccessible

Check the kernel and RHEL versions in the driver-toolkit image for the OpenShift release in use.

export OCP_VERSION='<OCP_VERSION>'
DTK=$(oc adm release info --image-for=driver-toolkit \
  "quay.io/openshift-release-dev/ocp-release:${OCP_VERSION}-x86_64")
podman run --rm "$DTK" cat /etc/driver-toolkit-release.json

Running oc-mirror¶

Log in to the source registries on the external mirror host and to the internal registry on the internal mirror host. For Topology A, log in to all registries on the same host.

docker login registry.redhat.io
docker login repo.rebellions.ai        # RBLN Portal account
docker login registry.internal:8443    # internal registry

Topology A — Mirroring directly to the internal registry¶

oc-mirror --v2 --config imageset-config.yaml --remove-signatures \
    --workspace file://./workspace docker://registry.internal:8443

Topology B — Via disk (air gap)¶

Save images to an archive on the external mirror host.

oc-mirror --v2 --config imageset-config.yaml --remove-signatures \
    file://./archive

Transfer ./archive and imageset-config.yaml to the internal mirror host on removable media.

Import the images into the internal registry.

oc-mirror --v2 --config imageset-config.yaml --from file://./archive \
    docker://registry.internal:8443

Configuring the Mirror Registry¶

Configure cluster credentials, CA trust, and image mirror paths.

Update the global pull secret

Add the internal registry credentials to the cluster global pull secret.

oc get secret/pull-secret -n openshift-config \
    --template='{{index .data ".dockerconfigjson" | base64decode}}' > pull-secret.json
oc registry login --registry="registry.internal:8443" \
    --auth-basic="<username>:<password>" --to=pull-secret.json
oc set data secret/pull-secret -n openshift-config \
    --from-file=.dockerconfigjson=pull-secret.json

Configure trust for the registry CA

Add the internal registry CA to the cluster trust bundle.

oc create configmap registry-ca -n openshift-config \
    --from-file=registry.internal..8443=rootCA.pem
oc patch image.config.openshift.io/cluster --type=merge \
    -p '{"spec":{"additionalTrustedCA":{"name":"registry-ca"}}}'

Apply the generated cluster resources

Apply the generated ImageDigestMirrorSet, ImageTagMirrorSet, and CatalogSource resources. Use ./workspace for <WORKSPACE_DIR> in Topology A and ./archive in Topology B.
1
oc apply -f <WORKSPACE_DIR>/working-dir/cluster-resources/

Disable the default OperatorHub sources

Disable the default sources so OLM uses the mirrored catalogs instead of external catalogs.

oc patch OperatorHub cluster --type json \
    -p '[{"op":"add","path":"/spec/disableAllDefaultSources","value":true}]'

Verifying the mirrored catalog¶

Before installing the Operator, confirm that the mirrored CatalogSource is working.

oc get catalogsource -n openshift-marketplace
oc get pods -n openshift-marketplace
oc get packagemanifests -n openshift-marketplace | grep -e rbln-npu-operator -e nfd

Confirm that the catalog Pod is Running and that the PackageManifest entries for rbln-npu-operator and nfd are listed.

Installation Steps¶

After mirroring images and configuring the registry, follow the RBLN NPU Operator Installation Guide for OpenShift. Set spec.source in the NFD Operator and RBLN NPU Operator Subscriptions to the appropriate mirrored CatalogSource name. Check the names in Verifying the mirrored catalog.

spec:
  source: cs-certified-operator-index-v4-21

For the NFD Operator, use the NFD catalog name, such as cs-redhat-operator-index-v4-21.

For RHCOS nodes based on RHEL 10.2 and equipped with Rebel100™, apply the pciehp unbinding configuration before deploying the driver.

Driver instances for each NPU family¶

For a single family, set spec.version in the installation guide's driver.json to the mirrored driver version.

For a mixed configuration, create two instances from the installation guide's driver.json. Run the following commands instead of oc apply -f driver.json, using the mirrored driver versions.

export ATOM_DRIVER_VERSION='<ATOM_DRIVER_VERSION>'
export REBEL100_DRIVER_VERSION='<REBEL100_DRIVER_VERSION>'
jq --arg version "$ATOM_DRIVER_VERSION" '
  .metadata.name = "rbln-driver-atom"
  | .spec.version = $version
  | .spec.nodeSelector = {"rebellions.ai/npu.family": "atom"}' \
  driver.json > driver-atom.json
jq --arg version "$REBEL100_DRIVER_VERSION" '
  .metadata.name = "rbln-driver-rebel100"
  | .spec.version = $version
  | .spec.nodeSelector = {"rebellions.ai/npu.family": "rebel100"}' \
  driver.json > driver-rebel100.json
oc apply -f driver-atom.json -f driver-rebel100.json

See Running Multiple Driver Versions for node selection and version settings.

Verifying the Installation¶

Check that the RBLNClusterPolicy instance and each family's RBLNDriver instance report ready. The output below shows the mixed configuration created above, using ATOM™ 3.4.1 and Rebel100™ 3.5.1 with one node per family.

$ oc get rblnclusterpolicy \
  -o 'custom-columns=NAME:.metadata.name,STATE:.status.state'
NAME                  STATE
rbln-cluster-policy   ready

$ oc get rblndriver \
  -o 'custom-columns=NAME:.metadata.name,VERSION:.spec.version,STATE:.status.state,READY:.status.readyNodes,DESIRED:.status.desiredNodes'
NAME                   VERSION   STATE   READY   DESIRED
rbln-driver-atom       3.4.1     ready   1       1
rbln-driver-rebel100   3.5.1     ready   1       1

oc get pods -n rbln-system -o wide

Confirm that each instance's VERSION matches the mirrored version and that the driver and RSMD Pods are Ready.