Skip to content

Installation Guide

Refer to the installation guide below. Some SDK components, including the RBLN Compiler (rebel-compiler) and vllm-rbln, require an RBLN Portal account for installation. If you need assistance, please contact us.

1. RBLN Driver

Check driver status using rbln-smi

Run rbln-smi to confirm the driver is loaded and each NPU is reachable. A healthy system produces output like this:

$ rbln-smi
+-------------------------------------------------------------------------------------------------+
|                                 Device Information KMD ver: N/A                                 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   | Device  |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| 0   | RBLN-CA12 | rbln0   |  0000:51:00.0 |  25C |   6.1W  | P14  |    0.0B / 15.7GiB   |   0.0 |
| 1   | RBLN-CA12 | rbln1   |  0000:d8:00.0 |  25C |   6.1W  | P14  |    0.0B / 15.7GiB   |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
+-------------------------------------------------------------------------------------------------+
|                                       Context Information                                       |
+-----+---------------------+--------------+-----+----------+------+---------------------+--------+
| NPU | Process             |     PID      | CTX | Priority | PTID |            Memalloc | Status |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
| N/A | N/A                 |   N/A        | N/A |   N/A    |  N/A |                 N/A |  N/A   |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+

Once you've confirmed the tables look like the sample above, the kernel driver is in healthy condition, so you can skip the remaining driver steps and continue to → 2. RBLN Compiler.

If rbln-smi is missing or the tables are empty, stay in this section and finish the driver installation before you move on to RBLN Compiler section.

The RBLN Driver contains the Linux kernel driver and firmware, enabling the OS to recognize RBLN NPU devices. It is pre-installed on most cloud servers.

Key Features

  • Kernel Driver & Firmware: Enables the OS to interface with the RBLN NPU.
  • Package Formats: Available as Ubuntu (.deb) and RedHat (.rpm) packages.

Installation

  • Ubuntu 

    1
    $ sudo dpkg -i rbln_driver.deb

  • RedHat 

    1
    $ sudo rpm -i rbln_driver.rpm

Additional Notes

  • Root privileges are required for installation on on-premise servers.
  • If you need .deb or .rpm files, please contact us.

2. RBLN Compiler

The RBLN Compiler is the core component of the RBLN SDK, used to convert pre-trained models into an NPU-executable format. It also provides runtime environments (Python and C/C++) and profiling tools.

Note

A RBLN Portal account required for installation.

Key Features

  • Compile API: Converts pre-trained models into RBLN NPU-executable formats.
  • Runtime API:
    1. Python runtime: Installed via a .whl package.
    2. C/C++ runtime: Requires GPG key registration and apt-based installation.
      See C/C++ runtime installation for details.
  • Profiler Support: Offers performance analysis and optimization with the RBLN Profiler.

Installation

  • Distributed as a .whl package. Install using pip:
1
$ pip3 install -i https://pypi.rbln.ai/simple/ rebel-compiler

3. RBLN Optimum (HuggingFace Model Support)

optimum-rbln integrates HuggingFace APIs, making it easy to compile pre-trained transformers and diffusers models to run on RBLN NPUs.

Key Features

  • HuggingFace Integration: Seamlessly supports transformers and diffusers for RBLN-based inference.
  • Easy Deployment: Simplifies model loading and optimization for RBLN NPUs.

Installation

  • Distributed as a .whl package:
1
$ pip3 install optimum-rbln

4. RBLN Model Zoo

RBLN Model Zoo provides ready-to-use examples for compiling and running pre-trained models on RBLN NPUs. It serves as a reference for adapting custom models.

Key Features

  • Pre-trained Models: Contains a diverse collection of scripts for various popular pre-trained models.
  • Implementation Guides: Offers step-by-step instructions on how to develop model compilation and execution scenarios using RBLN NPUs.

Installation

  • Hosted on GitHub. Clone the repository with:
1
$ git clone --recursive https://github.com/rebellions-sw/rbln-model-zoo.git

5. Serving Frameworks Support

RBLN NPUs integrate with popular serving solutions, including vLLM, Nvidia Triton Inference Server, and TorchServe.

Key Features

  • vLLM (vllm-rbln)
    1. vLLM plugin for serving large language models (LLMs) on RBLN NPUs.
    2. Distributed as a .whl package.
  • Nvidia Triton Inference Server
    1. Refer to Nvidia Triton Inference Server for configuration details.
  • TorchServe Support
    1. Refer to TorchServe for installation and usage instructions.

Installation

Congratulations on setting up the RBLN SDK. You can now run PyTorch and TensorFlow models on RBLN NPUs.

Explore Tutorials for further understanding on how to use the RBLN SDK.