Skip to content

Device Monitoring (rbln-smi)

rbln-smi is a command-line interface (CLI) utility for monitoring and managing RBLN NPUs. It supports:

  • NPU status monitoring (temperature, power, utilization, memory)
  • Context and process inspection
  • System topology inspection
  • RSD group management and group-level settings

rbln-smi is included in the RBLN Driver package. For the full, version-specific option reference, run rbln-smi --help.

Note

rbln-stat is deprecated and replaced by rbln-smi. Existing scripts using rbln-stat may still work, but new users should use rbln-smi.

Quick Start

$ rbln-smi

Expected output (example)

Monitor output (example)
+-------------------------------------------------------------------------------------------------+
|                                Device Information KMD ver: 3.5.1                                |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CR13 | rbln0   |  0000:03:00.0 |  38C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 1   | RBLN-CR13 | rbln1   |  0000:04:00.0 |  36C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 2   | RBLN-CR13 | rbln2   |  0000:55:00.0 |  35C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 3   | RBLN-CR13 | rbln3   |  0000:56:00.0 |  37C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 4   | RBLN-CR13 | rbln4   |  0000:83:00.0 |  34C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 5   | RBLN-CR13 | rbln5   |  0000:84:00.0 |  36C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 6   | RBLN-CR13 | rbln6   |  0000:d3:00.0 |  33C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 7   | RBLN-CR13 | rbln7   |  0000:d4:00.0 |  35C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
+-------------------------------------------------------------------------------------------------+
|                                       Context Information                                       |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
| NPU |       Process       |      PID     |    CTX    | Priority | PTID |    Memalloc   | Status |
+=====+=====================+==============+===========+==========+======+===============+========+
| N/A |         N/A         |      N/A     |    N/A    |    N/A   |  N/A |           N/A |   N/A  |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
Monitor output (example)
+-------------------------------------------------------------------------------------------------+
|                                Device Information KMD ver: 3.4.1                                |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CA25 | rbln0   |  0000:05:00.0 |  37C |  75.9W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 1   |           | rbln1   |  0000:06:00.0 |  35C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 2   |           | rbln2   |  0000:07:00.0 |  36C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 3   |           | rbln3   |  0000:08:00.0 |  34C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 4   | RBLN-CA25 | rbln4   |  0000:0c:00.0 |  33C |  74.2W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 5   |           | rbln5   |  0000:0d:00.0 |  35C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 6   |           | rbln6   |  0000:0e:00.0 |  34C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 7   |           | rbln7   |  0000:0f:00.0 |  36C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
+-------------------------------------------------------------------------------------------------+
|                                       Context Information                                       |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
| NPU |       Process       |      PID     |    CTX    | Priority | PTID |    Memalloc   | Status |
+=====+=====================+==============+===========+==========+======+===============+========+
| N/A |         N/A         |      N/A     |    N/A    |    N/A   |  N/A |           N/A |   N/A  |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+

Key Concepts and Terminology

Device selection

Use -d, --device <spec> to target specific NPUs. A spec is a comma-separated list of tokens, and each token is one of:

Token Example Selects
Device id rbln3 or 3 That physical NPU
Range rbln0..3 or 0..3 Physical functions 0 through 3, inclusive
All all Every device

The rbln-prefixed and bare forms are equivalent, and the names match the Device column (for example: rbln0, rbln1).

Ranges use .., not -: write 0..3, not 0-3. rbln-smi selects physical NPUs only — a virtual-function row appears in the listing but cannot be selected with -d.

The order of tokens in the spec does not change the order of the listing.

Output formats

  • Table (default): Human-readable summary for devices and contexts.
  • JSON (-j): Machine-readable output.
  • Query (-q): Space-separated (CSV-like) output suitable for scripts.

Common columns and performance state

The default monitor output typically includes:

Column Meaning
Name NPU card model (for example: RBLN-CA25, RBLN-CR13).
Power Power consumption.
Perf Performance state (P-state).
Temp Temperature (°C).
Util Utilization.
PID Process ID.
CTX Context ID.
Memalloc Allocated memory.

For the authoritative product/family mapping and supported card list, see Support Matrix.

The Perf column reports a P-state. Common interpretations include:

P-state Clock (Neural Engine) PCIe Note
P2 Nominal Gen5 -
P4 Nominal Gen4 -
P6 Half Gen4 -
P10 Half (No update) Thermal Throttling
P12 Minimal (No update) System Abort (Hang)
P14 Off (No update) Idle

Command Reference

Note

Some operations require sudo. In particular, subcommands such as group, tdr, timeout, sort, and mknod require sudo.

General usage

Common invocation patterns:

$ rbln-smi [global options]
$ rbln-smi --topo [options]
$ sudo rbln-smi <subcommand> [arguments]

Global options

The following options are available across all command modes:

Option Description
-h, --help Display help information.
-b, --byte-format Display values in raw units instead of human-readable units.
-j, --json Render the result as JSON.
-q, --query Print data in a space-separated (CSV-like) format.
-qd, --query-device <columns> Select specific device columns when using query mode.
-qc, --query-context <columns> Select specific context columns when using query mode.
-t, --topo Show device/system topology.
-L, --list List NPUs and their UUIDs.
-d, --device <spec> Choose NPUs by name, range, list, or all (see Device selection).
-g, --group Display output organized by RSD groups.
-v, --version Print version information and exit.

CLI Examples

Basic commands

Summary

Default view. Shows a snapshot of device and context information.

Command

Command
$ rbln-smi [options]

Output (excerpt)

Monitor (excerpt)
Mon Nov 17 14:15:26 2025
+-------------------------------------------------------------------------------------------------+
|                                Device Information KMD ver: 3.5.1                                |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CR13 | rbln0   |  0000:03:00.0 |  38C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 1   | RBLN-CR13 | rbln1   |  0000:04:00.0 |  36C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 2   | RBLN-CR13 | rbln2   |  0000:55:00.0 |  35C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 3   | RBLN-CR13 | rbln3   |  0000:56:00.0 |  37C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 4   | RBLN-CR13 | rbln4   |  0000:83:00.0 |  34C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 5   | RBLN-CR13 | rbln5   |  0000:84:00.0 |  36C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 6   | RBLN-CR13 | rbln6   |  0000:d3:00.0 |  33C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
| 7   | RBLN-CR13 | rbln7   |  0000:d4:00.0 |  35C |  100.0W |  P14 |   0.0B / 140.0GiB   |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
+-------------------------------------------------------------------------------------------------+
|                                       Context Information                                       |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
| NPU |       Process       |      PID     |    CTX    | Priority | PTID |    Memalloc   | Status |
+=====+=====================+==============+===========+==========+======+===============+========+
| N/A |         N/A         |      N/A     |    N/A    |    N/A   |  N/A |           N/A |   N/A  |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
Monitor (excerpt)
Mon Nov 17 14:15:26 2025
+-------------------------------------------------------------------------------------------------+
|                                Device Information KMD ver: 3.4.1                                |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CA25 | rbln0   |  0000:05:00.0 |  37C |  75.9W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 1   |           | rbln1   |  0000:06:00.0 |  35C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 2   |           | rbln2   |  0000:07:00.0 |  36C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 3   |           | rbln3   |  0000:08:00.0 |  34C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 4   | RBLN-CA25 | rbln4   |  0000:0c:00.0 |  33C |  74.2W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 5   |           | rbln5   |  0000:0d:00.0 |  35C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 6   |           | rbln6   |  0000:0e:00.0 |  34C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 7   |           | rbln7   |  0000:0f:00.0 |  36C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
+-------------------------------------------------------------------------------------------------+
|                                       Context Information                                       |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+
| NPU |       Process       |      PID     |    CTX    | Priority | PTID |    Memalloc   | Status |
+=====+=====================+==============+===========+==========+======+===============+========+
| N/A |         N/A         |      N/A     |    N/A    |    N/A   |  N/A |           N/A |   N/A  |
+-----+---------------------+--------------+-----------+----------+------+---------------+--------+

Values in these captures are illustrative; the layout matches the tool's output.

Summary

Prints the result in JSON format.

Command

Command
$ rbln-smi -j

Output (excerpt)

JSON output (excerpt)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
  "KMD_version": "3.5.1",
  "devices": [
    {
      "npu": 0,
      "name": "RBLN-CR13",
      "sid": "SAMPLE_SID_0000",
      "uuid": "11111111-2222-3333-4444-555555555555",
      "device": "rbln0",
      "temperature": "24C",
      "card_power": "100000000uW",
      "pstate": "P14",
      "memory": { "used": "0", "total": "150323855360" },
      "util": "0.0"
    }
  ],
  "contexts": []
}
JSON output (excerpt)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
  "KMD_version": "3.4.1",
  "devices": [
    {
      "npu": 0,
      "name": "RBLN-CA25",
      "sid": "SAMPLE_SID_0000",
      "uuid": "11111111-2222-3333-4444-555555555555",
      "device": "rbln0",
      "temperature": "24C",
      "card_power": "41448540uW",
      "pstate": "P14",
      "memory": { "used": "0", "total": "16877879296" },
      "util": "0.0"
    }
  ],
  "contexts": []
}

Summary

Prints the result in a space-separated (CSV-like) format.

Command

Command
$ rbln-smi -q

Output (excerpt)

Query output (excerpt)
driver_version:
  3.5.1
devices:
 npu      name        sid             uuid                                  device status ...
   0 RBLN-CR13 SAMPLE_SID_0000 11111111-2222-3333-4444-555555555555          rbln0 normal ...
   1 RBLN-CR13 SAMPLE_SID_0001 66666666-7777-8888-9999-AAAAAAAAAAAA          rbln1 normal ...
Query output (excerpt)
driver_version:
  3.4.1
devices:
 npu      name        sid             uuid                                  device status ...
   0 RBLN-CA25 SAMPLE_SID_0000 11111111-2222-3333-4444-555555555555          rbln0 normal ...
   1 RBLN-CA25 SAMPLE_SID_0001 66666666-7777-8888-9999-AAAAAAAAAAAA          rbln1 normal ...

Summary

Restricts output to the specified NPUs only.

Command

Command
$ rbln-smi -d rbln0,rbln1

Output (excerpt)

Device filter (excerpt)
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CR13 | rbln0   |  0000:05:00.0 |  24C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 1   | RBLN-CR13 | rbln1   |  0000:06:00.0 |  25C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
Device filter (excerpt)
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 0   | RBLN-CA25 | rbln0   |  0000:05:00.0 |  24C |  41.6W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 1   |           | rbln1   |  0000:06:00.0 |  25C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+

Summary

Prints device and system topology information, including the distance matrix.

Command

Command
$ rbln-smi --topo [--device <ids>]

Output (excerpt)

Topology (example)
Hardware Topology
Device Distance  n0  n1  n2  n3
rbln0        n0   0   4   4   4
rbln1        n1   4   0   4   4
rbln2        n2   4   4   0   4
rbln3        n3   4   4   4   0

Summary

Lists NPUs and their UUIDs.

Command

Command
$ rbln-smi -L

Output (excerpt)

-L (excerpt)
NPU 0: UUID: 11111111-2222-3333-4444-555555555555 (example)
NPU 1: UUID: 66666666-7777-8888-9999-AAAAAAAAAAAA (example)

Subcommands (sudo required)

Summary

Create or destroy RSD groups.

Command

Command
$ sudo rbln-smi group [-c <group_id> -a <npu_ids>] [-d <group_ids>]

Options

Argument Description
-c, --create <group_id> Create a new RSD group (use with -a). Specifying all assigns one group per device.
-a, --attach <npu_ids> Attach NPUs to the new group. Comma-separated NPU ids only.
-d, --destroy <spec> Remove one or more RSD groups. NPUs from a removed group are merged back into group 0. Accepts a group id, an inclusive range (1..3), a comma-separated list (1,2), or all to remove every group.

Note

When multiple IDs are passed to --destroy, each group is processed independently. A failure on one ID does not stop the others; failed IDs are reported on stderr as skip destroying group <id>: <reason> and the command exits non-zero with a final failed to destroy group(s): <ids> message that lists every ID that did not complete.

Example session

Group workflow (example)
$ sudo rbln-smi group -c 1 -a 0,1
RSD group 1 created with devices [0,1].

$ rbln-smi --group
+-----+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| Grp | NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 1   | 0   | RBLN-CR13 | rbln0   |  0000:05:00.0 |  23C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
| 1   | 1   | RBLN-CR13 | rbln1   |  0000:06:00.0 |  25C |  100.0W |  P14 |    0.0B / 140.0GiB  |   0.0 |
+-----+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+

$ sudo rbln-smi group -d 1
RSD group 1 destroyed.

# Destroy multiple groups in a single call (list or range).
$ sudo rbln-smi group -d 1,2,3
RSD group 1 destroyed.
RSD group 2 destroyed.
RSD group 3 destroyed.

$ sudo rbln-smi group -d 1..3
RSD group 1 destroyed.
RSD group 2 destroyed.
RSD group 3 destroyed.

# Per-ID aggregation: a failure on one ID does not block the rest.
$ sudo rbln-smi group -d 1,99,3
RSD group 1 destroyed.
skip destroying group 99: <reason>
RSD group 3 destroyed.
failed to destroy group(s): 99

$ sudo rbln-smi group -c all
All RSD groups created.

$ sudo rbln-smi group -d all
All RSD groups destroyed.
Group workflow (example)
$ sudo rbln-smi group -c 1 -a 0,1
RSD group 1 created with devices [0,1].

$ rbln-smi --group
+-----+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+
| Grp | NPU |    Name   |  Device |   PCI BUS ID  | Temp |  Power  | Perf |  Memory(used/total) |  Util |
+=====+=====+===========+=========+===============+======+=========+======+=====================+=======+
| 1   | 0   | RBLN-CA25 | rbln0   |  0000:05:00.0 |  23C |  41.0W  |  P14 |    0.0B / 15.7GiB   |   0.0 |
| 1   | 1   |           | rbln1   |  0000:06:00.0 |  25C |         |  P14 |    0.0B / 15.7GiB   |   0.0 |
+-----+-----+-----------+---------+---------------+------+---------+------+---------------------+-------+

$ sudo rbln-smi group -d 1
RSD group 1 destroyed.

# Destroy multiple groups in a single call (list or range).
$ sudo rbln-smi group -d 1,2,3
RSD group 1 destroyed.
RSD group 2 destroyed.
RSD group 3 destroyed.

$ sudo rbln-smi group -d 1..3
RSD group 1 destroyed.
RSD group 2 destroyed.
RSD group 3 destroyed.

# Per-ID aggregation: a failure on one ID does not block the rest.
$ sudo rbln-smi group -d 1,99,3
RSD group 1 destroyed.
skip destroying group 99: <reason>
RSD group 3 destroyed.
failed to destroy group(s): 99

$ sudo rbln-smi group -c all
All RSD groups created.

$ sudo rbln-smi group -d all
All RSD groups destroyed.

Summary

Enable or disable Timeout Detection and Recovery (TDR) for an RSD group. When TDR is enabled, the driver detects a hung job once the per-group timeout (see timeout) expires and runs an NPU reset cycle to recover the device; when disabled, an expired job is timed out but the NPU is not reset automatically.

Command

Command
$ sudo rbln-smi tdr -g <spec> -v <value>

Notes

<spec> takes a group id, an inclusive range (1..3), or a comma-separated list (1,2). Unlike group --destroy it does not accept all — a blanket target has to be spelled out. A group range must also run low-to-high; 3..1 is rejected, where a device range may run either way (see Device selection). <value> is 0 (disable) or 1 (enable); any non-zero value is treated as enable. The reset threshold itself is configured separately with the timeout subcommand. On success the command prints one Set tdr = <value> for rsd<N> line per group.

Output (example)

TDR set (example)
$ sudo rbln-smi tdr -g 1 -v 1
Set tdr = 1 for rsd1

Summary

Set the TDR timeout — the time (in seconds) the driver waits for a job to complete before treating it as hung. The threshold applies per RSD group and is consumed by the TDR mechanism described under tdr.

Command

Command
$ sudo rbln-smi timeout -g <spec> -v <value>

Notes

<spec> selects RSD groups exactly as tdr does, all included in what it rejects. <value> is a non-negative integer in seconds. 0 is a special value: the driver treats it as no timeout, so a job waits indefinitely. The reset cycle only runs when TDR is enabled — if tdr is 0, the timeout still trips but the NPU is not reset automatically. On success the command prints one Set timeout = <value> for rsd<N> line per group.

Output (example)

Timeout set (example)
$ sudo rbln-smi timeout -g 1 -v 10
Set timeout = 10 for rsd1

Summary

Sort NPU devices by PCI BDF and rebind them.

Command

Command
$ sudo rbln-smi sort

Notes

On success, this command typically prints no output.

Output (example)

Sort (example)
$ sudo rbln-smi sort
# (no output on success)

Summary

Create /dev/rbln* character device nodes for the NPUs visible through the RSD control device (--rsd). Useful in containers or initramfs environments where udev does not populate /dev/rbln* automatically. Idempotent — pre-existing nodes are silently skipped.

Command

Command
$ sudo rbln-smi mknod [--rsd <path>] [--dev-dir <path>]

Options

Argument Description
--rsd <path> RSD control device used to enumerate the NPUs. Default /dev/rsd0.
--dev-dir <path> Directory in which to create the device nodes. Default /dev.

Notes

Nodes are created with mode 0666, matching the host's udev defaults. The process umask is cleared while the nodes are created and restored on exit, so other files in --dev-dir are not affected.

Inside containers the caller needs CAP_MKNOD — the Linux capability that authorizes mknod(2) for character and block special files (see capabilities(7)); for example, run the container with --cap-add=MKNOD. Other mknod(2) errors (for example, EPERM when CAP_MKNOD is missing) are reported on stderr and the command exits non-zero.

Output (example)

Mknod first run (example)
$ sudo rbln-smi mknod
created /dev/rbln0
created /dev/rbln1
2 device(s) processed (2 created).
Mknod re-run (example)
$ sudo rbln-smi mknod
2 device(s) processed (0 created).
Mknod with custom dev dir (example)
$ sudo rbln-smi mknod --dev-dir /run/rbln-dev
created /run/rbln-dev/rbln0
created /run/rbln-dev/rbln1
2 device(s) processed (2 created).

Troubleshooting

Permission denied / subcommand requires sudo

Run the command with sudo (for example: sudo rbln-smi group -h).

No devices are shown

  • Confirm the driver is installed and NPUs are detected.
  • Try rbln-smi -L to list devices.

Topology output is missing

If --topo is not available or incomplete, check kernel/version requirements and try again with specific devices via --device <ids>.

See also