Bandwidth Benchmark (rblnBandwidthLatencyTest)¶

rblnBandwidthLatencyTest measures transfer performance between the host and NPUs, as well as between NPUs. It reports:

Bandwidth results (GB/s)
Latency results (us)
NPU connectivity (distance matrix) and topology context

The tool runs on every RBLN NPU, and the measurement options are shared across product families. Pinning memory to a specific chiplet requires a multi-chiplet device, currently Rebel100™; see Multi-chiplet memory affinity.

Quick Start¶

Default run (recommended)

1	`$ rblnBandwidthLatencyTest`

Environment and topology

1	`$ rblnBandwidthLatencyTest --test_env_info`

For concrete command+output pairs, see CLI Examples.
For --tc IDs and selection rules, see Test cases.

Key Concepts and Terminology¶

Transfer directions¶

H2D: host -> NPU
D2H: NPU -> host
D2D: NPU -> NPU (peer-to-peer)

Use --only_d2d when you want to focus on NPU-to-NPU transfers only.

Measurement configuration¶

--test_size: Payload size for the bandwidth tests (default: 512M). The latency tests do not read it.
--iteration_cnt: Measurement iterations.
--warmup: Run one warmup iteration before measurement.

Target selection¶

Identifiers
- NPU ID: Numeric index used by --npu_id_list (for example: 0, 1).
- Device label: Runtime label shown in output (for example: rbln0, rbln1).
Selection rules
- --npu_id_list: Comma-separated list of NPU IDs (for example: 0,1,2).
- For test cases 3-6, the set of NPUs involved depends on whether --npu_id_list is provided.

Multi-chiplet memory affinity¶

Rebel100™ (RBLN-CR13) carries several chiplets, each with its own DRAM. Context memory sits on chiplet 0 by default, and --src_cl/--dst_cl pin the source and destination buffers to another chiplet's DRAM instead. Use them to compare a transfer whose buffers are local to the path against one whose buffers are not.

--src_cl: pin source context memory to the given chiplet's DRAM.
--dst_cl: pin destination context memory to the given chiplet's DRAM.
Either option takes 15 (0xF) to turn affinity off.

The tool detects the capability at run time by querying the device's chiplet count. A single-chiplet device such as ATOM has nothing to pin to, so the run is reported as skipped rather than run; see Multi-chiplet options are skipped.

Test cases¶

The available test case IDs are printed by --test_list. The ID is the 0-based index of the entry in the list.

Based on the --test_list output shown later on this page:

ID (`--tc`)	`--test_list` entry	Purpose (high level)
`0`	`RBLN_PERF_TEST_ALL`	Run the default benchmark set (combined bandwidth + latency sweep).
`1`	`RBLN_PERF_BANDWIDTH_TEST_ALL`	Bandwidth-only sweep.
`2`	`RBLN_PERF_LATENCY_TEST_ALL`	Latency-only sweep.
`3`	`RBLN_PERF_BANDWIDTH_TEST`	Bandwidth test for a specific pair, or for the first device against all others if `--npu_id_list` is not provided.
`4`	`RBLN_PERF_LATENCY_TEST`	Latency test for a specific pair, or for the first device against all others if `--npu_id_list` is not provided.
`5`	`RBLN_PERF_BIDIR_BANDWIDTH_TEST`	Bidirectional bandwidth test between devices.
`6`	`RBLN_PERF_PARALLEL_BANDWIDTH_TEST`	Parallel bandwidth test across multiple devices.

You can run a specific test case by specifying its ID with the --tc option.

Note

If your environment prints a different --test_list order, treat the table above as a guide and rely on the --test_list output as the source of truth.

Command Reference¶

General usage¶

$ rblnBandwidthLatencyTest [options]
$ rblnBandwidthLatencyTest --tc <id> [options]

Global options¶

These options are accepted by the base command and the test-case execution path.

Common options

Option	Description
`-h, --help`	Show the help message and exit.
`-V, --version`	Show the version and exit.
`-t, --timeout <int>`	Set the per-job wait timeout in microseconds. Default: `30000000` (30 s).
`-i, --iteration_cnt <int>`	Set the iteration count.
`--warmup`	Run a warmup iteration before measurement.
`--test_size <str>`	Set the data size (e.g., `4K`, `128M`, `1G`). Default: `512M`. Must be smaller than `4G` except for `--tc 6`; a larger value silently falls back to the default.
`--npu_id_list <list>`	Provide a comma-separated list of NPU IDs (e.g., `0,1,2`). Up to 32 ids; a longer list is rejected.
`--only_d2d`	Execute only D2D transfers; skip H2D/D2H stages.
`--test_list`	Print the list of available test cases.
`--tc <int>`	Select the test case ID to run (0-based index). Use the index from `--test_list`.
`--test_env_info`	Print system and device information.
`--skip_test_env_info`	Skip printing environment information when running tests.

Advanced options

Option	Description
`-n, --device_name <int>`	NPU type required for context creation (`0`: any (default), `2`: ATOM, `5`: Rebel100™). This is not a device index; use `--npu_id_list` to select devices.
`--get_perf <int>`	No effect; accepted for compatibility.
`--priority <int>`	Set context priority (`0-2`: min/normal/high, `3`: kernel, `4`: real-time).
`-r, --hdma_read <int>`	Configure the HDMA direction (`0`: write, `1`: read).
`--numa_node <int>`	Force a NUMA node (default: `-1`).
`--data_pattern <int>`	Set the data pattern value.
`-f, --flags <int>`	Set context flags.
`-g, --group_id <int>`	Set the execution group ID. Default: `0`.

Multi-chiplet options

These options require a multi-chiplet device and are available on Rebel100™ only. See Multi-chiplet memory affinity.

Option	Description
`--src_cl <int>`	Pin source context memory to a chiplet. Valid values run from `0` to the device's chiplet count minus one (`0`-`3` on `RBLN-CR13`); `0` is the default and `15`/`0xF` turns affinity off.
`--dst_cl <int>`	Pin destination context memory to a chiplet. Valid values run from `0` to the device's chiplet count minus one (`0`-`3` on `RBLN-CR13`); `0` is the default and `15`/`0xF` turns affinity off.

Tip

If you are unsure about an option, start with Quick Start and rely on rblnBandwidthLatencyTest --help for the complete help text.

CLI Examples¶

Device enumeration is the one part of the output that differs by product family, so those blocks branch per family. Everything else on this page is shared.

Baseline (all tests)Test case listSystem & topologyBandwidth (tc 3)Latency (tc 4)

Summary

Output block	What it is
System + device enumeration	CPU and device list (typically includes PCI BDF and NUMA node).
`P2P Connectivity Distance`	NPU-to-NPU connectivity matrix (reported by the tool).
Bandwidth table	Throughput results (GB/s) across `HOST` and `rbln*` device labels.
Latency results	Synchronization latency in microseconds (us).

Command

Command

$ rblnBandwidthLatencyTest

Output (excerpt)

Rebel100™ATOM™

Default run — Rebel100™ (excerpt)

CPU : AMD EPYC 9355 32-Core Processor, Vendor ID : AuthenticAMD
Device :
0000:00:01.1/01:00.0
    0000:03:00.0 NODE 0, rbln0, RBLN-CR13(rev-02), pciDevID: 0x2130, NUMA: 0, IOMMU: 63
    0000:04:00.0 NODE 1, rbln1, RBLN-CR13(rev-02), pciDevID: 0x2130, NUMA: 0, IOMMU: 64
    ...
P2P Connectivity Distance
rbln0       0   4   4   4   8   8   8   8
rbln1       4   0   4   4   8   8   8   8
...

Device Write bandwidth (GB/s)
S \ D     HOST   rbln0   rbln1   rbln2   rbln3 ...
HOST      0.00   57.32   57.45   57.41   57.41 ...
rbln0    58.92    0.00   58.58   58.74   58.76 ...

Device P2P sync latency (us)
S \ D     HOST   rbln0   rbln1   rbln2   rbln3 ...
rbln0     0.00    0.00    2.76    2.76    2.78 ...

Default run — ATOM™ (excerpt)

CPU : AMD EPYC 9355 32-Core Processor, Vendor ID : AuthenticAMD
Device :
0000:00:01.1/01:00.0
    0000:05:00.0 NODE 0, rbln0, RBLN-CA25(rev-03), pciDevID: 0x1250, NUMA: 0, IOMMU: 88
    0000:06:00.0 NODE 1, rbln1, RBLN-CA25(rev-03), pciDevID: 0x1250, NUMA: 0, IOMMU: 89
    ...
P2P Connectivity Distance
rbln0       0   4   4   4   8   8   8   8 ...
rbln1       4   0   4   4   8   8   8   8 ...
...

Device Write bandwidth (GB/s)
S \ D     HOST   rbln0   rbln1   rbln2   rbln3 ...
HOST      0.00   50.04   50.13   50.14   50.12 ...
rbln0    59.13    0.00   58.74   58.94   58.93 ...

Device P2P sync latency (us)
S \ D     HOST   rbln0   rbln1   rbln2   rbln3 ...
rbln0     0.00    0.00    2.24    2.24    2.24 ...

Summary

Prints the list of available performance test cases.

Command

Command

$ rblnBandwidthLatencyTest --test_list

Output (excerpt)

--test_list (example)

List of tests:
-----------------

RBLN_PERF_TEST_ALL
RBLN_PERF_BANDWIDTH_TEST_ALL
RBLN_PERF_LATENCY_TEST_ALL
RBLN_PERF_BANDWIDTH_TEST
RBLN_PERF_LATENCY_TEST
RBLN_PERF_BIDIR_BANDWIDTH_TEST
RBLN_PERF_PARALLEL_BANDWIDTH_TEST

Summary

Prints CPU information and system topology details (PCI BDF, NUMA node, and PCIe link information).

Command

Command

$ rblnBandwidthLatencyTest --test_env_info

Output (excerpt)

Rebel100™ATOM™

--test_env_info — Rebel100™ (excerpt)

CPU : AMD EPYC 9355 32-Core Processor, Vendor ID : AuthenticAMD
Device :
0000:00:01.1/01:00.0
    0000:03:00.0 NODE 0, rbln0, RBLN-CR13(rev-02), pciDevID: 0x2130, NUMA: 0, IOMMU: 61
                                PCIe full bandwidth 32.0 GB/s (MAX: 32.0), PCIe Gen 5 (MAX: 5), 16 lane (MAX: 16)
                                MaxPayload 512 bytes, MaxReadReq 512 bytes
    0000:04:00.0 NODE 1, rbln1, RBLN-CR13(rev-02), pciDevID: 0x2130, NUMA: 0, IOMMU: 62
                                ...
0000:50:01.1/51:00.0
    0000:55:00.0 NODE 2, rbln2, RBLN-CR13(rev-02), pciDevID: 0x2130, NUMA: 0, IOMMU: 39
    ...

Device count, PCI addresses, and NUMA placement follow the host and card population.

--test_env_info — ATOM™ (excerpt)

CPU : AMD EPYC 9254 24-Core Processor, Vendor ID : AuthenticAMD
Device :
0000:00:01.1/01:00.0 PCI bridge: Broadcom / LSI PEX890xx PCIe Gen 5 Switch (rev b0)
    0000:05:00.0 NODE 0, rbln0, RBLN-CA25(rev-03), pciDevID: 0x1250, NUMA: 0, IOMMU: 70
                                PCIe full bandwidth 64.0 GB/s (MAX: 64.0), PCIe Gen 5 (MAX: 5), 16 lane (MAX: 16)
                                MaxPayload 512 bytes, MaxReadReq 512 bytes
...
    0000:ce:00.0 NODE 30, rbln30, RBLN-CA25(rev-03), pciDevID: 0x1250, NUMA: 1, IOMMU: 136
                                PCIe full bandwidth 64.0 GB/s (MAX: 64.0), PCIe Gen 5 (MAX: 5), 16 lane (MAX: 16)
                                MaxPayload 512 bytes, MaxReadReq 512 bytes
    0000:cf:00.0 NODE 31, rbln31, RBLN-CA25(rev-03), pciDevID: 0x1250, NUMA: 1, IOMMU: 137
                                PCIe full bandwidth 64.0 GB/s (MAX: 64.0), PCIe Gen 5 (MAX: 5), 16 lane (MAX: 16)
                                MaxPayload 512 bytes, MaxReadReq 512 bytes

Summary

Runs a single bandwidth test case (RBLN_PERF_BANDWIDTH_TEST) for a specific NPU pair.

Command

Command

$ rblnBandwidthLatencyTest --tc 3 --npu_id_list 0,1 --iteration_cnt 1 --test_size 1M --skip_test_env_info

Output (excerpt)

tc 3 (excerpt)

This test will run with size (0x100000)
==============================================================================
[      Run ] : RBLN_PERF_BANDWIDTH_TEST
==============================================================================

[Measure Iteration cnt : 1]
[Measure Buffer Size : 0x100000]

P2P Connectivity Distance
rbln0       0   4   4   4   8   ...
rbln1       4   0   4   4   8   ...

---------------------------------
Device bandwidth (GB/s) *Write
---------------------------------
[H2D] HOST -> rbln0 : MAX 14.71, MIN 14.71, AVG 14.71
[D2D] rbln0 -> rbln1 : MAX 10.29, MIN 10.29, AVG 10.29
[D2H] rbln1 -> HOST : MAX 19.41, MIN 19.41, AVG 19.41

==============================================================================
[     PASS ] : RBLN_PERF_BANDWIDTH_TEST
==============================================================================

Summary

Runs a single latency test case (RBLN_PERF_LATENCY_TEST) for a specific NPU pair.

Command

Command

$ rblnBandwidthLatencyTest --tc 4 --npu_id_list 0,1 --skip_test_env_info

Output (excerpt)

tc 4 (excerpt)

==============================================================================
[      Run ] : RBLN_PERF_LATENCY_TEST
==============================================================================

[Measure Iteration cnt : 1]
[Measure Buffer Size : 0x20000000]

P2P Connectivity Distance
rbln0       0   4   4   4   8   ...
rbln1       4   0   4   4   8   ...

---------------------------------
Device P2P sync latency (us)
---------------------------------
rbln0 -> rbln1 2.07 (us)

==============================================================================
[     PASS ] : RBLN_PERF_LATENCY_TEST
==============================================================================

Troubleshooting¶

Command timeout¶

Increase --timeout.
Reduce --test_size or --iteration_cnt to shorten the test.
Run --test_env_info first to confirm devices are detected and topology is printed.

Measurement variance¶

Use --warmup.
Increase --iteration_cnt and compare averages.
Keep the system load stable (pin workloads, avoid background jobs during benchmarking).

NPU-to-NPU-only output¶

Use --only_d2d to skip H2D/D2H stages and focus on NPU-to-NPU transfers.

Multi-chiplet options are skipped¶

A non-default --src_cl/--dst_cl needs a device with more than one chiplet. When any selected NPU has only one, the tool reports the device and marks the run skipped:

Unsupported device

Device 0 has 1 chiplet(s), affinity unsupported, skipping

Confirm the target is a multi-chiplet device. On a single-chiplet device such as ATOM, drop these options and run the defaults.
Check every NPU in --npu_id_list: the run is skipped if any one of them has a single chiplet.
A chiplet index beyond the device's chiplet count fails with chiplet index out of range instead of being skipped.

No devices are found¶

On a host whose RSD group is not the default (0), running without -g finds no devices at all. --test_env_info prints nothing after Device :, or a test ends like this:

Wrong running group

[RBLN_UMD_ioctl_ERR] rbln_rsd_get_dev_id_str: logical_dev_id is greater than device count(logical 0 dev_cnt 0)
[RBLN_UMD_api_ERR] rblnCreateContext: failed to init device
[RBLN_RSD_PERF_ERR] rbln_rsd_perf_create_ctx: failed to create rsd context (rc -22)

Check the group ID the devices belong to with rbln-smi -g, or read /sys/class/rebellions/rsd/rsd_group.
Pass that ID as -g <id> and run again.

Bandwidth Benchmark (rblnBandwidthLatencyTest)¶

Quick Start¶

Related sections¶

Key Concepts and Terminology¶

Transfer directions¶

Measurement configuration¶

Target selection¶

Multi-chiplet memory affinity¶

Test cases¶

Command Reference¶

General usage¶

Global options¶

CLI Examples¶

Troubleshooting¶

Command timeout¶

Measurement variance¶

NPU-to-NPU-only output¶

Multi-chiplet options are skipped¶

No devices are found¶

See also¶