Profiling on Rebel100™¶

torch.rbln.explain() tells you what the backend did that you did not write: fallbacks, recompiles, host round-trips. It does not tell you how long the device spent on each program. torch.profiler measures that. On Rebel100™ NPUs, setting RBLN_PROFILER=1 before the process starts makes the runtime record device activity, and RBLN PyTorch's Kineto bridge feeds it into the trace as PrivateUse1 events next to the ATen calls. On ATOM™ NPUs the bridge stays inactive and the same trace shows host activity only.

Prerequisites¶

• A host with a Rebel100™ NPU (an RBLN-CR card) and the RBLN Driver installed. On an ATOM™ host the script runs, but the trace has no device rows.
• Python: 3.10–3.14
• torch, torch-rbln, and rebel-compiler installed as described in Installation
• A trace viewer for Step 3: Perfetto or chrome://tracing

Step 1: Record a trace¶

profile_matmul.py warms up one matrix multiply, then profiles three more with both activities enabled, prints the aggregated table, lists the device events, and writes a Chrome trace:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import torch
from torch.profiler import DeviceType, ProfilerActivity, profile

x = torch.randn(512, 512, dtype=torch.float16, device="rbln")
with torch.inference_mode():
    (x @ x).cpu()  # warm-up: compile outside the profiled region

activities = [ProfilerActivity.CPU, ProfilerActivity.PrivateUse1]
with torch.inference_mode(), profile(activities=activities) as prof:
    for _ in range(3):
        (x @ x).cpu()

print(prof.key_averages().table(sort_by="cpu_time_total", row_limit=10))

device_events = [e for e in prof.events() if e.device_type == DeviceType.PrivateUse1]
print(
    f"{len(device_events)} device events, for example:",
    [e.name for e in device_events[:4]],
)

prof.export_chrome_trace("trace.json")  # open in chrome://tracing or Perfetto

RBLN_PROFILER is read by the runtime when it starts, so set it on the command line rather than inside the script:

$ RBLN_PROFILER=1 python3 profile_matmul.py

If the table has no Self RBLN columns and the script prints 0 device events, RBLN_PROFILER=1 was not set when the process started.

Two things print before the table. The runtime prints its own RBLN Profiler Generation Status Report and the name of the profile file it wrote (rbln_<timestamp>.pb); that file is the input of the RBLN Profiler tools and is not used here. Then the torch.profiler output follows. Abbreviated; times differ per machine:

-----------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------
                         Name    Self CPU %      Self CPU   CPU total %     CPU total  CPU time avg     Self RBLN   Self RBLN %    RBLN total  RBLN time avg    # of Calls
-----------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------
                     aten::mm        36.69%       1.590ms       161.84%       7.015ms       1.169ms       0.000us         0.00%       0.000us       0.000us             6
                 aten::matmul         0.11%       4.919us        81.31%       3.524ms       1.175ms       0.000us         0.00%       0.000us       0.000us             3
                    aten::abs        10.76%     466.546us        21.34%     925.134us      77.095us       0.000us         0.00%       0.000us       0.000us            12
                  aten::isinf         0.36%      15.809us        17.35%     751.945us     125.324us       0.000us         0.00%       0.000us       0.000us             6
                     aten::to         0.22%       9.583us        16.01%     694.167us      46.278us       0.000us         0.00%       0.000us       0.000us            15
               aten::_to_copy         0.45%      19.339us        15.79%     684.584us      45.639us       0.000us         0.00%       0.000us       0.000us            15
                  aten::copy_         0.46%      19.738us        15.17%     657.494us      31.309us       0.000us         0.00%       0.000us       0.000us            21
             aten::_copy_from        14.71%     637.756us        14.71%     637.756us      70.862us       0.000us         0.00%       0.000us       0.000us             9
                  aten::isnan         0.12%       5.391us         6.85%     296.958us      49.493us       0.000us         0.00%       0.000us       0.000us             6
                     aten::ne         6.73%     291.567us         6.73%     291.567us      48.595us       0.000us         0.00%       0.000us       0.000us             6
-----------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------
Self CPU time total: 4.335ms
Self RBLN time total: 36.959us

18 device events, for example: ['i1_0_W0_args_0_0', 'i1_0_W0_args_0_1', 'i1_0_W0_args_0_2', 'i1_0_W0_mm_0']

Step 2: Read the table¶

• The rows are host work. Every ATen call the three matrix multiplies made appears with its CPU time, including the dispatch of aten::mm and the .cpu() downloads (aten::to, aten::_to_copy, aten::copy_, aten::_copy_from).
• The NaN/Inf scan is visible. aten::abs, aten::isinf, aten::isnan, and aten::ne are not in the script. They are the pre-dispatch check that scans eager inputs for invalid values while TORCH_RBLN_DEPLOY is unset, and they take a measurable share of the host time on a small operator like this one. TORCH_RBLN_DEPLOY=ON removes them; see Execution Semantics › Pre-dispatch checks.
• Device time is recorded, but not on these rows. The Self RBLN columns read zero per operator while the footer reports a non-zero Self RBLN time total: at this release the bridge records device work as its own events rather than attributing it to the ATen row that launched it. Read device time from the events or from the trace, not from the per-operator columns.

Step 3: Read the device events and the trace¶

The 18 device events are the device side of the three matrix multiplies. Their names are the steps of the compiled program: the i1_0, i1_1, and i1_2 prefixes number the three calls, and each call has three args steps and three mm steps. The names come from the compiled program and can change between releases. Each event carries a device duration, which the script reads through e.device_time_total if you extend it.

trace.json shows the same events on a timeline. Open it in Perfetto or chrome://tracing: the host thread rows hold the ATen calls, a separate device row holds the device events, and the gap between a launch on the host row and its event on the device row is the submission latency that explain() cannot see.

What each tool answers¶

Next steps¶

• Execution Semantics › Graph execution and input shapes - Fewer, larger device events: compile the step as one program and profile it the same way.
• Environment Variables › Profiling - RBLN_PROFILER and RBLN_PROFILER_DIR.
• RBLN Profiler - The runtime's own profiler and its viewer for the .pb files.