RBLN NPU Architecture¶
Rebel100™ is the next-generation RBLN NPU architecture that succeeds ATOM™. It is a chiplet-based SoC: four homogeneous compute chiplets are packaged together and connected through UCIe-Advanced die-to-die links, so the four dies operate as a single device. Each chiplet is paired with its own HBM3E stack, for a total of 144GB of HBM3E and 4.8TB/s of memory bandwidth per device. Rebel100™ cards are identified by the RBLN-CRxx SKU pattern.
Inside each chiplet, neural cores, DMA engines, and synchronization units communicate over an on-chip mesh interconnect that extends across the die-to-die links. The software-configurable DMA engine moves data from local HBM3E, from the HBM3E attached to another chiplet, and from on-chip shared memory, with per-task QoS control. A hardware synchronization manager on each chiplet resolves dependencies both within the chiplet and across chiplets, so the four chiplets execute a workload as one device.
For more detailed information, refer to the RBLN white paper:
Commands¶
A command is a unit of the command stream that the RBLN Compiler generates, and it is what the RBLN Profiler measures on the device. The RBLN Profiler traces the following commands: [Host, Neural Engine Clusters, Neural DMA, Task DMA, External HDMA, Device HDMA, Device Sync, General Compute Engine]. Each command represents a specific operation, such as computation, data movement, or signal transmission between hardware components.
The RBLN Profiler chronologically traces these commands and visualizes them using Perfetto. More detailed information about Perfetto can be found in the Introduction to Perfetto. Each command appears on the track of the chiplet that executes it; see Trace Layout for how the tracks are grouped. To effectively analyze profiling results, refer to the image below, which illustrates the connections between the Rebel100™ hardware components and the commands described above.
Host¶
The Host command represents operations that are executed on the host CPU when they either run more efficiently there than on the NPU or are not supported by it. It also includes operations that adjust input shapes to better optimize execution on the RBLN NPU.
Neural Engine Clusters¶
The Neural Engine Clusters command represents operations that are running on the neural cores of a chiplet. The list of operations supported by the neural cores is summarized in the Supported OPs.
Neural DMA¶
The Neural DMA command represents data transfers between HBM3E and the scratchpad memory of the neural cores, including program binaries, input tensors, and kernel weights of the target model. The Neural DMA command can operate simultaneously with other commands, so the RBLN Compiler generates the required dependencies between the commands to ensure correct execution. These dependencies are managed by the Neural Engine's Task Manager at runtime.
Task DMA¶
The Task DMA command represents data transfers between HBM3E and the on-chip shared memory, including input tensors, intermediate tensors, and kernel weights of the target model. The RBLN Compiler leverages both Task DMA and Neural DMA commands to achieve optimal performance of the target workloads.
External/Device HDMA¶
The External HDMA command represents data transfers between the host DRAM and the device HBM3E, while the Device HDMA command represents data transfers between the HBM3E of different devices.
Device Sync¶
The Device Sync command represents synchronization between different devices. The RBLN Compiler ensures data synchronization across multiple devices while minimizing inter-device communication overhead, and Device Sync commands are managed by the Command Processor at runtime to verify that data communication is successfully completed.
General Compute Engine¶
The General Compute Engine command represents a program that the Command Processor executes to handle behavior decided at runtime, such as computing the parameters of a data transfer or the condition of a branch.
Trace Layout¶
This section describes how Perfetto groups the commands above and explains the tracks and slices that are not commands. Each Rebel100™ device appears as one top-level group with the following contents:
Host,Device Workload, andCCL Workload: device-level host-side tracks.CCL: a group for collective communication between devices, containing theCCL Commandtrack. Together with theCCL Workloadtrack, it appears only when the model uses collective communication.Chiplet 0toChiplet 3: one group per chiplet, each containing the command tracks of that chiplet (Neural Engine Clusters,Neural DMA,Task DMA,External HDMA,Device HDMA,Device Sync,General Compute Engine).
When operations execute asynchronously or concurrently, their slices overlap in time. Perfetto keeps such overlapping slices on one track and stacks them vertically, so a track such as Device Workload, CCL Workload, or Task DMA expands into several stacked slices while the operations it records overlap.
Chiplet¶
Commands appear on the track of the chiplet that executes them, so the same command name can appear under several Chiplet groups when a layer is distributed across chiplets.
Note
ATOM™ is a single-die SoC, so an ATOM™ trace has no Chiplet groups.
Device Workload¶
A Device Workload slice marks the device-side execution window of one runtime call, from the first command that starts on the device to the last command that completes. Rebel100™ executes runtime calls asynchronously, so the windows of consecutive calls can overlap; overlapping windows are stacked within the track. Every command name except Host carries a W{workload_index} token (see Understanding Chiplet, Workload, and CCL Tracks), and Device Workload_{workload_index} is the window that the command belongs to.
CCL¶
The CCL tracks represent collective communication operations issued when a model runs across multiple Rebel100™ devices using collective communication (CCL), as in tensor-parallel serving with RBLN vLLM. A run without collective communication has no CCL tracks. A CCL Workload slice covers the whole execution window of one collective, and the CCL Command slice with the same name covers the portion in which the command stream of that collective is running.
Profiling Overhead¶
The Host track carries slices named Profile Wait, Profile Readback, and Profile Workload. They measure the time the RBLN Profiler itself spends waiting for a runtime call to finish and reading its counters back from the device. These slices are profiling overhead: they do not exist when profiling is disabled, so exclude them when you evaluate the performance of the model.
ATOM™ is a multi-core System-on-Chip (SoC) that integrates all essential components for running deep neural networks into a single chip. It combines Neural Engines, a Command Processor, an on-chip local and global scratchpad memory hierarchy, a Network-on-Chip (NoC) bus fabric, and PCIe 5.0 and GDDR6 interfaces into a compact design. ATOM™ features 4MB of local SRAM (Scratch Pad) within each Neural Engine, 32MB of global SRAM (Shared Memory) shared across all Neural Engines, and 16GB of off-chip GDDR6 DRAM. This hierarchical structure of on-chip and off-chip memory reduces execution time while minimizing energy consumption for memory access.
For large-scale models, such as large language models (LLMs) that often require more than 16GB of memory, a single ATOM™ device may not be able to load an entire model at once. In this case, large-scale models are partitioned across multiple devices for parallel execution through the Rebellions Scalable Design (RSD) architecture. During this process, multiple devices must communicate and synchronize data to ensure proper execution.
For more detailed information, refer to the RBLN white papers:
Commands¶
A command is a unit of the command stream that the RBLN Compiler generates, and it is what the RBLN Profiler measures on the device. The RBLN Profiler traces a total of eight commands: [Host, Neural Engine Clusters, Neural DMA, Task DMA, External HDMA, Device HDMA, Device Sync, General Compute Engine]. Each command represents a specific operation, such as computation, data movement, or signal transmission between hardware components.
The RBLN Profiler chronologically traces these commands and visualizes them using Perfetto. More detailed information about Perfetto can be found in the Introduction to Perfetto. To effectively analyze profiling results, refer to the image below, which illustrates the connections between the ATOM™ hardware components and the commands described above.
Host¶
The Host command represents operations that are executed on the host CPU when they either run more efficiently there than on the NPU or are not supported by it. It also includes operations that adjust input shapes to better optimize execution on the RBLN NPU.
Neural Engine Clusters¶
The Neural Engine Clusters command represents operations that are running on the Neural Engines in ATOM™. The Neural Engines are designed for computation with a focus on low latency, high utilization, and flexibility. The list of operations supported by the Neural Engines is summarized in the Supported OPs.
Neural DMA¶
The Neural DMA command represents data transfers between the device DRAM and the Neural Engine's Scratch Pad, including program binaries, input tensors, and kernel weights of the target model. The Neural DMA command can operate simultaneously with other commands, so the RBLN Compiler generates the required dependencies between the commands to ensure correct execution. These dependencies are managed by the Neural Engine's Task Manager at runtime.
Task DMA¶
The Task DMA command represents data transfers between the device DRAM and the Shared Memory in the SoC, including input tensors, intermediate tensors, and kernel weights of the target model. The RBLN Compiler leverages both Task DMA and Neural DMA commands to achieve optimal performance of the target workloads.
External/Device HDMA¶
The External HDMA command represents data transfers between the host DRAM and the device DRAM, while the Device HDMA command represents data transfers between the device DRAMs or the Shared Memories across different devices under the RSD configuration.
Device Sync¶
The Device Sync command represents synchronization between different devices under the RSD configuration. The RBLN Compiler ensures data synchronization across multiple devices while minimizing inter-device communication overhead and maximizing effective memory bandwidth. Device Sync commands generated by the RBLN Compiler are managed by the Command Processor at runtime to verify that data communication is successfully completed and to enable the immediate execution of subsequent commands.
General Compute Engine¶
The General Compute Engine command represents a program that the Command Processor executes to handle behavior decided at runtime, such as computing the parameters of a data transfer or the condition of a branch.
Trace Layout¶
This section describes how Perfetto groups the commands above and explains the slices that are not commands. An ATOM™ trace has a flat layout: each ATOM™ device appears as one top-level group that directly holds the Host track and one track per command. Commands are named {sequence_index}_{layer_name}_{command_index}; see How to Analyze.
Device Workload¶
The Host track carries one Device Workload_{n} slice per runtime call, covering the device-side execution window of that call.
Profiling Overhead¶
The Host track also carries a Profiling Overhead slice per runtime call. It measures the time the RBLN Profiler itself spends reading the counters of that call back from the device. This slice is profiling overhead: it does not exist when profiling is disabled, so exclude it when you evaluate the performance of the model.