Skip to content

Getting Started with RBLN Profiler

This section outlines the basic steps for using the RBLN Profiler. The following example demonstrates how to get profiling data using a PyTorch ResNet50 model from the RBLN Model Zoo.

Prerequisites

Before we start, please make sure you have installed the following pip packages in your system.

Note

RBLN profiler is a feature in rebel-compiler package. Therefore, it is installed when you install rebel-compiler and you don't have to install RBLN profiler explicitly.

How to Extract Profiled Data

In this example, we will compile ResNet50, which has been trained for image classification.

Preliminaries

First, move to the project directory and install required packages.

1
2
$ cd RBLN_MODEL_ZOO_PATH/pytorch/vision/image_classification/torchvision
$ pip3 install -r requirements.txt

Compile Model

The ResNet50 model can be compiled with the folowing command. When the model compilation is completed, you will find resnet50.rbln file in the directory.

1
$ python3 compile.py --model_name resnet50

Extract Profiled Data with Inference

Profile the model by either setting the environment variable RBLN_PROFILER=1 while running inference, or by using the argument activate_profile=True in rebel.Runtime.

  • Using environment variable: set the environment variable RBLN_PROFILER=1 to enable RBLN Profiler 

    1
    $ RBLN_PROFILER=1 python3 inference.py --model_name resnet50

  • Using RBLN Runtime API: set activate_profiler=True to activate RBLN Profiler 

    1
2
3
4
5
    # Revise `inference.py` like below.
...
# Load compiled model to RBLN runtime module
module = rebel.Runtime(f"{model_name}.rbln", activate_profiler=True)
...

  • Using optimum-rbln: set rbln_activate_profiler=True to activate RBLN Profiler 

    1
2
3
4
5
6
7
8
9
    # Revise `inference.py` like below.
...
# Load compiled model
model = RBLNLlamaForCausalLM.from_pretrained(
    model_id=os.path.basename(model_id),
    export=False,
    rbln_activate_profiler=True,
)
...

After running one of the above commands, you can see the profiling results as below:

RBLN Profiler Generation Status Report shows all status information generated during the tracing and generation process of the RBLN profiler, displayed across three categories.

  • System Passes: monitor general system issues, verify memory allocation status as well as the profiling data generated by the Command Processor.

  • Compilation Passes: verify and modify model information generated by the Compiler based on system constraints.

  • Tracing Passes: collect and verify profiling data from the Command Processor, conduct analysis, perform visualization preprocessing through data analysis, layout reorganization and structure transformation for Protocol Buffer file generation and serialization.

Note

If the RBLN Profiler Generation Status Report consistently reports Failed, please contact to support@rebellions.ai.

  • Extracted Filename
    • Multi Stream (Full log) : rbln_{date}_{time}.pb
      • This file represents all tracing information from all inferences within a single profiler instance in one plot, recorded in a single *.pb file.
    • Single Stream (Log by sequence): rbln_{date}_{time}_{sequence_index}.pb
      • These files represents tracing information for a single module inference within a single profiler instance in one plot. Specifically, *.pb files are generated corresponding to the number of modules and their respective inference counts.

The generated *.pb files can be visualized using Perfetto.