Skip to content

Getting Started with RBLN Profiler

This section outlines the basic steps for using the RBLN Profiler. The following example demonstrates how to get profiling data using a PyTorch BERT model from the RBLN Model Zoo.

Prerequisites

Before we start, please make sure you have installed the following pip packages in your system.

Note

RBLN profiler is a feature in rebel-compiler package. Therefore, it is installed when you install rebel-compiler and you don't have to install RBLN profiler explicitly.

How to Extract Profiled Data

RBLN Profiler can extract profiling data from both rebel-compiler and optimum-rbln.

In this example, we will compile BERT-base, which has been trained for question answering. The model supports both rebel-compiler (Option 1) and optimum-rbln (Option 2) in the RBLN Model zoo, so we will handle how to extract profiling data in both options.

Option 1: Using rebel-compiler

You can extract the profiling data in the following way with rebel-compiler

Compile Model

The BERT-base model can be compiled with the folowing command. When the model compilation is completed, you will find bert-base.rbln file in the directory.

1
2
3
$ cd rbln-model-zoo/pytorch/nlp/bert/qa
$ pip3 install -r requirements.txt
$ python3 compile.py --model_name base

Activate RBLN Profiler

Profile the model by either setting the environment variable RBLN_PROFILER=1 while running inference, or by using the argument activate_profile=True in rebel.Runtime.

  • Using environment variable: set the environment variable RBLN_PROFILER=1 to enable RBLN Profiler. 

    1
    $ RBLN_PROFILER=1 python3 inference.py --model_name base

  • Using RBLN Runtime API: set activate_profiler=True to activate RBLN Profiler. 

    1
2
3
4
5
    # Revise `inference.py` like below.
...
# Load compiled model to RBLN runtime module
module = rebel.Runtime(f"bert-{args.model_name}.rbln", tensor_type="pt", activate_profiler=True)
...

Extract Profiled Data with Inference

  • Generally, you can obtain the profiling data without specifying any signature, as shown below: 

    1
2
3
4
5
    # Run inference with RBLN runtime module.
...
# Save to default path("./")
out = module.run(**inputs)
...

  • To configure profiling for the various controls, we support scoped profiling methods as follows: 

    1
2
3
4
5
6
7
    ...
# Configure profiling scope and save path as "./temp".
from rebel.profiler import profile

with profile(output_dir="./temp"):
    out = module.run(**inputs)
...

Option 2: Using optimum-rbln

You can extract the profiling data in the following way with optimum-rbln.

Compile Model

The BERT-base model from optimum-rbln can be compiled with the following command. When the model compilation is completed, you will find bert-base-cased-squad2 directory.

1
2
3
$ cd rbln-model-zoo/huggingface/transformers/question-answering/bert/compile.py
$ pip3 install -r requirements.txt
$ python3 compile.py --model_name base

Activate RBLN Profiler

Profile the model by either setting the environment variable RBLN_PROFILER=1 while running inference, or by using the argument activate_profile=True in optimum-rbln.

  • Using environment variable: set the environment variable RBLN_PROFILER=1 to enable RBLN Profiler. 

    1
    $ RBLN_PROFILER=1 python3 inference.py --model_name base

  • Using optimum-rbln API: set rbln_activate_profiler=True to activate RBLN Profiler. 

    1
2
3
4
5
6
7
8
9
    # Revise `inference.py` like below.
...
# Load compiled model
model = RBLNBertForQuestionAnswering.from_pretrained(
    model_id=os.path.basename(model_id),
    export=False,
    rbln_activate_profiler=True,
)
...

Extract Profiled Data with Inference

  • You can obtain the profiling data without specifying any signature: 

    1
2
3
4
    # Run inference with optimum-rbln API.
...
# Save to default path("./")
answer = pipe(question=args.question, context=args.context)

  • optimum-rbln API also support scoped profiling methods as shown below: 

    1
2
3
4
5
6
7
    ...
# Configure profiling scope and save path as "./temp".
from rebel.profiler import profile

with profile(output_dir="./temp"):
    answer = pipe(question=args.question, context=args.context)
...

Check the Profiling Result

After running one of the above commands, you can see the profiling results as below:

RBLN Profiler Generation Status Report shows all status information generated during the tracing and generation process of the RBLN profiler, displayed across three categories.

  • System Passes: monitor general system issues, verify memory allocation status as well as the profiling data generated by the Command Processor.

  • Compilation Passes: verify and modify model information generated by the Compiler based on system constraints.

  • Tracing Passes: collect and verify profiling data from the Command Processor, conduct analysis, perform visualization preprocessing through data analysis, layout reorganization and structure transformation for Protocol Buffer file generation and serialization.

Note

If the RBLN Profiler Generation Status Report consistently reports Failed, please contact to support@rebellions.ai.

  • Extracted Filename
    • Multi Stream (Full log) : rbln_{date}_{time}.pb
      • This file represents all tracing information from all inferences within a single profiler instance in one plot, recorded in a single *.pb file.
    • Single Stream (Log by sequence): rbln_{date}_{time}_{sequence_index}.pb
      • These files represents tracing information for a single module inference within a single profiler instance in one plot. Specifically, *.pb files are generated corresponding to the number of modules and their respective inference counts.

The generated *.pb files can be visualized using Perfetto.