Troubleshoot¶

Debug Dump Binaries (DDB) generation¶

The DDB contains useful information for functional debugging of the RBLN NPU, such as the input of the RBLN Compiler, the error log of each compile pass, and the progress status of the compilation. Note that all DDB files are securely encrypted.

To generate the DDB, set the environment variable RBLN_DEBUG_LEVEL:

• RBLN_DEBUG_LEVEL=1: DDB generation, without model parameters
• RBLN_DEBUG_LEVEL=2: DDB generation, including model parameters

Setting RBLN_DEBUG_LEVEL=2 is better for debugging, but if it is not possible to share the model parameters, setting RBLN_DEBUG_LEVEL=1 is a suitable option.

To diagnose a problem, rerun the compilation command for the affected model with RBLN_DEBUG_LEVEL=2.

The following example uses the PyTorch ResNet50 model in the RBLN Model Zoo to demonstrate DDB generation on an NPU:

1
2
3
4
5

$ cd rbln-model-zoo/ATOM/pytorch/vision/image_classification/torchvision
$ RBLN_DEBUG_LEVEL=2 python3 compile.py --model_name resnet50

# If a DDB directory was generated, list its files
$ ls ./debug_mm_dd_yyyy_hh_mm_ss/

If a DDB directory is generated, it contains files similar to the following:

1

0_graph.json.gz.enc  100_graph.json.gz.enc  error_log.txt.enc  progress.txt.enc

Create a tarball containing all DDB files and submit it via the RBLN Forum with detailed descriptions for further assistance:

1

$ tar -zcvf debug_mm_dd_yyyy_hh_mm_ss.tar.gz debug_mm_dd_yyyy_hh_mm_ss/

Device selection¶

If your system contains multiple RBLN NPU devices, you have the option to assign a specific device ID to your process by utilizing the RBLN_DEVICES environment variable. The device ID (NPU ID) can be obtained from the CLI tool rbln-smi.

1
2
3

$ RBLN_DEVICES=0 python inference.py # to use NPU=0
$ RBLN_DEVICES=0,1 python inference.py # to use NPU=0 and NPU=1
$ RBLN_VISIBLE_DEVICES=0,1 python inference.py # alias; comma-separated list, use NPU=0 and NPU=1

Performance tuning¶

While most of the compiled model consists of NPU operations, some CPU-based operations may still be present. The performance of these operations can vary based on the host CPU environment.

To improve the performance of the CPU-based operations, you can adjust the number of threads using the following methods.

1
2
3
4
5

# Export the environment variable
$ export RBLN_NUM_THREADS=<number_of_threads>

# Run your model script
$ python run_model.py

1

$ RBLN_NUM_THREADS=<number_of_threads> python run_model.py

1
2
3

import rebel

rebel.set_num_threads(<number_of_threads>)

1
2
3
4
5

import rebel
from rebel.core.tools import search_num_threads

module = rebel.Runtime("/path/to/saved_file.rbln")
search_num_threads(module)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

INFO [rebel-compiler] Max Concurrency = 48
INFO [rebel-compiler] Current num threads = 24
INFO [rebel-compiler] Testing with 1 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 143.10 µs
INFO [rebel-compiler] Testing with 2 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 80.15 µs
INFO [rebel-compiler] Testing with 4 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 49.03 µs
INFO [rebel-compiler] Testing with 8 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 43.47 µs
INFO [rebel-compiler] Testing with 16 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 39.80 µs
INFO [rebel-compiler] Testing with 32 threads.
INFO [rebel-compiler] 200 runs: Average execution time = 44.66 µs