Skip to content

Image Classification

Overview

This tutorial introduces how to run inference with a PyTorch ResNet50 model using the RBLN SDK C/C++ Runtime API. The model is compiled using the RBLN SDK Python API, and the resulting *.rbln file is used for inference using the RBLN SDK C/C++ Runtime API.
This approach combines the ease of model preparation in Python with the performance benefits of C/C++ for inference. The entire code used in this tutorial can be found in RBLN Model Zoo.

Setup & Installation

Before you begin, ensure that your system environment is properly configured and that all required packages are installed. This includes:

Note

RBLN SDK is distributed as a .whl package. Please note that the RBLN compiler and runtime require an RBLN Portal account.

Compilation with RBLN Python API

Prepare the model

import the ResNet50 model from TorchVision.

1
2
3
4
5
6
7
from torchvision import models  
import rebel  
import torch  

model_name = "resnet50"  
weights = models.get_model_weights(model_name).DEFAULT  
model = getattr(models, model_name)(weights=weights).eval()

Compile the model

Once the torch model (torch.nn.Module) is prepared, compile it using rebel.compile_from_torch(). Then, save the compiled model using compiled_model.save().

1
2
3
4
5
# Compile the model  
compiled_model = rebel.compile_from_torch(model, [("x", [1, 3, 224, 224], "float32")])  

# Save the compiled model to local storage  
compiled_model.save(f"{model_name}.rbln")

Complete compilation

The above compilation code snippets are included in the compile.py. To compile the model and generate the *.rbln file, execute the script as follows:

1
$ python compile.py --model-name=resnet50

Inference with RBLN SDK C/C++ Runtime API

Prepare CMake Build Script

This tutorial uses OpenCV for image pre/post-processing and argparse for CLI parameter parsing. The following CMake script describes external package dependencies and linking.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# Define dependencies for external Package  
include(FetchContent)  
include(cmake/opencv.cmake)  
include(cmake/argparse.cmake)  

# Define the name of executable  
add_executable(image_classification main.cc)  

# Update link info for package dependencies: OpenCV  
find_package(OpenCV CONFIG REQUIRED)  
target_link_libraries(image_classification ${OpenCV_LIBS})  

# Update link info for dependencies: RBLN  
find_package(rbln CONFIG REQUIRED)  
target_link_libraries(image_classification rbln::rbln_runtime)  

# Update including dependencies: argparse  
target_include_directories(image_classification PRIVATE ${argparse_INCLUDE_DIRS})

Prepare the input

Preprocess the input image using OpenCV APIs. The following code snippet reads an image, performs color conversion, resizes it while maintaining the aspect ratio, crops the center, and normalizes the pixel values.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
std::string input_path = "${SAMPLE_PATH}/tabby.jpg";  

// Read image  
cv::Mat input_image;  
try {  
    input_image = cv::imread(input_path);  
} catch (const cv::Exception &err) {  
    std::cerr << err.what() << std::endl;  
    std::exit(1);  
}  

// Convert BGR to RGB  
cv::Mat image;  
cv::cvtColor(input_image, image, cv::COLOR_BGR2RGB);  

// Resize while maintaining aspect ratio  
float scale = image.rows < image.cols ? 256. / image.rows : 256. / image.cols;  
cv::resize(image, image, cv::Size(), scale, scale, cv::INTER_LINEAR);  

// Center crop to 224x224  
image = image(cv::Rect((image.cols - 224) / 2, (image.rows - 224) / 2, 224, 224));  

// Normalize the image  
image.convertTo(image, CV_32F);  
cv::Vec3f mean(123.68, 116.28, 103.53);  
cv::Vec3f std(58.395, 57.120, 57.385);  
for (unsigned i = 0; i < image.rows; i++) {  
    for (unsigned j = 0; j < image.cols; j++) {  
        cv::subtract(image.at<cv::Vec3f>(i, j), mean, image.at<cv::Vec3f>(i, j));  
        cv::divide(image.at<cv::Vec3f>(i, j), std, image.at<cv::Vec3f>(i, j));  
    }  
}  

// Convert image to blob  
cv::Mat blob = cv::dnn::blobFromImage(image);

Run inference (Synchronous Execution)

The following code snippet shows synchronous inference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
std::string model_path = "${SAMPLE_PATH}/resnet50.rbln";  
RBLNModel *mod = rbln_create_model(model_path.c_str());  
RBLNRuntime *rt = rbln_create_runtime(mod, "default", 0, 0);  

// Set input data  
rbln_set_input(rt, 0, blob.data);  

// Run sync inference  
rbln_run(rt);  

// Get output results  
float *logits = static_cast<float *>(rbln_get_output(rt, 0));

Run inference (Asynchronous Execution)

The following code snippet shows asynchronous inference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
std::string model_path = "${SAMPLE_PATH}/resnet50.rbln";  
RBLNModel *mod = rbln_create_model(model_path.c_str());  
RBLNRuntime *rt = rbln_create_async_runtime(mod, "default", 0, 0);  

// Allocate output buffer  
auto buf_size = rbln_get_layout_nbytes(rbln_get_output_layout(rt, 0));  
std::vector<float> logits(buf_size/sizeof(float));  

// Run async inference  
int rid = rbln_async_run(rt, blob.data, logits.data());  

// Wait for inference completion  
rbln_async_wait(rt, rid, 1000);

Post Processing

The output logits is a float32 array of size (1,1000), where each element represents the score for a corresponding category in ImageNet. We derive the top-1 index and print the corresponding category.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
// Postprocessing  
size_t max_idx = 0;  
float max_val = std::numeric_limits<float>::min();  
for (size_t i = 0; i < 1000; i++) {  
    if (logits[i] > max_val) {  
        max_val = logits[i];  
        max_idx = i;  
    }  
}  

// Print the predicted category  
std::cout << "Predicted category: " << IMAGENET_CATEGORIES[max_idx] << std::endl;

Release resources

Release the runtime and model.

1
2
3
4
5
// Release Runtime  
rbln_destroy_runtime(rt);  

// Release Model  
rbln_destroy_model(mod);

How to build using CMake

The complete code for the above example is included in the RBLN Model Zoo C++ examples. To compile the code and generate the executable binary, run the following commands:

1
2
3
4
$ mkdir ${SAMPLE_PATH}/build  
$ cd ${SAMPLE_PATH}/build  
$ cmake ..  
$ make

How to run Executable file

After completing all steps, you can find the executable in the build directory.

1
2
3
4
5
# Synchronous execution  
$ ${SAMPLE_PATH}/build/image_classification -i ${SAMPLE_PATH}/tabby.jpg  -m ${SAMPLE_PATH}/resnet50.rbln  

# Asynchronous execution  
$ ${SAMPLE_PATH}/build/image_classification_async -i ${SAMPLE_PATH}/tabby.jpg  -m ${SAMPLE_PATH}/resnet50.rbln

The results will look like this: 

1
Predicted category: tabby

Summary and References

This tutorial demonstrated how to compile a PyTorch ResNet50 model using the RBLN SDK Python API and run inference using the RBLN SDK C/C++ Runtime API. The resulting executable generates a prediction that can be decoded into a human-readable category.

References: