Tutorials

Explore the following tutorials for a better understanding on how to use the RBLN SDK.

RBLN Compiler

These tutorials demonstrate how to use RBLN Python API (Compile and Runtime APIs) for PyTorch and TensorFlow models.

• PyTorch Resnet50 (Vision)
  Using TorchVision library with RBLN Compile API with a ResNet50 example.

• PyTorch BERT (NLP)
  Using PyTorch with RBLN Compile API with a BERT-base example.

• TensorFlow EfficientNetB0 (Vision)
  Using TF Keras Applications library with the RRBLN Compile API with an EfficientNet-B0 example.

• TensorFlow BERT (NLP)
  Using TensorFlow with RBLN Compile APIs with a BERT-base example.

• Concurrent Processing
  Executing AI models asynchronously with RBLN Runtime API (AsyncRuntime).

• Bucketing
  Compiling models to support multiple input shapes efficiently with a single runtime.

RBLN C/C++ Runtime API

These tutorials demonstrate how to deploy precompiled models using RBLN C/C++ Runtime API.

• Image Classification
  Deploying the PyTorch ResNet50 model with RBLN C/C++ Runtime API.

• Object Detection
  Deploying the PyTorch YOLOv8 model with RBLN C/C++ Runtime API.

• Text Generation
  Deploying the Llama3-8b model with RBLN C/C++ Runtime API.

RBLN Optimum (HuggingFace Model Support)

These tutorials demonstrate how to compile and run inference on HuggingFace models using optimum-rbln.

• SDXL-Turbo (Image Generation)
  Compiling and deploying SDXL-turbo and generating images using optimum-rbln.

• Llama3-8B (Chatbot)
  Compiling and deploying Llama3-8b models using optimum-rbln across multiple RBLN NPUs.

RBLN Profiler

These tutorials demonstrate how to profile and analyze models during inference using RBLN Runtime.

• Model Profiling
  Profiling inference models using RBLN Runtime.

• Analyze with Perfetto
  Analyzing the profiled results using the visualization tool, Perfetto.

• YOLOv8 (Object Detection)
  Profiling YOLOv8l and analyzing results with Perfetto.

• Stable Diffusion 3 (Image Generation)
  Profiling stable-diffusion-3-text-to-image and analyzing results with Perfetto.

• Llama3-8B (Text Generation)
  Profiling Llama3-8B and analyzing results with Perfetto.

RBLN PyTorch

These tutorials demonstrate how to run torch-rbln.

• Running and Debugging with torch-rbln
  Running a simple example consisting of torch.add and debugging it.

• Llama3.2-1B
  Executing transformers model using torch-rbln

Model Serving

These tutorials demonstrate how to serve precompiled AI models using Nvidia Triton Inference Server and TorchServe, both of which support vLLM.

• vllm-rbln
  Serving LLMs using vLLM Plugin

  • Llama3-8B
  • Llama3.1-8B with Flash Attention
  • Qwen2.5-VL-7B (Multimodal)
  • Inference with Dynamic Batch Sizes

• OpenAI Compatible Server
  Serving vLLM-based LLMs using OpenAI Compatible Server

• Nvidia Triton Inference Server
  Serving LLMs using Nvidia Triton Inference Server

  • Resnet50
  • YOLOv8
  • Llama3-8
  • Llama3.1-8B with Flash Attention

• TorchServe
  Serving LLMs using TorchServe

  • Resnet50
  • YOLOv8
  • Llama3-8B
  • Llama3.1-8B with Flash Attention