RBLN Compiler API Overview¶
The design philosophy of the RBLN SDK is to simplify the deployment process, allowing users to focus on developing their models and fully leveraging the benefits of RBLN NPUs with ease. We understand that dealing with a new codebase can be challenging, so we have made significant efforts to ensure that our user APIs are as straightforward and simple as possible.
The user APIs support the two most popular deep learning frameworks, PyTorch and TensorFlow. To deploy models pre-trained with PyTorch and TensorFlow, you simply need to replace the runtime part of your code with our APIs. The structure of the RBLN SDK's APIs is summarized in the diagram below.
The RBLN Compiler can directly compile the graph representations of PyTorch and TensorFlow, specifically ① and ② torch.nn.Module for PyTorch and ③ tf.function for TensorFlow_v2, ④ GraphDef for TensorFlow_v1.
The compilation process involves three steps: Model Conversion, Graph Generation, and Graph Optimization. This process generates compiled objects that are directly used in RBLN Runtime or ⑤ are saved to disk as .rbln files for deployment.
They are consumed by the RBLN Runtime ⑥ Runtime() or AsyncRuntime() objects, which provide ⑦ a run() method for running inference on RBLN NPUs. The run() method supports both torch.Tensor and numpy.ndarray for its input and output data types.
For a better understanding of the RBLN Compiler and the functionality of each API, we provide the following documents: