Release Notes
2024.09.27
SDK Version |
Driver |
Compiler |
Optimum RBLN |
vLLM RBLN |
Model Zoo |
RBLNServe |
2024.09.27.0 |
v1.1.67 |
v0.5.10 |
v0.1.11 |
v0.0.7 |
v0.5.2 |
v0.3.0 |
-
Install Command
| pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.10 optimum-rbln==0.1.11 vllm-rbln==0.0.7
|
-
RBLN Driver
:
- Added runtime power management with dynamic PCIe link speed change and PCIe ASPM (Active State Power Management)
- Improved P2P throughput
- Enhanced stability for Rebellions Scalable Design (RSD)
RBLN Compiler
:
- Improved internal memory management algorithm
- Updated the runtime description to show NPU version
- Refactored .rbln file format
Optimum RBLN
:
- Added functions:
- RBLNBertModel()
- RBLNBartModel()
- RBLNLlavaNextForConditionalGeneration()
- Updated
RBLNWhisperForConditionalGeneration
to support generating token timestamps and long-form transcription
vLLM RBLN
:
- Updated to support Llava-Next, BART, and T5 models
RBLN Model Zoo
:
- Added new models
- HuggingFace
- Llava-Next
- E5-Base-4k
- KoBART
- BGE-Reranker-Base/Large
- PyTorch
- MotionBERT Action Recognition
- Updated Whisper models to support generating token timestamps and long-form transcription
Others
2024.08.30
SDK Version |
Driver |
Compiler |
Optimum RBLN |
vLLM RBLN |
Model Zoo |
RBLNServe |
2024.08.30.0 |
v1.0.1 |
v0.5.9 |
v0.1.9 |
- |
v0.5.1 |
v0.3.0 |
2024.08.30.1 |
v1.0.5 |
v0.5.9 |
v0.1.9 |
v0.0.6 |
v0.5.1 |
v0.3.0 |
-
Install Command
| pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.9 optimum-rbln==0.1.9 vllm-rbln==0.0.6
|
-
RBLN Compiler
:
- Added
model_description()
method in Runtime
class
- Updated to support
where
and einsum
operations
- Fixed bug for
strided_slice
operation
Optimum RBLN
:
- Added functions:
- RBLNGemmaForCausalLM()
- RBLNMistralForCausalLM()
- RBLNDistilBertForQuestionAnswering()
vLLM RBLN
:
- Updated to support Gemma and Mistral architectures
RBLN Model Zoo
:
- Added new models
- HuggingFace
- Gemma-2B
- Gemma-7B
- Mistral-7B
- DistilBERT
- PyTorch
- PyTorch Dynamo
- YOLOv3
- YOLOv4
- YOLOv5
- YOLOv6
- YOLOvX
2024.08.16
SDK Version |
Driver |
Compiler |
Optimum RBLN |
vLLM RBLN |
Model Zoo |
RBLNServe |
2024.08.16.0 |
v1.0.1 |
v0.5.8 |
v0.1.8 |
- |
v0.5.0 |
v0.3.0 |
2024.08.16.1 |
v1.0.5 |
v0.5.8 |
v0.1.8 |
v0.0.4 |
v0.5.0 |
v0.3.0 |
-
Install Command
| pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.8 optimum-rbln==0.1.8 vllm-rbln==0.0.4
|
-
RBLN Compiler
:
- Improved visualization of the compilation progress bar
- Optimized performance for long sequence LLM models
- Reduced DRAM memory consumption for RSD models
- Fixed bug for PReLU handling logic
- Initial release of C/C++ runtime libraries:
Optimum RBLN
:
- Added functions:
- RBLNRobertaForMaskedLM()
- RBLNRobertaForSequenceClassification()
- RBLNXLMRobertaModel()
- RBLNXLMRobertaForSequenceClassification()
vLLM RBLN
:
- Updated to support GPT2 and Mi:dm architectures
RBLN Model Zoo
:
- Initial release to support
torch.compile()
in PyTorch2.0:
- Added new models (HuggingFace)
- Mi:dm-7b
- BGE-M3
- BGE-Reranker-v2-M3
- SecureBERT
- Roberta
2024.07.25
SDK Version |
Driver |
Compiler |
Optimum RBLN |
vLLM RBLN |
Model Zoo |
RBLNServe |
2024.07.25.0 |
v1.0.1 |
v0.5.7 |
v0.1.7 |
- |
v0.4.1 |
v0.3.0 |
2024.07.25.1 |
v1.0.5 |
v0.5.7 |
v0.1.7 |
v0.0.3 |
v0.4.1 |
v0.3.0 |
-
Install Command
| pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.7 optimum-rbln==0.1.7 vllm-rbln==0.0.3
|
-
RBLN Compiler
:
- Optimized RSD performance for long sequence LLMs
Optimum RBLN
:
- Added warning messge for dependency version compatibilities
- Added RBLNDPTForDepthEstimation() functiuon
- Fixed bug for memory leak in GPT models
RBLN Model Zoo
:
- Added a new model (HuggingFace)
Others
2024.07.10
SDK Version |
Driver |
Compiler |
Optimum RBLN |
vLLM RBLN |
Model Zoo |
RBLNServe |
2024.07.10.0 |
v1.0.1 |
v0.5.2 |
v0.1.4 |
- |
v0.4.0 |
v0.3.0 |
2024.07.10.1 |
v1.0.5 |
v0.5.2 |
v0.1.4 |
v0.0.3 |
v0.4.0 |
v0.3.0 |
RBLN Driver
:
- Enhanced stability for Rebellions Scalable Design (RSD)
RBLN Compiler
:
- Updated to support continuous batching
Optimum RBLN
:
- Updated
LlamaForCausalLM()
class to support continuous batching
vLLM RBLN
- Initial release to support continuous batching
- Updated the LLM Serving page to include information on continuous batching
RBLN Model Zoo
:
- Public release of the
RBLN Model Zoo
:
- Added a new model (PyTorch)
- Miscellaneous:
- Removed
pipeline()
from bert mlm inference.py
- Removed
pipeline()
from bert qa inference.py
- Added
trust_remote_code=True
to load_dataset()
methods in ast & wav2vec examples.
2024.06.11: Breaking changes
SDK Version |
Driver |
Compiler |
Optimum RBLN |
Model Zoo |
RBLNServe |
2024.05.23.0 |
v0.10.42 |
v0.3.11 |
v0.1.0 |
v0.3.6 |
v0.1.5 |
2024.06.11.0 |
v1.0.1 |
v0.5.0 |
v0.1.1 |
v0.3.10 |
v0.3.0 |
Note
BREAKING CHANGES
: Please update the RBLN Compiler
to the appropriate version as below for compatibility with the updated RBLN Driver
. You can check your RBLN Driver
version with the rbln-stat -j | grep KMD_version
command.
0.10.42
: pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.3.11
1.0.1
: pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.0
RBLN Driver
:
- Stable release for Rebellions Scalable Design (RSD)
RBLN Compiler
:
- Updated
RBLN Compiler
to be compatible with RBLN Driver
- Added utility APIs:
npu_is_available()
get_npu_name()
Optimum RBLN
:
- Updated model APIs
- Please refer to
RBLN Model Zoo
below
RBLN Model Zoo
:
- Added new models (HuggingFace)
- With Rebellions Scalable Design (RSD)
- Llama3-8b
- SOLAR-10.7b
- EEVE-Korean-10.8b
- SDXL-base-1.0
- ControlNet
RBLNServe
:
- Pinned rebel-compiler version to <0.6, >=0.5
2024.05.23: Breaking Changes
SDK Version |
Driver |
Compiler |
Optimum RBLN |
Model Zoo |
RBLNServe |
2024.05.23.0 |
v0.10.42 |
v0.3.11 |
v0.1.0 |
v0.3.6 |
v0.1.5 |
2024.05.23.1 |
v0.12.37 |
v0.4.0 |
v0.1.0 |
v0.3.6 |
v0.2.0 |
Note
BREAKING CHANGES
: Please update the RBLN Compiler
to the appropriate version as below for compatibility with the updated RBLN Driver
. You can check your RBLN Driver
version with the rbln-stat -j | grep KMD_version
command.
0.10.42
: pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.3.11
0.12.37
: pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.4.0
RBLN Driver
:
- Added support for Rebellions Scalable Design (RSD)
- rbln-stat (CLI tool) update: Added new columns
Name
and Power
for NPU version and power consumption, respectively
RBLN Compiler
:
- Updated
RBLN Compiler
to be compatible with the RBLN Driver
- Updated input arguments of python user APIs
- Added new user APIs for concurrent processing
- Enabled LLM compilation & inference for Rebellions Scalable Design (RSD)
- Added a new page - Nvidia Triton Inference Server
Optimum RBLN
:
RBLN Model Zoo
:
- Added new models (HuggingFace)
- With Rebellions Scalable Design (RSD)
- GPT2, GPT2-medium/large/xl
- T5-small/base/large/3B
- BART-base/large
- BERT-base/large
- Stable Diffusion v1.5
- SDXL-turbo
- Whisper-tiny/base/small/medium/large
- Wav2Vec2
- Audio Spectrogram Transformer
RBLNServe
:
- Pinned rebel-compiler version to <0.5, >=0.4
2024.01.31: Breaking Changes
SDK Version |
Driver |
Compiler |
Model Zoo |
RBLNServe |
2024.01.31.0 |
v0.10.42 |
v0.3.5 |
v0.2.0 |
v0.1.5 |
Note
BREAKING CHANGES
: Please update the RBLN Compiler
to the latest version (v0.3.5
or higher) for compatibility with the updated RBLN Driver
.
RBLN Driver
:
- Refactored device internal command processing logic for stability & scalability
RBLN Compiler
:
- Updated
RBLN Compiler
to be compatible with the RBLN Driver
- Updated device memory scheduling logic
- Enhanced functionality for operation fusion logic
- Updated supported OP list for both TensorFlow and Pytorch
RBLN Model Zoo
:
- Added new models (PyTorch):
- YOLOv4: v4/v4-csp-s-mish/v4-csp-x-mish
- Video ResNet: r3d_18/mc3_18/r2plus1D_18
- Video S3D: s3d
- Changed default input size:
- YOLOv3/4/5/6/7/8
- deeplabv3_resnet50/resnet101/mobilenetv3_large, fcn_resnet50/101, unet
- Restructured directories:
- PyTorch image classification examples are moved from
rbln_model_zoo/pytorch/vision/classification
to rbln_model_zoo/pytorch/vision/image_classification
RBLNServe
:
- Set rebel-compiler version pinned to <0.4, >=0.3
2023.10.06
SDK Version |
Driver |
Compiler |
Model Zoo |
RBLNServe |
2023.10.06.0 |
v0.9.34 |
v0.2.13 |
v0.1.9 |
v0.1.4 |
RBLN Compiler
:
- Updated version parsing module of runtime APIs
- Updated runtime input size calculation logic
- Enhanced functionality for tensor slicing operations
RBLNServe
:
- Updated configuration for gRPC/REST protocol
2023.09.12
SDK Version |
Driver |
Compiler |
Model Zoo |
RBLNServe |
2023.09.12.0 |
v0.9.34 |
v0.2.10 |
v0.1.9 |
v0.1.1 |
RBLN Compiler
:
- Enabled
print()
for the rebel.Runtime
module - print(module)
will show basic information of the loaded model
- Refactored compiler internal large op handling passes for scalability
- Updated error message handling logic
- Fixed bug in a type cast pass
RBLN Model Zoo
:
- Updated submodule - YOLOv3
RBLNServe
:
2023.08.18
SDK Version |
Driver |
Compiler |
Model Zoo |
RBLNServe |
2023.08.18.0 |
v0.9.34 |
v0.2.1 |
v0.1.8 |
v0.1.0 |
RBLN Compiler
:
- Fixed bug for the destruction issue in
rebel.Runtime
RBLNServe
:
2023.08.12: Breaking Changes
SDK Version |
Driver |
Compiler |
Model Zoo |
2023.08.12.0 |
v0.9.34 |
v0.2.0 |
v0.1.8 |
Note
BREAKING CHANGES
: Please update the RBLN Compiler
to the latest version (v0.2.0
or higher) for compatibility with the updated RBLN Driver
.
RBLN Driver
:
- Refactored host-device communication protocol for stability & scalability
RBLN Compiler
:
- Updated
RBLN Compiler
to be compatible with the RBLN Driver
Others
:
2023.07.31
SDK Version |
Driver |
Compiler |
Model Zoo |
2023.07.31.0 |
v0.8.44 |
v0.1.17 |
v0.1.8 |
RBLN Compiler
:
- Enhanced functionality for normalization operations
- Updated compiler internal scheduling logic
- Updated error message handling logic
RBLN Model Zoo
:
- Updated requirements.txt to use ultralytics 8.0.145
- Applied ultralytics 8.0.145 to YOLOv8
2023.07.10
SDK Version |
Driver |
Compiler |
Model Zoo |
2023.07.10.0 |
v0.8.44 |
v0.1.14 |
v0.1.7 |
RBLN Driver
:
- Enhanced stability for device reset and recovery
- rbln-stat (CLI tool) update: status categorization of the process
RBLN Compiler
:
- Updated input arguments for
compile_from_torchscript()
- Enhanced functionality for unary and binary operations
- Optimized build time
RBLN Model Zoo
:
- Added new models (PyTorch):
- YOLOv6: v6s/v6n/v6m/v6l
- YOLOv7: v7-tiny/v7/v7x
- YOLOv8: v8s/v8n/v8m/v8l/v8x
- Added new models (TF Keras Applications)
- MobileNetV3: Small/Large
- ConvNeXt: Tiny/Small/Base/Large/XLarge
- RegNetX: 002/004/006/008/016/032/040/064/080/120/160/320
- RegNetY: 002/004/006/008/016/032/040/064/080/120/160/320
2023.06.20
SDK Version |
Driver |
Compiler |
Model Zoo |
2023.06.20.0 |
v0.7.34 |
v0.1.8 |
v0.1.5 |
RBLN Compiler
:
- Added a new compile function -
compile_from_torchscript()
- Enhanced functionality for matrix multiplication and pooling operations
- Optimized device memory scheduling
RBLN Model Zoo
:
- Added new models (PyTorch):
- YOLOv3: v3-tiny/v3/v3-spp
- YOLOv5: v5s/v5n/v5m/v5l/v5x
2023.05.26
SDK Version |
Driver |
Compiler |
Model Zoo |
2023.05.26.0 |
v0.7.34 |
v0.1.5 |
v0.1.4 |