Common¶
주요 클래스¶
RBLNModel
: HuggingFace PreTrainedModel 을 RBLN NPU에서 실행하기 위한 베이스 모델 클래스.RBLNModelConfig
:RBLNModel
의 실행/컴파일 설정을 위한 클래스.
API 참조¶
Classes¶
RBLNAutoConfig
¶
Functions¶
load(path, passed_rbln_config=None, kwargs={}, return_unused_kwargs=False)
staticmethod
¶
Load RBLNModelConfig from a path.
Class name is automatically inferred from the rbln_config.json
file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
Path to the RBLNModelConfig. |
required |
passed_rbln_config
|
Optional[RBLNModelConfig]
|
RBLNModelConfig to be passed runtime options. |
None
|
kwargs
|
Optional[Dict[str, Any]]
|
Additional keyword arguments for runtime options. |
{}
|
return_unused_kwargs
|
bool
|
Whether to return unused keyword arguments. |
False
|
Returns:
Name | Type | Description |
---|---|---|
RBLNModelConfig |
RBLNModelConfig
|
The loaded RBLNModelConfig. |
RBLNModelConfig
¶
Base configuration class for RBLN models that handles compilation settings, runtime options, and submodules.
This class provides functionality for:
- Managing compilation configurations for RBLN devices
- Configuring runtime behavior such as device placement
- Handling nested configuration objects for complex model architectures
- Serializing and deserializing configurations
Examples:
Using with RBLNModel.from_pretrained():
Save and load configuration:
Converting between configuration formats:
Configuration for language models:
Working with models that have submodules:
Advanced multi-device deployment with tensor parallelism:
Compilation without runtime creation (create_runtimes=False):
Two-stage workflow with separate compilation and runtime:
Functions¶
__init__(cls_name=None, create_runtimes=None, optimize_host_memory=None, device=None, device_map=None, activate_profiler=None, npu=None, tensor_parallel_size=None, optimum_rbln_version=None, _compile_cfgs=[], **kwargs)
¶
Initialize a RBLN model configuration with runtime options and compile configurations.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
cls_name
|
Optional[str]
|
The class name of the configuration. Defaults to the current class name. |
None
|
create_runtimes
|
Optional[bool]
|
Whether to create RBLN runtimes. Defaults to True if an NPU is available. |
None
|
optimize_host_memory
|
Optional[bool]
|
Whether to optimize host memory usage. Defaults to True. |
None
|
device
|
Optional[Union[int, List[int]]]
|
The device(s) to load the model onto. Can be a single device ID or a list. |
None
|
device_map
|
Optional[Dict[str, Union[int, List[int]]]]
|
Mapping from compiled model names to device IDs. |
None
|
activate_profiler
|
Optional[bool]
|
Whether to activate the profiler for performance analysis. |
None
|
npu
|
Optional[str]
|
The NPU device name to use for compilation. |
None
|
tensor_parallel_size
|
Optional[int]
|
Size for tensor parallelism to distribute the model across devices. |
None
|
optimum_rbln_version
|
Optional[str]
|
The optimum-rbln version used for this configuration. |
None
|
_compile_cfgs
|
List[RBLNCompileConfig]
|
List of compilation configurations for the model. |
[]
|
**kwargs
|
Dict[str, Any]
|
Additional keyword arguments. |
{}
|
Raises:
Type | Description |
---|---|
ValueError
|
If unexpected keyword arguments are provided. |
Classes¶
RBLNModel
¶
Functions¶
from_pretrained(model_id, export=False, rbln_config=None, **kwargs)
classmethod
¶
The from_pretrained()
function is utilized in its standard form as in the HuggingFace transformers library.
User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model_id
|
Union[str, Path]
|
The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler. |
required |
export
|
bool
|
A boolean flag to indicate whether the model should be compiled. |
False
|
rbln_config
|
Optional[Union[Dict, RBLNModelConfig]]
|
Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., |
None
|
kwargs
|
Dict[str, Any]
|
Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library. |
{}
|
Returns:
Type | Description |
---|---|
Self
|
A RBLN model instance ready for inference on RBLN NPU devices. |
from_model(model, *, rbln_config=None, **kwargs)
classmethod
¶
Converts and compiles a pre-trained HuggingFace library model into a RBLN model. This method performs the actual model conversion and compilation process.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
PreTrainedModel
|
The PyTorch model to be compiled. The object must be an instance of the HuggingFace transformers PreTrainedModel class. |
required |
rbln_config
|
Optional[Union[Dict, RBLNModelConfig]]
|
Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., |
None
|
kwargs
|
Dict[str, Any]
|
Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library. |
{}
|
The method performs the following steps:
- Compiles the PyTorch model into an optimized RBLN graph
- Configures the model for the specified NPU device
- Creates the necessary runtime objects if requested
- Saves the compiled model and configurations
Returns:
Type | Description |
---|---|
Self
|
A RBLN model instance ready for inference on RBLN NPU devices. |
save_pretrained(save_directory)
¶
Saves a model and its configuration file to a directory, so that it can be re-loaded using the
[from_pretrained
] class method.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
save_directory
|
Union[str, PathLike]
|
The directory to save the model and its configuration files. Will be created if it doesn't exist. |
required |