Auto Classes¶

The Auto Classes automatically retrieves relevant transformers models, including the weights, configurations, and vocabularies, based on their names or paths. This feature allows users to easily load and use models without needing to know their exact model architecture.

Key Classes¶

RBLNAutoConfig: Configuration class for auto models.
RBLNAutoModel: The auto model for running transformers models supported on RBLN NPU.
RBLNAutoModelForCTC: The auto model class for running Connectionist Temporal Classification (CTC) head models on RBLN NPU.
RBLNAutoModelForCausalLM: The auto model class for running Casual Language Models on RBLN NPU.
RBLNAutoModelForSeq2SeqLM: The auto model class for running Sequence-to-Sequence Language Models on RBLN NPU.
RBLNAutoModelForDepthEstimation: The auto model class for running Sequence-to-Depth Estimation Models on RBLN NPU.
RBLNAutoModelForSequenceClassification: The auto model class for running Sequence Classification Models on RBLN NPU.
RBLNAutoModelForSpeechSeq2Seq: The auto model class for running SpeechSequence-to-Sequence Generation Models on RBLN NPU.
RBLNAutoModelForVision2Seq: The auto model class for running Vision-to-Sequence Generation Models on RBLN NPU.
RBLNAutoModelForImageTextToText: The auto model class for running Image-Text-To-Text Generation Models on RBLN NPU.
RBLNAutoModelForMaskedLM: The auto model class for running Masked Lanuage Models on RBLN NPU.
RBLNAutoModelForAudioClassification: The auto model class for running Audio Classification Models on RBLN NPU.
RBLNAutoModelForImageClassification: The auto model class for running Image Classification Models on RBLN NPU.
RBLNAutoModelForQuestionAnswering: The auto model class for running Question Answering Models on RBLN NPU.
RBLNAutoModelForTextEncoding: The auto model class for running Text Encoding Models on RBLN NPU.
RBLNAutoModelForZeroShotObjectDetection: The auto model class for running Zero Shot Object Detection Models on RBLN NPU.

Register Custom Classes¶

The AutoClass has a method register() that lets you extend it with your own custom classes. For example, if you've created a custom model called RBLNMistralNeMoForTextUpsampler, along with its configuration class RBLNMistralNeMoForTextUpsamplerConfig, you can add them to the AutoClass so they can be automatically loaded.

Compile

from cosmos_upsampler import RBLNMistralNeMoForTextUpsampler, RBLNMistralNeMoForTextUpsamplerConfig

model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Compile model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_id=upsampler_model_id,
    export=True,
    rbln_config={
        "batch_size": 1,
        "max_seq_len": 1024,
        "tensor_parallel_size": 4,
        "create_runtimes": False,
    },
)

model.save_pretrained(os.path.basename(model_id))

Inference

from cosmos_upsampler import RBLNMistralNeMoForTextUpsampler, RBLNMistralNeMoForTextUpsamplerConfig
model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"
model_dir = os.path.basename(model_id)

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Load from compiled model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_dir,
    export=False,
)
...

Supported Models¶

CTC

Model	Model Architecture	AutoClass
Wav2Vec2	Wav2Vec2ForCTC	RBLNAutoModelForCTC

CausalLM

Model	Model Architecture	AutoClass
DeepSeek-R1-Distill-Llama-8b	LlamaForCausalLM	RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Llama-70b	LlamaForCausalLM	RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-1.5b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-7b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-14b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-32b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Llama3.3-70b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama3.2-3b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama3.1-70b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama3.1-8b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama3-8b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama3-8b + LoRA	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama2-7b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Llama2-13b	LlamaForCausalLM	RBLNAutoModelForCausalLM
Phi-2	PhiForCausalLM	RBLNAutoModelForCausalLM
Gemma-7b	GemmaForCausalLM	RBLNAutoModelForCausalLM
Gemma-2b	GemmaForCausalLM	RBLNAutoModelForCausalLM
OPT-2.7b	OPTForCausalLM	RBLNAutoModelForCausalLM
Mistral-7b	MistralForCausalLM	RBLNAutoModelForCausalLM
A.X-4.0-Light	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2-7b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-0.5b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-1.5b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-3b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-7b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-14b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-32b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen2.5-72b	Qwen2ForCausalLM	RBLNAutoModelForCausalLM
Qwen3-0.6b	Qwen3ForCausalLM	RBLNAutoModelForCausalLM
Qwen3-1.7b	Qwen3ForCausalLM	RBLNAutoModelForCausalLM
Qwen3-4b	Qwen3ForCausalLM	RBLNAutoModelForCausalLM
Qwen3-8b	Qwen3ForCausalLM	RBLNAutoModelForCausalLM
Qwen3-32b	Qwen3ForCausalLM	RBLNAutoModelForCausalLM
Midm-2.0-Mini	LlamaForCausalLM	RBLNAutoModelForCausalLM
Midm-2.0-Base	LlamaForCausalLM	RBLNAutoModelForCausalLM
Salamandra-7b	LlamaForCausalLM	RBLNAutoModelForCausalLM
KONI-Llama3.1-8b	LlamaForCausalLM	RBLNAutoModelForCausalLM
EXAONE-3.0-7.8b	ExaoneForCausalLM	RBLNAutoModelForCausalLM
EXAONE-3.5-2.4b	ExaoneForCausalLM	RBLNAutoModelForCausalLM
EXAONE-3.5-7.8b	ExaoneForCausalLM	RBLNAutoModelForCausalLM
EXAONE-3.5-32b	ExaoneForCausalLM	RBLNAutoModelForCausalLM
GPT2	GPT2LMHeadModel	RBLNAutoModelForCausalLM
GPT2-medium	GPT2LMHeadModel	RBLNAutoModelForCausalLM
GPT2-large	GPT2LMHeadModel	RBLNAutoModelForCausalLM
GPT2-xl	GPT2LMHeadModel	RBLNAutoModelForCausalLM
OPT-6.7b	OPTForCausalLM	RBLNAutoModelForCausalLM
SOLAR-10.7b	LlamaForCausalLM	RBLNAutoModelForCausalLM
EEVE-Korean-10.8b	LlamaForCausalLM	RBLNAutoModelForCausalLM

Seq2SeqLM

Model	Model Architecture	AutoClass
T5-11b	T5ForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
T5-small	T5ForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
T5-base	T5ForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
T5-large	T5ForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
T5-3b	T5ForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
BART-base	BartForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
BART-large	BartForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
KoBART-base	BartForConditionalGeneration	RBLNAutoModelForSeq2SeqLM
Pegasus	PegasusForConditionalGeneration	RBLNAutoModelForSeq2SeqLM

DepthEsitmation

Model	Model Architecture	AutoClass
Depth-Anything-V2-Small	DepthAnythingForDepthEstimation	RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Base	DepthAnythingForDepthEstimation	RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Large	DepthAnythingForDepthEstimation	RBLNAutoModelForDepthEstimation
DPT-large	DPTForDepthEstimation	RBLNAutoModelForDepthEstimation

SequenceClassification

Model	Model Architecture	AutoClass
RoBERTa	RobertaForSequenceClassification	RBLNAutoModelForSequenceClassification
BGE-Reranker-V2-M3	XLMRobertaForSequenceClassification	RBLNAutoModelForSequenceClassification
BGE-Reranker-Base	XLMRobertaForSequenceClassification	RBLNAutoModelForSequenceClassification
BGE-Reranker-Large	XLMRobertaForSequenceClassification	RBLNAutoModelForSequenceClassification
Ko-Reranker	XLMRobertaForSequenceClassification	RBLNAutoModelForSequenceClassification

SpeechSeq2Seq

Model	Model Architecture	AutoClass
Whisper-tiny	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq
Whisper-base	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq
Whisper-small	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq
Whisper-medium	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3-turbo	WhisperForConditionalGeneration	RBLNAutoModelForSpeechSeq2Seq

Vision2Seq

Model	Model Architecture	AutoClass
Qwen2-VL-7b	Qwen2VLForConditionalGeneration	RBLNAutoModelForVision2Seq
Qwen2.5-VL-7b	Qwen2_5_VLForConditionalGeneration	RBLNAutoModelForVision2Seq
Idefics3-8B-Llama3	Idefics3ForConditionalGeneration	RBLNAutoModelForVision2Seq
Llava-v1.5-7b	LlavaForConditionalGeneration	RBLNAutoModelForVision2Seq
Llava-v1.6-mistral-7b	LlavaNextForConditionalGeneration	RBLNAutoModelForVision2Seq
Pixtral-12b	LlavaForConditionalGeneration	RBLNAutoModelForVision2Seq
BLIP2-6.7b	RBLNBlip2ForConditionalGeneration	RBLNAutoModelForVision2Seq
BLIP2-2.7b	RBLNBlip2ForConditionalGeneration	RBLNAutoModelForVision2Seq

ImageTextToText

Model	Model Architecture	AutoClass
Gemma3-4b	Gemma3ForConditionalGeneration	RBLNAutoModelForImageTextToText
Gemma3-12b	Gemma3ForConditionalGeneration	RBLNAutoModelForImageTextToText
Gemma3-27b	Gemma3ForConditionalGeneration	RBLNAutoModelForImageTextToText
Qwen2-VL-7b	Qwen2VLForConditionalGeneration	RBLNAutoModelForImageTextToText
Qwen2.5-VL-7b	Qwen2_5_VLForConditionalGeneration	RBLNAutoModelForImageTextToText
Idefics3-8B-Llama3	Idefics3ForConditionalGeneration	RBLNAutoModelForImageTextToText
Llava-v1.5-7b	LlavaForConditionalGeneration	RBLNAutoModelForImageTextToText
Llava-v1.6-mistral-7b	LlavaNextForConditionalGeneration	RBLNAutoModelForImageTextToText
Pixtral-12b	LlavaForConditionalGeneration	RBLNAutoModelForImageTextToText
BLIP2-6.7b	RBLNBlip2ForConditionalGeneration	RBLNAutoModelForImageTextToText
BLIP2-2.7b	RBLNBlip2ForConditionalGeneration	RBLNAutoModelForImageTextToText

MaskedLM

Model	Model Architecture	AutoClass
BERT-base	BertForMaskedLM	RBLNAutoModelForMaskedLM
BERT-large	BertForMaskedLM	RBLNAutoModelForMaskedLM
SecureBERT	RobertaForMaskedLM	RBLNAutoModelForMaskedLM

AudioClassification

Model	Model Architecture	AutoClass
Audio-Spectogram-Transformer	ASTForAudioClassification	RBLNAutoModelForAudioClassification

ImageClassification

Model	Model Architecture	AutoClass
ViT-large	ViTForImageClassification	RBLNAutoModelForImageClassification
ResNet50	ResNetForImageClassification	RBLNAutoModelForImageClassification

QuestionAnswering

Model	Model Architecture	AutoClass
BERT-base	BertForQuestionAnswering	RBLNAutoModelForQuestionAnswering
BERT-large	BertForQuestionAnswering	RBLNAutoModelForQuestionAnswering

TextEncoding

Model	Model Architecture	AutoClass
T5-Enc-11b	T5EncoderModel	RBLNAutoModelForTextEncoding
Qwen3-Embedding-4b	Qwen3Model	RBLNAutoModelForTextEncoding
Qwen3-Embedding-0.6b	Qwen3Model	RBLNAutoModelForTextEncoding
E5-base-4K	BertModel	RBLNAutoModelForTextEncoding
KR-SBERT-V40K-klueNLI-augSTS	BertModel	RBLNAutoModelForTextEncoding
BGE-Small-EN-v1.5	RBLNBertModel	RBLNAutoModelForTextEncoding
BGE-Large-EN-v1.5	RBLNBertModel	RBLNAutoModelForTextEncoding
BGE-M3/Dense-Embedding	XLMRobertaModel	RBLNAutoModelForTextEncoding
BGE-M3/Multi-Vector	XLMRobertaModel	RBLNAutoModelForTextEncoding
BGE-M3/Sparse-Embedding	XLMRobertaModel	RBLNAutoModelForTextEncoding

Zero-Shot Object Detection

Model	Model Architecture	AutoClass
GroundingDino-Tiny	GroundingDinoForObjectDetection	RBLNAutoModelForZeroShotObjectDetection
GroundingDino-Base	GroundingDinoForObjectDetection	RBLNAutoModelForZeroShotObjectDetection

API Reference¶

Classes¶

`RBLNAutoConfig` ¶

Resolver and factory for RBLN model configurations.

This class selects the concrete RBLNModelConfig subclass, validates the provided data, and returns a frozen configuration object that serves as the single source of truth during export and load. It does not define the schema or control model behavior.

Functions¶

`register(config, exist_ok=False)` `staticmethod` ¶

Register a new configuration for this class.

Parameters:

Name	Type	Description	Default
`config`	`RBLNModelConfig`	The config to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`load(path, passed_rbln_config=None, kwargs={}, return_unused_kwargs=False)` `staticmethod` ¶

Load RBLNModelConfig from a path. Class name is automatically inferred from the rbln_config.json file.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig.	required
`passed_rbln_config`	`Optional[RBLNModelConfig]`	RBLNModelConfig to pass its runtime options.	`None`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, Tuple[RBLNModelConfig, Dict[str, Any]]]`	The loaded RBLNModelConfig.

Classes¶

`RBLNAutoModel` ¶

Bases: _BaseAutoModelClass

Automatically detect all supported transformers models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForCTC` ¶

Bases: _BaseAutoModelClass

Automatically detect Connectionist Temporal Classification (CTC) head Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForCausalLM` ¶

Bases: _BaseAutoModelClass

Automatically detect Casual Language Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForSeq2SeqLM` ¶

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Language Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForSpeechSeq2Seq` ¶

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Generation Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForDepthEstimation` ¶

Bases: _BaseAutoModelClass

Automatically detect Speech Sequence to Sequence Language Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForSequenceClassification` ¶

Bases: _BaseAutoModelClass

Automatically detect Sequence Classification Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForVision2Seq` ¶

Bases: _BaseAutoModelClass

Automatically detect Vision to Sequence Generation Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForImageTextToText` ¶

Bases: _BaseAutoModelClass

Automatically detect Image and Text to Text Generation Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForMaskedLM` ¶

Bases: _BaseAutoModelClass

Automatically detect Masked Lanuage Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForAudioClassification` ¶

Bases: _BaseAutoModelClass

Automatically detect Audio Classification Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForImageClassification` ¶

Bases: _BaseAutoModelClass

Automatically detect Image Classification Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForQuestionAnswering` ¶

Bases: _BaseAutoModelClass

Automatically detect Question Answering Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForTextEncoding` ¶

Bases: _BaseAutoModelClass

Automatically detect Text Encoding Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

`RBLNAutoModelForZeroShotObjectDetection` ¶

Bases: _BaseAutoModelClass

Automatically detect Zero Shot Object Detection Models.

Functions¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

Compile from a HuggingFace checkpoint to an RBLN model
Or load an already-compiled RBLN model directory/repository

Parameters:

Name	Type	Description	Default
`model_id`	`Union[str, Path]`	HF repo id or local path. For compiled models, this should point to a directory (optionally under `subfolder`) that contains `*.rbln` files and `rbln_config.json`.	required
`export`	`bool`	Force compilation from a HuggingFace checkpoint. When `None`, this is inferred by checking whether compiled artifacts exist at `model_id`.	`None`
`rbln_config`	`Optional[Union[Dict, RBLNModelConfig]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., `RBLNLlamaForCausalLMConfig`).	`None`
`kwargs`		Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., `revision`, `token`, `trust_remote_code`, `cache_dir`, `subfolder`, `local_files_only`).	`{}`

Returns:

Type	Description
	An instantiated RBLN model ready for inference on RBLN NPUs.

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	A HuggingFace model instance to convert.	required
`config`	`Optional[PretrainedConfig]`	The configuration object associated with the model.	`None`
`rbln_config`	`Optional[Union[RBLNModelConfig, Dict]]`	RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.	`None`
`kwargs`	`Any`	Additional keyword arguments. - Arguments prefixed with `rbln_` are forwarded to the RBLN config.	`{}`

Returns:

Type	Description
`RBLNBaseModel`	An instantiated RBLN model ready for inference on RBLN NPUs.

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

Register a new RBLN model class.

Parameters:

Name	Type	Description	Default
`rbln_cls`	`Type[RBLNBaseModel]`	The RBLN model class to register.	required
`exist_ok`	`bool`	Whether to allow registering an already registered model.	`False`

Auto Classes¶

Key Classes¶

Register Custom Classes¶

Supported Models¶

API Reference¶

Classes¶

RBLNAutoConfig ¶

Functions¶

register(config, exist_ok=False) staticmethod ¶

load(path, passed_rbln_config=None, kwargs={}, return_unused_kwargs=False) staticmethod ¶

Classes¶

RBLNAutoModel ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForCTC ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForCausalLM ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForSeq2SeqLM ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForSpeechSeq2Seq ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForDepthEstimation ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForSequenceClassification ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForVision2Seq ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForImageTextToText ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForMaskedLM ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForAudioClassification ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForImageClassification ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForQuestionAnswering ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

register(rbln_cls, exist_ok=False) staticmethod ¶

RBLNAutoModelForTextEncoding ¶

Functions¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

from_model(model, config=None, rbln_config=None, **kwargs) classmethod ¶

`RBLNAutoConfig` ¶

`register(config, exist_ok=False)` `staticmethod` ¶

`load(path, passed_rbln_config=None, kwargs={}, return_unused_kwargs=False)` `staticmethod` ¶

`RBLNAutoModel` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForCTC` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForCausalLM` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForSeq2SeqLM` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForSpeechSeq2Seq` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForDepthEstimation` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForSequenceClassification` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForVision2Seq` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForImageTextToText` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForMaskedLM` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForAudioClassification` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForImageClassification` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForQuestionAnswering` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForTextEncoding` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶

`RBLNAutoModelForZeroShotObjectDetection` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`from_model(model, config=None, rbln_config=None, **kwargs)` `classmethod` ¶

`register(rbln_cls, exist_ok=False)` `staticmethod` ¶