Skip to content

Auto Classes

The Auto Classes automatically retrieve the relevant transformers models, including the weights, configurations, and vocabularies, based on their names or paths. This lets you load and use a model without specifying its exact architecture.

Key Classes

Register Custom Classes

The AutoClass has a method register() that lets you extend it with your own custom classes. For example, if you have created a custom model called RBLNMistralNeMoForTextUpsampler, along with its configuration class RBLNMistralNeMoForTextUpsamplerConfig, you can register them with RBLNAutoModel and RBLNAutoConfig as shown below.

cosmos_upsampler is not a PyPI package: it is the cosmos_upsampler/ directory of the Model Zoo Cosmos-Predict1-7B example, so run the code below from that directory. Load the compiled model through its own class as shown; RBLNAutoModel.from_pretrained() does not accept this custom architecture.

  • Compile
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import os

from cosmos_upsampler import (
    RBLNMistralNeMoForTextUpsampler,
    RBLNMistralNeMoForTextUpsamplerConfig,
)
from optimum.rbln import RBLNAutoConfig, RBLNAutoModel

model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Compile model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_id=model_id,
    export=True,
    rbln_config={
        "batch_size": 1,
        "max_seq_len": 1024,
        "num_devices": 4,
        "create_runtimes": False,
    },
)

model.save_pretrained(os.path.basename(model_id))
  • Inference
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
import os

from cosmos_upsampler import (
    RBLNMistralNeMoForTextUpsampler,
    RBLNMistralNeMoForTextUpsamplerConfig,
)
from optimum.rbln import RBLNAutoConfig, RBLNAutoModel

model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"
model_dir = os.path.basename(model_id)

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Load from compiled model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_dir,
    export=False,
)

Supported Models

  • CTC
Model Model Architecture AutoClass
Wav2Vec2 Wav2Vec2ForCTC RBLNAutoModelForCTC
  • CausalLM
Model Model Architecture AutoClass
gpt-oss-20b GptOssForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Llama-8b LlamaForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Llama-70b LlamaForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-1.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-14b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-32b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Llama3.3-70b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.2-3b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.1-70b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.1-8b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3-8b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3-8b + LoRA LlamaForCausalLM RBLNAutoModelForCausalLM
Llama2-7b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama2-13b LlamaForCausalLM RBLNAutoModelForCausalLM
Phi-2 PhiForCausalLM RBLNAutoModelForCausalLM
Gemma-7b GemmaForCausalLM RBLNAutoModelForCausalLM
Gemma-2b GemmaForCausalLM RBLNAutoModelForCausalLM
Gemma2-9b Gemma2ForCausalLM RBLNAutoModelForCausalLM
OPT-2.7b OPTForCausalLM RBLNAutoModelForCausalLM
Mistral-7b MistralForCausalLM RBLNAutoModelForCausalLM
A.X-4.0-Light Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-0.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-1.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-3b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-14b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-32b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-72b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen3-0.6b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-1.7b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-4b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-8b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-32B Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-30B-A3B Qwen3MoeForCausalLM RBLNAutoModelForCausalLM
Midm-2.0-Mini LlamaForCausalLM RBLNAutoModelForCausalLM
Midm-2.0-Base LlamaForCausalLM RBLNAutoModelForCausalLM
Salamandra-7b LlamaForCausalLM RBLNAutoModelForCausalLM
KONI-Llama3.1-8b LlamaForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.0-7.8b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-2.4b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-7.8b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-32b ExaoneForCausalLM RBLNAutoModelForCausalLM
GPT2 GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-medium GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-large GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-xl GPT2LMHeadModel RBLNAutoModelForCausalLM
OPT-6.7b OPTForCausalLM RBLNAutoModelForCausalLM
SOLAR-10.7b LlamaForCausalLM RBLNAutoModelForCausalLM
  • Seq2SeqLM
Model Model Architecture AutoClass
T5-11b T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-small T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-base T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-large T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-3b T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
BART-base BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
BART-large BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
KoBART-base BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
Pegasus PegasusForConditionalGeneration RBLNAutoModelForSeq2SeqLM
  • DepthEstimation
Model Model Architecture AutoClass
Depth-Anything-V2-Small DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Base DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Large DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
DPT-large DPTForDepthEstimation RBLNAutoModelForDepthEstimation
  • SequenceClassification
Model Model Architecture AutoClass
RoBERTa RobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-V2-M3 XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-Base XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-Large XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
Ko-Reranker XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
  • SpeechSeq2Seq
Model Model Architecture AutoClass
Whisper-tiny WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-base WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-small WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-medium WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3 WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3-turbo WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
  • ImageTextToText
Model Model Architecture AutoClass
Qwen3-VL-2b Qwen3VLForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3-VL-4b Qwen3VLForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3-VL-8b Qwen3VLForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3-VL-32b Qwen3VLForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3-VL-MoE-30b-A3B Qwen3VLMoeForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.5-0.8b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.5-2b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.5-4b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.5-9b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.5-27b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.6-27b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen3.8-27b Qwen3_5ForConditionalGeneration RBLNAutoModelForImageTextToText
EXAONE-4.5-33b Exaone4_5_ForConditionalGeneration RBLNAutoModelForImageTextToText
PaliGemma-3b PaliGemmaForConditionalGeneration RBLNAutoModelForImageTextToText
PaliGemma2-3b PaliGemmaForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma3-4b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma3-12b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma3-27b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma4-31b Gemma4ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma4-26b-a4b Gemma4ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen2-VL-7b Qwen2VLForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen2.5-VL-7b Qwen2_5_VLForConditionalGeneration RBLNAutoModelForImageTextToText
Idefics3-8B-Llama3 Idefics3ForConditionalGeneration RBLNAutoModelForImageTextToText
Llava-v1.5-7b LlavaForConditionalGeneration RBLNAutoModelForImageTextToText
Llava-v1.6-mistral-7b LlavaNextForConditionalGeneration RBLNAutoModelForImageTextToText
Pixtral-12b LlavaForConditionalGeneration RBLNAutoModelForImageTextToText
BLIP2-6.7b Blip2ForConditionalGeneration RBLNAutoModelForImageTextToText
BLIP2-2.7b Blip2ForConditionalGeneration RBLNAutoModelForImageTextToText
  • MaskedLM
Model Model Architecture AutoClass
SecureBERT RobertaForMaskedLM RBLNAutoModelForMaskedLM
BERT-base BertForQuestionAnswering RBLNAutoModelForMaskedLM
BERT-large BertForQuestionAnswering RBLNAutoModelForMaskedLM
ModernBERT-base ModernBertForMaskedLM RBLNAutoModelForMaskedLM
ModernBERT-large ModernBertForMaskedLM RBLNAutoModelForMaskedLM
  • AudioClassification
Model Model Architecture AutoClass
Audio-Spectogram-Transformer ASTForAudioClassification RBLNAutoModelForAudioClassification
  • ImageClassification
Model Model Architecture AutoClass
ViT-large ViTForImageClassification RBLNAutoModelForImageClassification
ResNet50 ResNetForImageClassification RBLNAutoModelForImageClassification
  • QuestionAnswering
Model Model Architecture AutoClass
BERT-base BertForQuestionAnswering RBLNAutoModelForQuestionAnswering
BERT-large BertForQuestionAnswering RBLNAutoModelForQuestionAnswering
  • TextEncoding
Model Model Architecture AutoClass
T5-Enc-11b T5EncoderModel RBLNAutoModelForTextEncoding
Qwen3-Embedding-4b Qwen3Model RBLNAutoModelForTextEncoding
Qwen3-Embedding-0.6b Qwen3Model RBLNAutoModelForTextEncoding
E5-base-4K BertModel RBLNAutoModelForTextEncoding
KR-SBERT-V40K-klueNLI-augSTS BertModel RBLNAutoModelForTextEncoding
BGE-Small-EN-v1.5 BertModel RBLNAutoModelForTextEncoding
BGE-Large-EN-v1.5 BertModel RBLNAutoModelForTextEncoding
BGE-M3/Dense-Embedding XLMRobertaModel RBLNAutoModelForTextEncoding
BGE-M3/Multi-Vector XLMRobertaModel RBLNAutoModelForTextEncoding
BGE-M3/Sparse-Embedding XLMRobertaModel RBLNAutoModelForTextEncoding
  • Zero-Shot Object Detection
Model Model Architecture AutoClass
GroundingDino-Tiny GroundingDinoForObjectDetection RBLNAutoModelForZeroShotObjectDetection
GroundingDino-Base GroundingDinoForObjectDetection RBLNAutoModelForZeroShotObjectDetection

API Reference

RBLNAutoConfig

Resolver and factory for RBLN model configurations.

This class selects the concrete RBLNModelConfig subclass, validates the provided data, and returns a frozen configuration object that serves as the single source of truth during export and load. It does not define the schema or control model behavior.

register(config, exist_ok=False) staticmethod

Register a new configuration for this class.

Parameters:

Name Type Description Default
config RBLNModelConfig

The config to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load RBLNModelConfig from a path. Class name is automatically inferred from the rbln_config.json file.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig.

 required
rbln_config dict[str, Any] | None

Additional configuration to override.

 None
return_unused_kwargs bool

Whether to return unused kwargs.

 False
kwargs Any

Additional keyword arguments to override configuration values.

 {}

Returns:

Name Type Description
RBLNModelConfig Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]

The loaded RBLNModelConfig.

Examples:

1
    config = RBLNAutoConfig.from_pretrained("/path/to/model")

RBLNAutoModel

Bases: _BaseAutoModelClass

Automatically detect all supported transformers models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForCTC

Bases: _BaseAutoModelClass

Automatically detect Connectionist Temporal Classification (CTC) head Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForCausalLM

Bases: _BaseAutoModelClass

Automatically detect Casual Language Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSeq2SeqLM

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Language Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSpeechSeq2Seq

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Generation Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForDepthEstimation

Bases: _BaseAutoModelClass

Automatically detect Speech Sequence to Sequence Language Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSequenceClassification

Bases: _BaseAutoModelClass

Automatically detect Sequence Classification Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForImageTextToText

Bases: _BaseAutoModelClass

Automatically detect Image and Text to Text Generation Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForMaskedLM

Bases: _BaseAutoModelClass

Automatically detect Masked Lanuage Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForMultimodalLM

Bases: _BaseAutoModelClass

Automatically detect Multimodal Language Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForAudioClassification

Bases: _BaseAutoModelClass

Automatically detect Audio Classification Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForImageClassification

Bases: _BaseAutoModelClass

Automatically detect Image Classification Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForQuestionAnswering

Bases: _BaseAutoModelClass

Automatically detect Question Answering Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForTextEncoding

Bases: _BaseAutoModelClass

Automatically detect Text Encoding Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForZeroShotObjectDetection

Bases: _BaseAutoModelClass

Automatically detect Zero Shot Object Detection Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForObjectDetection

Bases: _BaseAutoModelClass

Automatically detect Object Detection Models.

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod

Load an RBLN-accelerated model from a pretrained checkpoint or a compiled RBLN artifact.

This convenience method determines the concrete RBLN* model class that matches the underlying HuggingFace architecture and dispatches to that class's from_pretrained() implementation. Depending on whether a compiled RBLN folder is detected (or if export=True is passed), it will either:

  • Compile from a HuggingFace checkpoint to an RBLN model
  • Or load an already-compiled RBLN model directory/repository

Parameters:

Name Type Description Default
model_id str | Path

HF repo id or local path. For compiled models, this should point to a directory (optionally under subfolder) that contains *.rbln files and rbln_config.json.

 required
export bool | None

Force compilation from a HuggingFace checkpoint. When None, this is inferred by checking whether compiled artifacts exist at model_id.

 None
rbln_config dict | RBLNModelConfig | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class (e.g., RBLNLlamaForCausalLMConfig).

 None
kwargs dict[str, Any] | None

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config. - Remaining arguments are forwarded to the HuggingFace loader (e.g., revision, token, trust_remote_code, cache_dir, subfolder, local_files_only).

 {}

Returns:

Name Type Description
RBLNBaseModel RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

from_model(model, config=None, rbln_config=None, **kwargs) classmethod

Convert and compile an in-memory HuggingFace model into an RBLN model.

This method resolves the appropriate concrete RBLN* class from the input model's class name (e.g., LlamaForCausalLM -> RBLNLlamaForCausalLM) and then delegates to that class's from_model() implementation.

Parameters:

Name Type Description Default
model PreTrainedModel

A HuggingFace model instance to convert.

 required
config PretrainedConfig | None

The configuration object associated with the model.

 None
rbln_config RBLNModelConfig | dict | None

RBLN compilation/runtime configuration. May be provided as a dictionary or as an instance of the specific model's config class.

 None
kwargs Any

Additional keyword arguments. - Arguments prefixed with rbln_ are forwarded to the RBLN config.

 {}

Returns:

Type Description
RBLNBaseModel

An instantiated RBLN model ready for inference on RBLN NPUs.

register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False