Skip to content

Auto Classes

The Auto Classes automatically retrieves relevant transformers models, including the weights, configurations, and vocabularies, based on their names or paths. This feature allows users to easily load and use models without needing to know their exact model architecture.

Key Classes

Register Custom Classes

The AutoClass has a method register() that lets you extend it with your own custom classes. For example, if you've created a custom model called RBLNMistralNeMoForTextUpsampler, along with its configuration class RBLNMistralNeMoForTextUpsamplerConfig, you can add them to the AutoClass so they can be automatically loaded.

  • Compile
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
from cosmos_upsampler import RBLNMistralNeMoForTextUpsampler, RBLNMistralNeMoForTextUpsamplerConfig

model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Compile model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_id=upsampler_model_id,
    export=True,
    rbln_config={
        "batch_size": 1,
        "max_seq_len": 1024,
        "tensor_parallel_size": 4,
        "create_runtimes": False,
    },
)

model.save_pretrained(os.path.basename(model_id))
  • Inference
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from cosmos_upsampler import RBLNMistralNeMoForTextUpsampler, RBLNMistralNeMoForTextUpsamplerConfig
model_id = "nvidia/Cosmos-UpsamplePrompt1-12B-Text2World"
model_dir = os.path.basename(model_id)

# Register Custom Class
RBLNAutoModel.register(RBLNMistralNeMoForTextUpsampler, exist_ok=True)
RBLNAutoConfig.register(RBLNMistralNeMoForTextUpsamplerConfig, exist_ok=True)

# Load from compiled model
model = RBLNMistralNeMoForTextUpsampler.from_pretrained(
    model_dir,
    export=False,
)
...

Supported Models

  • CTC
Model Model Architecture AutoClass
Wav2Vec2 Wav2Vec2ForCTC RBLNAutoModelForCTC
  • CausalLM
Model Model Architecture AutoClass
DeepSeek-R1-Distill-Llama-8b LlamaForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Llama-70b LlamaForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-1.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-14b Qwen2ForCausalLM RBLNAutoModelForCausalLM
DeepSeek-R1-Distill-Qwen-32b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Llama3.3-70b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.2-3b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.1-70b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3.1-8b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3-8b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama3-8b + LoRA LlamaForCausalLM RBLNAutoModelForCausalLM
Llama2-7b LlamaForCausalLM RBLNAutoModelForCausalLM
Llama2-13b LlamaForCausalLM RBLNAutoModelForCausalLM
Phi-2 PhiForCausalLM RBLNAutoModelForCausalLM
Gemma-7b GemmaForCausalLM RBLNAutoModelForCausalLM
Gemma-2b GemmaForCausalLM RBLNAutoModelForCausalLM
OPT-2.7b OPTForCausalLM RBLNAutoModelForCausalLM
Mistral-7b MistralForCausalLM RBLNAutoModelForCausalLM
A.X-4.0-Light Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-0.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-1.5b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-3b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-7b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-14b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-32b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen2.5-72b Qwen2ForCausalLM RBLNAutoModelForCausalLM
Qwen3-0.6b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-1.7b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-4b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-8b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Qwen3-32b Qwen3ForCausalLM RBLNAutoModelForCausalLM
Midm-2.0-Mini LlamaForCausalLM RBLNAutoModelForCausalLM
Midm-2.0-Base LlamaForCausalLM RBLNAutoModelForCausalLM
Salamandra-7b LlamaForCausalLM RBLNAutoModelForCausalLM
KONI-Llama3.1-8b LlamaForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.0-7.8b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-2.4b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-7.8b ExaoneForCausalLM RBLNAutoModelForCausalLM
EXAONE-3.5-32b ExaoneForCausalLM RBLNAutoModelForCausalLM
GPT2 GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-medium GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-large GPT2LMHeadModel RBLNAutoModelForCausalLM
GPT2-xl GPT2LMHeadModel RBLNAutoModelForCausalLM
OPT-6.7b OPTForCausalLM RBLNAutoModelForCausalLM
SOLAR-10.7b LlamaForCausalLM RBLNAutoModelForCausalLM
EEVE-Korean-10.8b LlamaForCausalLM RBLNAutoModelForCausalLM
  • Seq2SeqLM
Model Model Architecture AutoClass
T5-11b T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-small T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-base T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-large T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
T5-3b T5ForConditionalGeneration RBLNAutoModelForSeq2SeqLM
BART-base BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
BART-large BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
KoBART-base BartForConditionalGeneration RBLNAutoModelForSeq2SeqLM
Pegasus PegasusForConditionalGeneration RBLNAutoModelForSeq2SeqLM
  • DepthEsitmation
Model Model Architecture AutoClass
Depth-Anything-V2-Small DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Base DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
Depth-Anything-V2-Large DepthAnythingForDepthEstimation RBLNAutoModelForDepthEstimation
DPT-large DPTForDepthEstimation RBLNAutoModelForDepthEstimation
  • SequenceClassification
Model Model Architecture AutoClass
RoBERTa RobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-V2-M3 XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-Base XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
BGE-Reranker-Large XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
Ko-Reranker XLMRobertaForSequenceClassification RBLNAutoModelForSequenceClassification
  • SpeechSeq2Seq
Model Model Architecture AutoClass
Whisper-tiny WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-base WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-small WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-medium WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3 WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
Whisper-large-v3-turbo WhisperForConditionalGeneration RBLNAutoModelForSpeechSeq2Seq
  • Vision2Seq
Model Model Architecture AutoClass
Qwen2.5-VL-7b Qwen2_5_VLForConditionalGeneration RBLNAutoModelForVision2Seq
Idefics3-8B-Llama3 Idefics3ForConditionalGeneration RBLNAutoModelForVision2Seq
Llava-v1.5-7b LlavaForConditionalGeneration RBLNAutoModelForVision2Seq
Llava-v1.6-mistral-7b LlavaNextForConditionalGeneration RBLNAutoModelForVision2Seq
Pixtral-12b LlavaForConditionalGeneration RBLNAutoModelForVision2Seq
BLIP2-6.7b RBLNBlip2ForConditionalGeneration RBLNAutoModelForVision2Seq
BLIP2-2.7b RBLNBlip2ForConditionalGeneration RBLNAutoModelForVision2Seq
  • ImageTextToText
Model Model Architecture AutoClass
Gemma3-4b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma3-12b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Gemma3-27b Gemma3ForConditionalGeneration RBLNAutoModelForImageTextToText
Qwen2.5-VL-7b Qwen2_5_VLForConditionalGeneration RBLNAutoModelForImageTextToText
Idefics3-8B-Llama3 Idefics3ForConditionalGeneration RBLNAutoModelForImageTextToText
Llava-v1.5-7b LlavaForConditionalGeneration RBLNAutoModelForImageTextToText
Llava-v1.6-mistral-7b LlavaNextForConditionalGeneration RBLNAutoModelForImageTextToText
Pixtral-12b LlavaForConditionalGeneration RBLNAutoModelForImageTextToText
BLIP2-6.7b RBLNBlip2ForConditionalGeneration RBLNAutoModelForImageTextToText
BLIP2-2.7b RBLNBlip2ForConditionalGeneration RBLNAutoModelForImageTextToText
  • MaskedLM
Model Model Architecture AutoClass
BERT-base BertForMaskedLM RBLNAutoModelForMaskedLM
BERT-large BertForMaskedLM RBLNAutoModelForMaskedLM
SecureBERT RobertaForMaskedLM RBLNAutoModelForMaskedLM
  • AudioClassification
Model Model Architecture AutoClass
Audio-Spectogram-Transformer ASTForAudioClassification RBLNAutoModelForAudioClassification
  • ImageClassification
Model Model Architecture AutoClass
ViT-large ViTForImageClassification RBLNAutoModelForImageClassification
ResNet50 ResNetForImageClassification RBLNAutoModelForImageClassification
  • QuestionAnswering
Model Model Architecture AutoClass
BERT-base BertForQuestionAnswering RBLNAutoModelForQuestionAnswering
BERT-large BertForQuestionAnswering RBLNAutoModelForQuestionAnswering
  • TextEncoding
Model Model Architecture AutoClass
T5-Enc-11b T5EncoderModel RBLNAutoModelForTextEncoding
Qwen3-Embedding-4b Qwen3Model RBLNAutoModelForTextEncoding
Qwen3-Embedding-0.6b Qwen3Model RBLNAutoModelForTextEncoding
E5-base-4K BertModel RBLNAutoModelForTextEncoding
KR-SBERT-V40K-klueNLI-augSTS BertModel RBLNAutoModelForTextEncoding
BGE-Small-EN-v1.5 RBLNBertModel RBLNAutoModelForTextEncoding
BGE-Large-EN-v1.5 RBLNBertModel RBLNAutoModelForTextEncoding
BGE-M3 XLMRobertaModel RBLNAutoModelForTextEncoding

API Reference

Classes

RBLNAutoConfig

Functions

register(config, exist_ok=False) staticmethod

Register a new configuration for this class.

Parameters:

Name Type Description Default
config [`RBLNModelConfig`]

The config to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False
load(path, passed_rbln_config=None, kwargs={}, return_unused_kwargs=False) staticmethod

Load RBLNModelConfig from a path. Class name is automatically inferred from the rbln_config.json file.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig.

 required
passed_rbln_config Optional[RBLNModelConfig]

RBLNModelConfig to pass its runtime options.

 None

Returns:

Name Type Description
RBLNModelConfig Union[RBLNModelConfig, Tuple[RBLNModelConfig, Dict[str, Any]]]

The loaded RBLNModelConfig.

Classes

RBLNAutoModel

Bases: _BaseAutoModelClass

Automatically detect all supported transformers models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForCTC

Bases: _BaseAutoModelClass

Automatically detect Connectionist Temporal Classification (CTC) head Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForCausalLM

Bases: _BaseAutoModelClass

Automatically detect Casual Language Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSeq2SeqLM

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Language Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSpeechSeq2Seq

Bases: _BaseAutoModelClass

Automatically detect Sequence to Sequence Generation Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForDepthEstimation

Bases: _BaseAutoModelClass

Automatically detect Speech Sequence to Sequence Language Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForSequenceClassification

Bases: _BaseAutoModelClass

Automatically detect Sequence Classification Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForVision2Seq

Bases: _BaseAutoModelClass

Automatically detect Vision to Sequence Generation Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForImageTextToText

Bases: _BaseAutoModelClass

Automatically detect Image and Text to Text Generation Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForMaskedLM

Bases: _BaseAutoModelClass

Automatically detect Masked Lanuage Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForAudioClassification

Bases: _BaseAutoModelClass

Automatically detect Audio Classification Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForImageClassification

Bases: _BaseAutoModelClass

Automatically detect Image Classification Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForQuestionAnswering

Bases: _BaseAutoModelClass

Automatically detect Question Answering Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False

RBLNAutoModelForTextEncoding

Bases: _BaseAutoModelClass

Automatically detect Text Encoding Models.

Functions

from_pretrained(model_id, *args, **kwargs) classmethod

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name Type Description Default
model_id

The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.

 required
args

Variable argument list. The format matches the original retrieved autoclass definition.

 ()
kwargs

Additional keyword arguments. Arguments with the prefix 'rbln_' are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.

 {}

Returns:

Type Description

A RBLN model instance ready for inference on RBLN NPU devices.
register(rbln_cls, exist_ok=False) staticmethod

Register a new RBLN model class.

Parameters:

Name Type Description Default
rbln_cls Type[RBLNBaseModel]

The RBLN model class to register.

 required
exist_ok bool

Whether to allow registering an already registered model.

 False