Qwen3.5¶

Qwen3.5 모델은 텍스트와 이미지를 입력받아 텍스트를 생성하는 Qwen 시리즈의 통합 비전-언어 모델입니다. 멀티모달 토큰을 초기에 융합하는 얼리 퓨전(early fusion) 방식으로 학습되어 하나의 모델이 텍스트 전용 작업과 비전-언어 작업을 모두 처리하며, 비전-언어 변형과 텍스트 전용 인과적 언어 모델 변형이 함께 제공됩니다. 하이브리드 디코더는 Gated DeltaNet 방식의 선형 어텐션(linear attention) 레이어와 게이트 방식의 풀 어텐션(gated full attention) 레이어를 교차 배치하여 긴 컨텍스트를 생성할 때 효율을 유지합니다. 네이티브 컨텍스트는 262,144 토큰이고, 201개 언어와 방언을 지원합니다. RBLN NPU는 Optimum RBLN을 사용하여 Qwen3.5 모델 추론을 가속화할 수 있습니다.

API 참조¶

Classes¶

`RBLNQwen3_5TextModel` ¶

Bases: RBLNDecoderOnlyModel

The bare Qwen3.5 text backbone (no LM head).

Qwen3.5 is a hybrid decoder: full_attention layers use the standard paged KV cache, while linear_attention (GatedDeltaNet) layers carry a conv_state + recurrent_state instead. The two state tensors reuse the layer's two past_key_values slots positionally. This class owns the hybrid wiring — get_input_info (per-layer tensor specs), setup_runtime (the mask-injecting RBLNQwen3_5RuntimeModel), _get_compile_context (mark conv/recurrent static) and _update_rbln_config (validate layer_types). RBLNQwen3_5ForCausalLM adds the LM head on top, mirroring how RBLNDecoderOnlyModelForCausalLM extends RBLNDecoderOnlyModel.

Methods:¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name	Type	Description	Default
`model_id`	`str \| Path`	The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.	required
`export`	`bool \| None`	A boolean flag to indicate whether the model should be compiled. If None, it will be determined based on the existence of the compiled model files in the model_id.	`None`
`rbln_config`	`dict \| RBLNModelConfig \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

Saves a model and its configuration file to a directory, so that it can be re-loaded using the [~optimum.rbln.modeling_base.RBLNBaseModel.from_pretrained] class method.

Parameters:

Name	Type	Description	Default
`save_directory`	`str \| Path`	Directory where to save the model file.	required
`push_to_hub`	`bool`	Whether or not to push your model to the HuggingFace model hub after saving it.	`False`

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

Converts and compiles a pre-trained HuggingFace library model into a RBLN model. This method performs the actual model conversion and compilation process.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	The PyTorch model to be compiled. The object must be an instance of the HuggingFace transformers PreTrainedModel class.	required
`config`	`PretrainedConfig \| None`	The configuration object associated with the model.	`None`
`rbln_config`	`RBLNModelConfig \| dict \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

The method performs the following steps:

Compiles the PyTorch model into an optimized RBLN graph
Configures the model for the specified NPU device
Creates the necessary runtime objects if requested
Saves the compiled model and configurations

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`forward(input_ids=None, inputs_embeds=None, attention_mask=None, position_ids=None, position_embed=None, output_hidden_states=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`input_ids`	`LongTensor`	The input IDs to the model.	`None`
`inputs_embeds`	`Tensor`	The input embeddings to the model.	`None`
`attention_mask`	`LongTensor`	The attention mask to the model.	`None`
`kwargs`	`dict[str, Any]`	Additional keyword arguments.	`{}`

Returns:

Type	Description
`BaseModelOutputWithPast`	Dataclass containing the last hidden states of the model.

`RBLNQwen3_5ForCausalLM` ¶

Bases: RBLNQwen3_5TextModel, RBLNDecoderOnlyModelForCausalLM

RBLNQwen3_5ForCausalLM is the text-only (causal language modeling) variant of Qwen3.5, optimized for RBLN NPUs. It runs the hybrid Qwen3.5 decoder — GatedDeltaNet linear_attention layers interleaved with gated full_attention layers — without the vision encoder.

This model inherits from [RBLNDecoderOnlyModelForCausalLM]. Check the superclass documentation for the generic methods the library implements for all its models.

Important Note

This model includes a Large Language Model (LLM). For optimal performance, it is highly recommended to use tensor parallelism for the language model. This can be achieved by using the rbln_config parameter in the from_pretrained method. Refer to the from_pretrained documentation and the RBLNQwen3_5ForCausalLMConfig class for details.

Examples:

from optimum.rbln.transformers.models.qwen3_5 import RBLNQwen3_5ForCausalLM

model = RBLNQwen3_5ForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-0.8B",
    export=True,
    rbln_config={
        "num_devices": 1,
        "kvcache_partition_len": 4096,
        "max_seq_len": 8192,
        "device": 0,
    },
)

model.save_pretrained("compiled-qwen3.5-0.8b")

Methods:¶

`generate(input_ids, attention_mask=None, generation_config=None, **kwargs)` ¶

The generate function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to generate text from the model. Check the HuggingFace transformers documentation for more details.

Parameters:

Name	Type	Description	Default
`input_ids`	`LongTensor`	The input ids to the model.	required
`attention_mask`	`LongTensor`	The attention mask to the model.	`None`
`generation_config`	`GenerationConfig`	The generation configuration to be used as base parametrization for the generation call. **kwargs passed to generate matching the attributes of generation_config will override them. If generation_config is not provided, the default will be used, which had the following loading priority: 1) from the generation_config.json model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit GenerationConfig’s default values.	`None`
`kwargs`	`dict[str, Any]`	Additional arguments passed to the generate function. See the HuggingFace transformers documentation for more details.	`{}`

Returns:

Type	Description
`ModelOutput \| LongTensor`	A ModelOutput (if return_dict_in_generate=True or when config.return_dict_in_generate=True) or a torch.LongTensor.

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name	Type	Description	Default
`model_id`	`str \| Path`	The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.	required
`export`	`bool \| None`	A boolean flag to indicate whether the model should be compiled. If None, it will be determined based on the existence of the compiled model files in the model_id.	`None`
`rbln_config`	`dict \| RBLNModelConfig \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

Saves a model and its configuration file to a directory, so that it can be re-loaded using the [~optimum.rbln.modeling_base.RBLNBaseModel.from_pretrained] class method.

Parameters:

Name	Type	Description	Default
`save_directory`	`str \| Path`	Directory where to save the model file.	required
`push_to_hub`	`bool`	Whether or not to push your model to the HuggingFace model hub after saving it.	`False`

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

Converts and compiles a pre-trained HuggingFace library model into a RBLN model. This method performs the actual model conversion and compilation process.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	The PyTorch model to be compiled. The object must be an instance of the HuggingFace transformers PreTrainedModel class.	required
`config`	`PretrainedConfig \| None`	The configuration object associated with the model.	`None`
`rbln_config`	`RBLNModelConfig \| dict \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

The method performs the following steps:

Compiles the PyTorch model into an optimized RBLN graph
Configures the model for the specified NPU device
Creates the necessary runtime objects if requested
Saves the compiled model and configurations

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`set_adapter(adapter_name)` ¶

Sets the active adapter(s) for the model using adapter name(s).

Parameters:

Name	Type	Description	Default
`adapter_name`	`str \| list[str]`	The name(s) of the adapter(s) to be activated. Can be a single adapter name or a list of adapter names.	required

Raises:

Type	Description
`ValueError`	If the model is not configured with LoRA or if the adapter name is not found.

`RBLNQwen3_5VisionModel` ¶

Bases: RBLNModel

Qwen3.5 vision encoder for RBLN — a Qwen3-VL-style vision tower WITHOUT deepstack.

The per-image window padding / rotary / position embedding interpolation helpers are defined here.

Methods:¶

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

Converts and compiles a pre-trained HuggingFace library model into a RBLN model. This method performs the actual model conversion and compilation process.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	The PyTorch model to be compiled. The object must be an instance of the HuggingFace transformers PreTrainedModel class.	required
`config`	`PretrainedConfig \| None`	The configuration object associated with the model.	`None`
`rbln_config`	`RBLNModelConfig \| dict \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

The method performs the following steps:

Compiles the PyTorch model into an optimized RBLN graph
Configures the model for the specified NPU device
Creates the necessary runtime objects if requested
Saves the compiled model and configurations

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name	Type	Description	Default
`model_id`	`str \| Path`	The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.	required
`export`	`bool \| None`	A boolean flag to indicate whether the model should be compiled. If None, it will be determined based on the existence of the compiled model files in the model_id.	`None`
`rbln_config`	`dict \| RBLNModelConfig \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

Saves a model and its configuration file to a directory, so that it can be re-loaded using the [~optimum.rbln.modeling_base.RBLNBaseModel.from_pretrained] class method.

Parameters:

Name	Type	Description	Default
`save_directory`	`str \| Path`	Directory where to save the model file.	required
`push_to_hub`	`bool`	Whether or not to push your model to the HuggingFace model hub after saving it.	`False`

`RBLNQwen3_5ForConditionalGeneration` ¶

Bases: RBLNQwen3_5Model, RBLNDecoderOnlyModelForCausalLM

RBLNQwen3_5ForConditionalGeneration is a multi-modal model that integrates vision and language processing capabilities, optimized for RBLN NPUs. It is designed for conditional generation tasks that involve both image and text inputs. It pairs a vision encoder with the hybrid Qwen3.5 text backbone — GatedDeltaNet linear_attention layers interleaved with gated full_attention layers.

This model inherits from [RBLNDecoderOnlyModelForCausalLM]. Check the superclass documentation for the generic methods the library implements for all its models.

Important Note

This model includes a Large Language Model (LLM). For optimal performance, it is highly recommended to use tensor parallelism for the language model. This can be achieved by using the rbln_config parameter in the from_pretrained method. Refer to the from_pretrained documentation and the RBLNQwen3_5ForConditionalGenerationConfig class for details.

Examples:

from optimum.rbln import RBLNQwen3_5ForConditionalGeneration

model = RBLNQwen3_5ForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3.5-27B",
    export=True,
    rbln_config={
        "visual": {
            "num_devices": 8,
            "max_seq_len": 16384,
            "device": [0, 1, 2, 3, 4, 5, 6, 7],
        },
        "num_devices": 8,
        "kvcache_partition_len": 16384,
        "max_seq_len": 262144,
        "device": [0, 1, 2, 3, 4, 5, 6, 7],
    },
)

model.save_pretrained("qwen3.5-27B")

Methods:¶

`generate(input_ids, attention_mask=None, generation_config=None, **kwargs)` ¶

The generate function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to generate text from the model. Check the HuggingFace transformers documentation for more details.

Parameters:

Name	Type	Description	Default
`input_ids`	`LongTensor`	The input ids to the model.	required
`attention_mask`	`LongTensor`	The attention mask to the model.	`None`
`generation_config`	`GenerationConfig`	The generation configuration to be used as base parametrization for the generation call. **kwargs passed to generate matching the attributes of generation_config will override them. If generation_config is not provided, the default will be used, which had the following loading priority: 1) from the generation_config.json model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit GenerationConfig’s default values.	`None`
`kwargs`	`dict[str, Any]`	Additional arguments passed to the generate function. See the HuggingFace transformers documentation for more details.	`{}`

Returns:

Type	Description
`ModelOutput \| LongTensor`	A ModelOutput (if return_dict_in_generate=True or when config.return_dict_in_generate=True) or a torch.LongTensor.

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

The from_pretrained() function is utilized in its standard form as in the HuggingFace transformers library. User can use this function to load a pre-trained model from the HuggingFace library and convert it to a RBLN model to be run on RBLN NPUs.

Parameters:

Name	Type	Description	Default
`model_id`	`str \| Path`	The model id of the pre-trained model to be loaded. It can be downloaded from the HuggingFace model hub or a local path, or a model id of a compiled model using the RBLN Compiler.	required
`export`	`bool \| None`	A boolean flag to indicate whether the model should be compiled. If None, it will be determined based on the existence of the compiled model files in the model_id.	`None`
`rbln_config`	`dict \| RBLNModelConfig \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

Saves a model and its configuration file to a directory, so that it can be re-loaded using the [~optimum.rbln.modeling_base.RBLNBaseModel.from_pretrained] class method.

Parameters:

Name	Type	Description	Default
`save_directory`	`str \| Path`	Directory where to save the model file.	required
`push_to_hub`	`bool`	Whether or not to push your model to the HuggingFace model hub after saving it.	`False`

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

Converts and compiles a pre-trained HuggingFace library model into a RBLN model. This method performs the actual model conversion and compilation process.

Parameters:

Name	Type	Description	Default
`model`	`PreTrainedModel`	The PyTorch model to be compiled. The object must be an instance of the HuggingFace transformers PreTrainedModel class.	required
`config`	`PretrainedConfig \| None`	The configuration object associated with the model.	`None`
`rbln_config`	`RBLNModelConfig \| dict \| None`	Configuration for RBLN model compilation and runtime. This can be provided as a dictionary or an instance of the model's configuration class (e.g., `RBLNLlamaForCausalLMConfig` for Llama models). For detailed configuration options, see the specific model's configuration class documentation.	`None`
`kwargs`	`Any`	Additional keyword arguments. Arguments with the prefix `rbln_` are passed to rbln_config, while the remaining arguments are passed to the HuggingFace library.	`{}`

The method performs the following steps:

Compiles the PyTorch model into an optimized RBLN graph
Configures the model for the specified NPU device
Creates the necessary runtime objects if requested
Saves the compiled model and configurations

Returns:

Type	Description
`RBLNModel`	A RBLN model instance ready for inference on RBLN NPU devices.

`set_adapter(adapter_name)` ¶

Sets the active adapter(s) for the model using adapter name(s).

Parameters:

Name	Type	Description	Default
`adapter_name`	`str \| list[str]`	The name(s) of the adapter(s) to be activated. Can be a single adapter name or a list of adapter names.	required

Raises:

Type	Description
`ValueError`	If the model is not configured with LoRA or if the adapter name is not found.

Classes¶

`RBLNQwen3_5ForCausalLMConfig` ¶

Bases: RBLNDecoderOnlyModelForCausalLMConfig

Configuration class for RBLN Qwen3.5 (text backbone) causal language models.

Qwen3.5 is a hybrid decoder: most layers are linear_attention (GatedDeltaNet) and a minority are full_attention (gated softmax attention). Full-attention layers use the standard paged KV cache; linear-attention layers instead carry a conv_state and a recurrent_state. Which layers are linear is read directly from the HF config.layer_types (no RBLN-config field); this config extends RBLNDecoderOnlyModelForCausalLMConfig only with gdn_chunk_size.

Example usage:

from optimum.rbln import RBLNQwen3_5ForCausalLM, RBLNQwen3_5ForCausalLMConfig

config = RBLNQwen3_5ForCausalLMConfig(
    batch_size=1,
    max_seq_len=32768,
    tensor_parallel_size=4,
)
model = RBLNQwen3_5ForCausalLM.from_pretrained("Qwen/Qwen3.5-27B", export=True, rbln_config=config)

Attributes¶

`is_auto_num_blocks` `property` ¶

Returns True if kvcache_num_blocks will be automatically determined during compilation to fit within the available DRAM on the NPU.

Methods:¶

`init(gdn_chunk_size=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`gdn_chunk_size`	`Optional[int]`	GatedDeltaNet prefill sub-chunk size. Each prefill window is split into `prefill_chunk_size // gdn_chunk_size` sub-chunks processed by the chunked delta rule. Must divide `prefill_chunk_size`. `None` -> `prefill_chunk_size` (no split).	`None`
`kwargs`	`Any`	Additional arguments passed to `RBLNDecoderOnlyModelForCausalLMConfig`.	`{}`

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`dict[str, Any] \| None`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNQwen3_5TextModelConfig` ¶

Bases: RBLNDecoderOnlyModelConfig

Configuration class for the bare RBLN Qwen3.5 text backbone (no LM head, text-only).

Linear-attention layers are read from the HF config.layer_types; see RBLNQwen3_5ForCausalLMConfig for gdn_chunk_size.

Attributes¶

`is_auto_num_blocks` `property` ¶

Returns True if kvcache_num_blocks will be automatically determined during compilation to fit within the available DRAM on the NPU.

Methods:¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`dict[str, Any] \| None`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNQwen3_5VisionModelConfig` ¶

Bases: RBLNModelConfig

Vision encoder config for Qwen3.5: per-image max_seq_len.

Methods:¶

`init(max_seq_len=None, batch_size=1, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`max_seq_len`	`Optional[Union[int, List[int]]]`	Vision Transformer attention max sequence length(s) = number of (merged) patches per image/video. RBLN runs inference per image, so set this to the max expected resolution to bound compute. Required.	`None`
`batch_size`	`int`	the vision encoder runs one image at a time (the parent config forces this by default).	`1`
`kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

Raises:

Type	Description
`ValueError`	If `max_seq_len` is None or not provided, or if `batch_size` is not 1.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`dict[str, Any] \| None`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNQwen3_5ModelConfig` ¶

Bases: RBLNDecoderOnlyModelConfig

Configuration for the bare Qwen3.5 model (vision encoder + hybrid text, no LM head).

Qwen3.5 is natively vision-language, so this is the multimodal model config. Independent of the Qwen3-VL config (inherits RBLNDecoderOnlyModelConfig directly), carrying its own visual submodule handling plus the Qwen3.5-specific gdn_chunk_size. Which layers are linear is read from the HF config.text_config.layer_types. The vision encoder output is injected into inputs_embeds (use_inputs_embeds=True).

Attributes¶

`is_auto_num_blocks` `property` ¶

Returns True if kvcache_num_blocks will be automatically determined during compilation to fit within the available DRAM on the NPU.

Methods:¶

`init(gdn_chunk_size=None, visual=None, _load_visual_runtime=True, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`gdn_chunk_size`	`Optional[int]`	GatedDeltaNet prefill sub-chunk size. Each prefill window is split into `prefill_chunk_size // gdn_chunk_size` sub-chunks processed by the chunked delta rule. Must divide `prefill_chunk_size`. `None` -> `prefill_chunk_size` (no split).	`None`
`visual`	`Optional[RBLNModelConfig]`	Configuration for the vision encoder submodule.	`None`
`_load_visual_runtime`	`bool`	Whether to create the visual encoder runtime (False on decoder-only nodes in a disaggregated setup). Defaults to True.	`True`
`kwargs`	`Any`	Additional arguments passed to `RBLNDecoderOnlyModelConfig`.	`{}`

Raises:

Type	Description
`ValueError`	If `use_inputs_embeds` is False.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`dict[str, Any] \| None`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNQwen3_5ForConditionalGenerationConfig` ¶

Bases: RBLNDecoderOnlyModelForCausalLMConfig

Configuration for RBLNQwen3_5ForConditionalGeneration (vision-language).

Qwen3.5 pairs a Qwen3-VL-style vision encoder (no deepstack) with the hybrid Qwen3.5 text backbone (linear_attention GatedDeltaNet layers + full_attention gated layers). The vision encoder output is injected into inputs_embeds (use_inputs_embeds=True). Which layers are linear is read from the HF config.text_config.layer_types (no RBLN-config field).

Independent of the Qwen3-VL config: inherits RBLNDecoderOnlyModelForCausalLMConfig directly (like the Qwen3-VL config does), carrying its own visual submodule handling plus the Qwen3.5-specific gdn_chunk_size.

Example usage:

from optimum.rbln import RBLNQwen3_5ForConditionalGeneration

model = RBLNQwen3_5ForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3.5-...", export=True,
    rbln_config={"max_seq_len": 32768, "tensor_parallel_size": 4, "visual": {"max_seq_len": 6400}},
)

Attributes¶

`is_auto_num_blocks` `property` ¶

Returns True if kvcache_num_blocks will be automatically determined during compilation to fit within the available DRAM on the NPU.

Methods:¶

`init(gdn_chunk_size=None, use_inputs_embeds=True, visual=None, _load_visual_runtime=True, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`gdn_chunk_size`	`Optional[int]`	GatedDeltaNet prefill sub-chunk size. Each prefill window is split into `prefill_chunk_size // gdn_chunk_size` sub-chunks. Must divide `prefill_chunk_size`. `None` -> `prefill_chunk_size` (no split). See rbln_chunk_gated_delta_rule.	`None`
`use_inputs_embeds`	`bool`	Must be True — the vision encoder output is injected into inputs_embeds.	`True`
`visual`	`Optional[RBLNModelConfig]`	Configuration for the vision encoder submodule.	`None`
`_load_visual_runtime`	`bool`	Whether to create the visual encoder runtime. Set False on decoder-only nodes in a disaggregated setup (then pre-computed image_embeds must be fed to forward()). Defaults to True.	`True`
`kwargs`	`Any`	Additional arguments passed to `RBLNDecoderOnlyModelForCausalLMConfig`.	`{}`

Raises:

Type	Description
`ValueError`	If `use_inputs_embeds` is False.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`dict[str, Any] \| None`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Union[RBLNModelConfig, tuple[RBLNModelConfig, dict[str, Any]]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

Qwen3.5¶

API 참조¶

Classes¶

RBLNQwen3_5TextModel ¶

Methods:¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

save_pretrained(save_directory, push_to_hub=False, **kwargs) ¶

from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs) classmethod ¶

forward(input_ids=None, inputs_embeds=None, attention_mask=None, position_ids=None, position_embed=None, output_hidden_states=None, **kwargs) ¶

RBLNQwen3_5ForCausalLM ¶

Methods:¶

generate(input_ids, attention_mask=None, generation_config=None, **kwargs) ¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

save_pretrained(save_directory, push_to_hub=False, **kwargs) ¶

from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs) classmethod ¶

set_adapter(adapter_name) ¶

RBLNQwen3_5VisionModel ¶

Methods:¶

from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs) classmethod ¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

save_pretrained(save_directory, push_to_hub=False, **kwargs) ¶

RBLNQwen3_5ForConditionalGeneration ¶

Methods:¶

generate(input_ids, attention_mask=None, generation_config=None, **kwargs) ¶

from_pretrained(model_id, export=None, rbln_config=None, **kwargs) classmethod ¶

save_pretrained(save_directory, push_to_hub=False, **kwargs) ¶

from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs) classmethod ¶

set_adapter(adapter_name) ¶

Classes¶

RBLNQwen3_5ForCausalLMConfig ¶

Attributes¶

is_auto_num_blocks property ¶

Methods:¶

__init__(gdn_chunk_size=None, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNQwen3_5TextModelConfig ¶

Attributes¶

is_auto_num_blocks property ¶

Methods:¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNQwen3_5VisionModelConfig ¶

Methods:¶

__init__(max_seq_len=None, batch_size=1, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNQwen3_5ModelConfig ¶

Attributes¶

is_auto_num_blocks property ¶

Methods:¶

__init__(gdn_chunk_size=None, visual=None, _load_visual_runtime=True, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNQwen3_5ForConditionalGenerationConfig ¶

Attributes¶

is_auto_num_blocks property ¶

Methods:¶

__init__(gdn_chunk_size=None, use_inputs_embeds=True, visual=None, _load_visual_runtime=True, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

`RBLNQwen3_5TextModel` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

`forward(input_ids=None, inputs_embeds=None, attention_mask=None, position_ids=None, position_embed=None, output_hidden_states=None, **kwargs)` ¶

`RBLNQwen3_5ForCausalLM` ¶

`generate(input_ids, attention_mask=None, generation_config=None, **kwargs)` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

`set_adapter(adapter_name)` ¶

`RBLNQwen3_5VisionModel` ¶

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

`RBLNQwen3_5ForConditionalGeneration` ¶

`generate(input_ids, attention_mask=None, generation_config=None, **kwargs)` ¶

`from_pretrained(model_id, export=None, rbln_config=None, **kwargs)` `classmethod` ¶

`save_pretrained(save_directory, push_to_hub=False, **kwargs)` ¶

`from_model(model, config=None, rbln_config=None, model_save_dir=None, subfolder='', **kwargs)` `classmethod` ¶

`set_adapter(adapter_name)` ¶

`RBLNQwen3_5ForCausalLMConfig` ¶

`is_auto_num_blocks` `property` ¶

`init(gdn_chunk_size=None, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNQwen3_5TextModelConfig` ¶

`is_auto_num_blocks` `property` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNQwen3_5VisionModelConfig` ¶

`init(max_seq_len=None, batch_size=1, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNQwen3_5ModelConfig` ¶

`is_auto_num_blocks` `property` ¶

`init(gdn_chunk_size=None, visual=None, _load_visual_runtime=True, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNQwen3_5ForConditionalGenerationConfig` ¶

`is_auto_num_blocks` `property` ¶

`init(gdn_chunk_size=None, use_inputs_embeds=True, visual=None, _load_visual_runtime=True, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶