Cosmos¶
Cosmos World Foundation Models는 물리 법칙을 정확하게 반영하는 비디오와 가상 세계의 상태를 생성하는 데 특화되어 있습니다. 디퓨전 기술을 기반으로 텍스트, 이미지 등 다양한 입력을 활용해 역동적이고 품질 높은 비디오를 만들어냅니다. 따라서 가상 세계 생성과 관련된 여러 연구 및 응용 분야의 핵심적인 기반 기술로 사용됩니다. RBLN Optimum을 사용하면 RBLN NPU에서 Cosmos 파이프라인을 가속할 수 있습니다.
지원 NPU
Diffusers 파이프라인은 ATOM™에서만 실행됩니다. Rebel100™은 아직 지원하지 않습니다.
지원하는 파이프라인¶
RBLN Optimum은 다음 Cosmos 파이프라인을 지원합니다:
| 파이프라인 | 클래스 | 설명 |
|---|---|---|
| Cosmos-Predict1 Text2World | RBLNCosmosTextToWorldPipeline | 텍스트 프롬프트에서 비디오 생성. |
| Cosmos-Predict1 Video2World | RBLNCosmosVideoToWorldPipeline | 입력 비디오와 텍스트 프롬프트에서 비디오 생성. |
| Cosmos-Predict2 Text2Image | RBLNCosmos2TextToImagePipeline | 텍스트 프롬프트에서 이미지 생성. |
| Cosmos-Predict2 Video2World | RBLNCosmos2VideoToWorldPipeline | 입력 비디오와 텍스트 프롬프트에서 비디오 생성. |
| Cosmos-Predict2.5 | RBLNCosmos2_5_PredictBasePipeline | 텍스트, 이미지, 비디오 입력을 하나의 컴파일된 파이프라인으로 받아 비디오 생성. |
| Cosmos-Transfer2.5 | RBLNCosmos2_5_TransferPipeline | Cosmos-Predict2.5를 기반으로 edge 등 공간 제어 입력을 따르는 비디오 생성. |
중요: Cosmos 가드레일¶
모든 Cosmos 파이프라인은 입출력의 안전성을 검사하기 위한 가드레일 모델을 safety_checker 서브모듈로 로드하며, 아래 예제처럼 다른 서브모듈과 함께 설정합니다.
가드레일은 cosmos-guardrail 패키지가 제공하며, RBLN Optimum을 설치해도 함께 설치되지 않습니다. Cosmos 파이프라인을 로드하기 전에 설치해야 합니다.
nltk는 3.9.4로 고정해야 합니다. nltk 3.10에서는 새로 받은 Hugging Face 캐시에서 가드레일 로드가 실패합니다. 최신 huggingface_hub는 가드레일 데이터 파일을 허브 공용 blobs/ 디렉터리에 저장하는데, nltk 3.10은 실제 위치가 허용된 경로 밖인 데이터 파일을 거부하기 때문입니다(Resource 'wordnet' not found, 3.10.3부터는 PermissionError).
NVIDIA Open Model License
NVIDIA Open Model License 정책에 따라 safety_checker 서브모듈이 입력 프롬프트와 생성된 비디오에 대한 가드레일 검사를 파이프라인 내부에서 자동으로 적용합니다. safety_checker를 제거하거나 비활성화해서는 안 됩니다. 용도에 맞는 동등한 수준의 가드레일로 대체하지 않는 한, 가드레일을 우회하거나 그 효력을 떨어뜨리면 라이선스에 따른 권리가 자동으로 종료됩니다.
사용 예제¶
텍스트-비디오¶
비디오-비디오¶
API 참조¶
RBLNCosmosTextToWorldPipeline ¶
Bases: RBLNDiffusionMixin, CosmosTextToWorldPipeline
RBLN-accelerated implementation of Cosmos Text to World pipeline for text-to-video generation.
This pipeline compiles Cosmos Text to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.
from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod ¶
Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.
It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_id | `str` | The model ID or path to the pretrained model to load. Can be either:
| required |
export | bool | If True, takes a PyTorch model from | False |
safety_checker | RBLNCosmosSafetyChecker | None | Optional custom safety checker to use instead of the default one. Only used when | None |
rbln_config | dict[str, Any] | None | Configuration options for RBLN compilation. Can include settings for specific submodules such as | None |
kwargs | Any | Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used. | {} |
RBLNCosmosVideoToWorldPipeline ¶
Bases: RBLNDiffusionMixin, CosmosVideoToWorldPipeline
RBLN-accelerated implementation of Cosmos Video to World pipeline for video-to-video generation.
This pipeline compiles Cosmos Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.
from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod ¶
Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.
It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_id | `str` | The model ID or path to the pretrained model to load. Can be either:
| required |
export | bool | If True, takes a PyTorch model from | False |
safety_checker | RBLNCosmosSafetyChecker | None | Optional custom safety checker to use instead of the default one. Only used when | None |
rbln_config | dict[str, Any] | None | Configuration options for RBLN compilation. Can include settings for specific submodules such as | None |
kwargs | Any | Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used. | {} |
RBLNCosmos2TextToImagePipeline ¶
Bases: RBLNDiffusionMixin, Cosmos2TextToImagePipeline
RBLN-accelerated implementation of Cosmos Predict2 Text to Image pipeline for text-to-image generation.
This pipeline compiles Cosmos Predict2 Text to Image models to run efficiently on RBLN NPUs, enabling high-performance inference for generating images that follow physical laws with enhanced visual quality.
RBLNCosmos2VideoToWorldPipeline ¶
Bases: RBLNDiffusionMixin, Cosmos2VideoToWorldPipeline
RBLN-accelerated implementation of Cosmos Predict2 Video to World pipeline for video-to-video generation.
This pipeline compiles Cosmos Predict2 Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.
RBLNCosmos2_5_PredictBasePipeline ¶
Bases: RBLNDiffusionMixin, Cosmos2_5_PredictBasePipeline
RBLN-accelerated implementation of Cosmos-Predict2.5 pipeline.
This pipeline compiles Cosmos-Predict2.5 models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.
One pipeline serves the three conditioning modes: Text2World (no visual input), Image2World (image=...) and Video2World (video=...).
RBLNCosmos2_5_TransferPipeline ¶
Bases: RBLNDiffusionMixin, Cosmos2_5_TransferPipeline
RBLN-accelerated implementation of the Cosmos-Transfer2.5 pipeline.
Transfer2.5 generates videos that follow a control signal (edge/depth/seg/blur maps): a 4-block ControlNet computes per-step residuals that are injected into the Predict2.5 base transformer. Long videos are generated auto-regressively in num_frames_per_chunk windows, so the compiled shapes are per chunk and the chunk loop runs on the host.
The official Hub layout keeps the ControlNet on its own revision, so a typical export is:
RBLNCosmosPipelineBaseConfig ¶
Bases: RBLNModelConfig
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | RBLNT5EncoderModelConfig | None | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | RBLNCosmosTransformer3DModelConfig | None | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | RBLNAutoencoderKLCosmosConfig | None | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided. | None |
safety_checker | RBLNCosmosSafetyCheckerConfig | None | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | int | None | Batch size for inference, applied to all submodules. | None |
height | int | None | Height of the generated videos. | None |
width | int | None | Width of the generated videos. | None |
num_frames | int | None | The number of frames in the generated video. | None |
max_seq_len | int | None | Maximum sequence length supported by the model. | None |
kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
RBLNCosmosTextToWorldPipelineConfig ¶
Bases: RBLNCosmosPipelineBaseConfig
Config for Cosmos-Predict1 Text2World Pipeline
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | RBLNT5EncoderModelConfig | None | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | RBLNCosmosTransformer3DModelConfig | None | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | RBLNAutoencoderKLCosmosConfig | None | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided. | None |
safety_checker | RBLNCosmosSafetyCheckerConfig | None | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | int | None | Batch size for inference, applied to all submodules. | None |
height | int | None | Height of the generated videos. | None |
width | int | None | Width of the generated videos. | None |
num_frames | int | None | The number of frames in the generated video. | None |
max_seq_len | int | None | Maximum sequence length supported by the model. | None |
kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
RBLNCosmosVideoToWorldPipelineConfig ¶
Bases: RBLNCosmosPipelineBaseConfig
Config for Cosmos-Predict1 Video2World Pipeline
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | RBLNT5EncoderModelConfig | None | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | RBLNCosmosTransformer3DModelConfig | None | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | RBLNAutoencoderKLCosmosConfig | None | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided. | None |
safety_checker | RBLNCosmosSafetyCheckerConfig | None | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | int | None | Batch size for inference, applied to all submodules. | None |
height | int | None | Height of the generated videos. | None |
width | int | None | Width of the generated videos. | None |
num_frames | int | None | The number of frames in the generated video. | None |
max_seq_len | int | None | Maximum sequence length supported by the model. | None |
kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
RBLNCosmos2PipelineBaseConfig ¶
Bases: RBLNModelConfig
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | Optional[RBLNT5EncoderModelConfig] | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | Optional[RBLNCosmosTransformer3DModelConfig] | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | Optional[RBLNAutoencoderKLWanConfig] | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided. | None |
safety_checker | Optional[RBLNCosmosSafetyCheckerConfig] | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | Optional[int] | Batch size for inference, applied to all submodules. | None |
height | Optional[int] | Height of the generated videos. | None |
width | Optional[int] | Width of the generated videos. | None |
num_frames | Optional[int] | The number of frames in the generated video. | None |
max_seq_len | Optional[int] | Maximum sequence length supported by the model. | None |
**kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
RBLNCosmos2TextToImagePipelineConfig ¶
Bases: RBLNCosmos2PipelineBaseConfig
Config for Cosmos-Predict2 Text2Image Pipeline
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | Optional[RBLNT5EncoderModelConfig] | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | Optional[RBLNCosmosTransformer3DModelConfig] | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | Optional[RBLNAutoencoderKLWanConfig] | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided. | None |
safety_checker | Optional[RBLNCosmosSafetyCheckerConfig] | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | Optional[int] | Batch size for inference, applied to all submodules. | None |
height | Optional[int] | Height of the generated videos. | None |
width | Optional[int] | Width of the generated videos. | None |
num_frames | Optional[int] | The number of frames in the generated video. | None |
max_seq_len | Optional[int] | Maximum sequence length supported by the model. | None |
**kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
RBLNCosmos2VideoToWorldPipelineConfig ¶
Bases: RBLNCosmos2PipelineBaseConfig
Config for Cosmos-Predict2 Video2World Pipeline
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | Optional[RBLNT5EncoderModelConfig] | Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided. | None |
transformer | Optional[RBLNCosmosTransformer3DModelConfig] | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | Optional[RBLNAutoencoderKLWanConfig] | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided. | None |
safety_checker | Optional[RBLNCosmosSafetyCheckerConfig] | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | Optional[int] | Batch size for inference, applied to all submodules. | None |
height | Optional[int] | Height of the generated videos. | None |
width | Optional[int] | Width of the generated videos. | None |
num_frames | Optional[int] | The number of frames in the generated video. | None |
max_seq_len | Optional[int] | Maximum sequence length supported by the model. | None |
**kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
RBLNCosmos2_5_PredictBasePipelineConfig ¶
Bases: RBLNModelConfig
Config for Cosmos-Predict2.5 Pipeline
__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text_encoder | Optional[RBLNQwen2_5_VLForConditionalGenerationConfig] | Configuration for the text encoder component. Initialized as RBLNQwen2_5_VLForConditionalGenerationConfig if not provided. | None |
transformer | Optional[RBLNCosmosTransformer3DModelConfig] | Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided. | None |
vae | Optional[RBLNAutoencoderKLWanConfig] | Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided. | None |
safety_checker | Optional[RBLNCosmosSafetyCheckerConfig] | Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided. | None |
batch_size | Optional[int] | Batch size for inference, applied to all submodules. | None |
height | Optional[int] | Height of the generated videos. | None |
width | Optional[int] | Width of the generated videos. | None |
num_frames | Optional[int] | The number of frames in the generated video. | None |
max_seq_len | Optional[int] | Maximum sequence length supported by the model. | None |
**kwargs | Any | Additional arguments passed to the parent RBLNModelConfig. | {} |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples:
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
RBLNCosmos2_5_TransferPipelineConfig ¶
Bases: RBLNCosmos2_5_PredictBasePipelineConfig
Config for Cosmos-Transfer2.5 Pipeline (Predict2.5 base + ControlNet).
__init__(controlnet=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
controlnet | Optional[RBLNCosmosControlNetModelConfig] | Configuration for the ControlNet component. Initialized as RBLNCosmosControlNetModelConfig if not provided; its latent geometry is copied from the transformer submodule at compile time. | None |
batch_size | Optional[int] | Batch size for inference, applied to all submodules. | None |
height | Optional[int] | Height of the generated videos. | None |
width | Optional[int] | Width of the generated videos. | None |
num_frames | Optional[int] | The number of frames per generated chunk. | None |
max_seq_len | Optional[int] | Maximum sequence length supported by the model. | None |
**kwargs | Any | Additional arguments passed to the parent config (text_encoder/transformer/vae/safety_checker). | {} |
initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶
Build the config of a submodule from the parent's runtime options and kwargs.
Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.
get_load_overrides() ¶
Extract the options that must cross the load boundary, for passing to from_pretrained.
Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Dictionary of load overrides, keyed like the |
from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶
Load a RBLNModelConfig from a path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path | str | Path to the RBLNModelConfig file or directory containing the config file. | required |
rbln_config | dict[str, Any] | None | Additional configuration to override. | None |
return_unused_kwargs | bool | Whether to return unused kwargs. | False |
kwargs | Any | Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration. | {} |
Returns:
| Name | Type | Description |
|---|---|---|
RBLNModelConfig | Self | tuple[Self, dict[str, Any]] | The loaded configuration instance. |
Note
This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.
Examples: