콘텐츠로 이동

Cosmos

Cosmos World Foundation Models는 물리 법칙을 정확하게 반영하는 비디오와 가상 세계의 상태를 생성하는 데 특화되어 있습니다. 디퓨전 기술을 기반으로 텍스트, 이미지 등 다양한 입력을 활용해 역동적이고 품질 높은 비디오를 만들어냅니다. 따라서 가상 세계 생성과 관련된 여러 연구 및 응용 분야의 핵심적인 기반 기술로 사용됩니다. RBLN Optimum을 사용하면 RBLN NPU에서 Cosmos 파이프라인을 가속할 수 있습니다.

지원 NPU

Diffusers 파이프라인은 ATOM™에서만 실행됩니다. Rebel100™은 아직 지원하지 않습니다.

지원하는 파이프라인

RBLN Optimum은 다음 Cosmos 파이프라인을 지원합니다:

파이프라인 클래스 설명
Cosmos-Predict1 Text2World RBLNCosmosTextToWorldPipeline 텍스트 프롬프트에서 비디오 생성.
Cosmos-Predict1 Video2World RBLNCosmosVideoToWorldPipeline 입력 비디오와 텍스트 프롬프트에서 비디오 생성.
Cosmos-Predict2 Text2Image RBLNCosmos2TextToImagePipeline 텍스트 프롬프트에서 이미지 생성.
Cosmos-Predict2 Video2World RBLNCosmos2VideoToWorldPipeline 입력 비디오와 텍스트 프롬프트에서 비디오 생성.
Cosmos-Predict2.5 RBLNCosmos2_5_PredictBasePipeline 텍스트, 이미지, 비디오 입력을 하나의 컴파일된 파이프라인으로 받아 비디오 생성.
Cosmos-Transfer2.5 RBLNCosmos2_5_TransferPipeline Cosmos-Predict2.5를 기반으로 edge 등 공간 제어 입력을 따르는 비디오 생성.

중요: Cosmos 가드레일

모든 Cosmos 파이프라인은 입출력의 안전성을 검사하기 위한 가드레일 모델을 safety_checker 서브모듈로 로드하며, 아래 예제처럼 다른 서브모듈과 함께 설정합니다.

가드레일은 cosmos-guardrail 패키지가 제공하며, RBLN Optimum을 설치해도 함께 설치되지 않습니다. Cosmos 파이프라인을 로드하기 전에 설치해야 합니다.

pip install "cosmos-guardrail==0.3.1" "nltk==3.9.4"

nltk는 3.9.4로 고정해야 합니다. nltk 3.10에서는 새로 받은 Hugging Face 캐시에서 가드레일 로드가 실패합니다. 최신 huggingface_hub는 가드레일 데이터 파일을 허브 공용 blobs/ 디렉터리에 저장하는데, nltk 3.10은 실제 위치가 허용된 경로 밖인 데이터 파일을 거부하기 때문입니다(Resource 'wordnet' not found, 3.10.3부터는 PermissionError).

NVIDIA Open Model License

NVIDIA Open Model License 정책에 따라 safety_checker 서브모듈이 입력 프롬프트와 생성된 비디오에 대한 가드레일 검사를 파이프라인 내부에서 자동으로 적용합니다. safety_checker를 제거하거나 비활성화해서는 안 됩니다. 용도에 맞는 동등한 수준의 가드레일로 대체하지 않는 한, 가드레일을 우회하거나 그 효력을 떨어뜨리면 라이선스에 따른 권리가 자동으로 종료됩니다.

사용 예제

텍스트-비디오

import torch
from diffusers.utils import export_to_video
from optimum.rbln import (
    RBLNCosmosTextToWorldPipeline,
    RBLNCosmosTextToWorldPipelineConfig,
)

# Create a configuration object (optional, can use defaults)
config = RBLNCosmosTextToWorldPipelineConfig(
    height=704,
    width=1280,
    transformer={
        "num_devices": 4,
        "device": [0, 1, 2, 3],
    },
    text_encoder={
        "device": 4,
    },
    vae={
        "device": 4,
    },
    safety_checker={
        "qwen3guard": {"device": 3},
        "face_blur_filter": {"device": 3},
    },
)

# Load and compile the Cosmos model for RBLN NPU
pipe = RBLNCosmosTextToWorldPipeline.from_pretrained(
    "nvidia/Cosmos-1.0-Diffusion-7B-Text2World",
    torch_dtype=torch.bfloat16,
    export=True,
    rbln_config=config,
)

# Generate a Video
prompt = "A sleek, humanoid robot stands in a vast warehouse filled with neatly stacked cardboard boxes on industrial shelves. The robot's metallic body gleams under the bright, even lighting, highlighting its futuristic design and intricate joints. A glowing blue light emanates from its chest, adding a touch of advanced technology. The background is dominated by rows of boxes, suggesting a highly organized storage system. The floor is lined with wooden pallets, enhancing the industrial setting. The camera remains static, capturing the robot's poised stance amidst the orderly environment, with a shallow depth of field that keeps the focus on the robot while subtly blurring the background for a cinematic effect."
output = pipe(prompt=prompt).frames[0]

# Save the generated video
export_to_video(output, "output.mp4", fps=30)
print("Video saved as output.mp4")

비디오-비디오

import torch
from diffusers.utils import export_to_video, load_video
from optimum.rbln import (
    RBLNCosmosVideoToWorldPipeline,
    RBLNCosmosVideoToWorldPipelineConfig,
)

# Create a configuration object (optional, can use defaults)
config = RBLNCosmosVideoToWorldPipelineConfig(
    height=704,
    width=1280,
    transformer={
        "num_devices": 4,
        "device": [0, 1, 2, 3],
    },
    text_encoder={
        "device": 4,
    },
    vae={
        "device_map": {"encoder": 4, "decoder": 5},
    },
    safety_checker={
        "qwen3guard": {"device": 5},
        "face_blur_filter": {"device": 5},
    },
)

# Load and compile the Cosmos model for RBLN NPU
pipe = RBLNCosmosVideoToWorldPipeline.from_pretrained(
    "nvidia/Cosmos-1.0-Diffusion-7B-Video2World",
    torch_dtype=torch.bfloat16,
    export=True,
    rbln_config=config,
)

# Generate a Video
video = load_video(
    "https://github.com/nvidia-cosmos/cosmos-predict1/raw/refs/heads/main/assets/diffusion/video2world_input1.mp4"
)
prompt = "A dynamic and visually captivating video showcases a sleek, dark-colored SUV driving along a narrow dirt road that runs parallel to a vast, expansive ocean. The setting is a rugged coastal landscape, with the road cutting through dry, golden-brown grass that stretches across rolling hills. The ocean, a deep blue, extends to the horizon, providing a stunning backdrop to the scene. The SUV moves swiftly along the road, kicking up a trail of dust that lingers in the air behind it, emphasizing the speed and power of the vehicle. The camera maintains a steady tracking shot, following the SUV from a slightly elevated angle, which allows for a clear view of both the vehicle and the surrounding scenery. The lighting is natural, suggesting a time of day when the sun is high, casting minimal shadows and highlighting the textures of the grass and the glint of the ocean. The video captures the essence of freedom and adventure, with the SUV navigating the isolated road with ease, suggesting a journey or exploration theme. The consistent motion of the vehicle and the dust trail create a sense of continuity and fluidity throughout the video, making it engaging and immersive."
output = pipe(video=video, prompt=prompt).frames[0]

# Save the generated video
export_to_video(output, "output.mp4", fps=30)
print("Video saved as output.mp4")

API 참조

RBLNCosmosTextToWorldPipeline

Bases: RBLNDiffusionMixin, CosmosTextToWorldPipeline

RBLN-accelerated implementation of Cosmos Text to World pipeline for text-to-video generation.

This pipeline compiles Cosmos Text to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod

Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.

This method has two distinct operating modes
  • When export=True: Takes a PyTorch-based diffusion model, compiles it for RBLN NPUs, and loads the compiled model
  • When export=False: Loads an already compiled RBLN model from model_id without recompilation

It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.

Parameters:

Name Type Description Default
model_id `str`

The model ID or path to the pretrained model to load. Can be either:

  • A model ID from the HuggingFace Hub
  • A local path to a saved model directory
required
export bool

If True, takes a PyTorch model from model_id and compiles it for RBLN NPU execution. If False, loads an already compiled RBLN model from model_id without recompilation.

False
safety_checker RBLNCosmosSafetyChecker | None

Optional custom safety checker to use instead of the default one. Only used when export=True.

None
rbln_config dict[str, Any] | None

Configuration options for RBLN compilation. Can include settings for specific submodules such as text_encoder, unet, and vae. Configuration can be tailored to the specific pipeline being compiled.

None
kwargs Any

Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used.

{}

RBLNCosmosVideoToWorldPipeline

Bases: RBLNDiffusionMixin, CosmosVideoToWorldPipeline

RBLN-accelerated implementation of Cosmos Video to World pipeline for video-to-video generation.

This pipeline compiles Cosmos Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod

Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.

This method has two distinct operating modes
  • When export=True: Takes a PyTorch-based diffusion model, compiles it for RBLN NPUs, and loads the compiled model
  • When export=False: Loads an already compiled RBLN model from model_id without recompilation

It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.

Parameters:

Name Type Description Default
model_id `str`

The model ID or path to the pretrained model to load. Can be either:

  • A model ID from the HuggingFace Hub
  • A local path to a saved model directory
required
export bool

If True, takes a PyTorch model from model_id and compiles it for RBLN NPU execution. If False, loads an already compiled RBLN model from model_id without recompilation.

False
safety_checker RBLNCosmosSafetyChecker | None

Optional custom safety checker to use instead of the default one. Only used when export=True.

None
rbln_config dict[str, Any] | None

Configuration options for RBLN compilation. Can include settings for specific submodules such as text_encoder, unet, and vae. Configuration can be tailored to the specific pipeline being compiled.

None
kwargs Any

Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used.

{}

RBLNCosmos2TextToImagePipeline

Bases: RBLNDiffusionMixin, Cosmos2TextToImagePipeline

RBLN-accelerated implementation of Cosmos Predict2 Text to Image pipeline for text-to-image generation.

This pipeline compiles Cosmos Predict2 Text to Image models to run efficiently on RBLN NPUs, enabling high-performance inference for generating images that follow physical laws with enhanced visual quality.

RBLNCosmos2VideoToWorldPipeline

Bases: RBLNDiffusionMixin, Cosmos2VideoToWorldPipeline

RBLN-accelerated implementation of Cosmos Predict2 Video to World pipeline for video-to-video generation.

This pipeline compiles Cosmos Predict2 Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

RBLNCosmos2_5_PredictBasePipeline

Bases: RBLNDiffusionMixin, Cosmos2_5_PredictBasePipeline

RBLN-accelerated implementation of Cosmos-Predict2.5 pipeline.

This pipeline compiles Cosmos-Predict2.5 models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

One pipeline serves the three conditioning modes: Text2World (no visual input), Image2World (image=...) and Video2World (video=...).

RBLNCosmos2_5_TransferPipeline

Bases: RBLNDiffusionMixin, Cosmos2_5_TransferPipeline

RBLN-accelerated implementation of the Cosmos-Transfer2.5 pipeline.

Transfer2.5 generates videos that follow a control signal (edge/depth/seg/blur maps): a 4-block ControlNet computes per-step residuals that are injected into the Predict2.5 base transformer. Long videos are generated auto-regressively in num_frames_per_chunk windows, so the compiled shapes are per chunk and the chunk loop runs on the host.

The official Hub layout keeps the ControlNet on its own revision, so a typical export is:

1
2
3
4
5
6
7
controlnet = CosmosControlNetModel.from_pretrained(
    "nvidia/Cosmos-Transfer2.5-2B", revision="diffusers/controlnet/general/edge"
)
pipe = RBLNCosmos2_5_TransferPipeline.from_pretrained(
    "nvidia/Cosmos-Transfer2.5-2B", revision="diffusers/general",
    controlnet=controlnet, export=True, ...
)

RBLNCosmosPipelineBaseConfig

Bases: RBLNModelConfig

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder RBLNT5EncoderModelConfig | None

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer RBLNCosmosTransformer3DModelConfig | None

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae RBLNAutoencoderKLCosmosConfig | None

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.

None
safety_checker RBLNCosmosSafetyCheckerConfig | None

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size int | None

Batch size for inference, applied to all submodules.

None
height int | None

Height of the generated videos.

None
width int | None

Width of the generated videos.

None
num_frames int | None

The number of frames in the generated video.

None
max_seq_len int | None

Maximum sequence length supported by the model.

None
kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

RBLNCosmosTextToWorldPipelineConfig

Bases: RBLNCosmosPipelineBaseConfig

Config for Cosmos-Predict1 Text2World Pipeline

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder RBLNT5EncoderModelConfig | None

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer RBLNCosmosTransformer3DModelConfig | None

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae RBLNAutoencoderKLCosmosConfig | None

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.

None
safety_checker RBLNCosmosSafetyCheckerConfig | None

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size int | None

Batch size for inference, applied to all submodules.

None
height int | None

Height of the generated videos.

None
width int | None

Width of the generated videos.

None
num_frames int | None

The number of frames in the generated video.

None
max_seq_len int | None

Maximum sequence length supported by the model.

None
kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

RBLNCosmosVideoToWorldPipelineConfig

Bases: RBLNCosmosPipelineBaseConfig

Config for Cosmos-Predict1 Video2World Pipeline

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder RBLNT5EncoderModelConfig | None

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer RBLNCosmosTransformer3DModelConfig | None

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae RBLNAutoencoderKLCosmosConfig | None

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.

None
safety_checker RBLNCosmosSafetyCheckerConfig | None

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size int | None

Batch size for inference, applied to all submodules.

None
height int | None

Height of the generated videos.

None
width int | None

Width of the generated videos.

None
num_frames int | None

The number of frames in the generated video.

None
max_seq_len int | None

Maximum sequence length supported by the model.

None
kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

RBLNCosmos2PipelineBaseConfig

Bases: RBLNModelConfig

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder Optional[RBLNT5EncoderModelConfig]

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer Optional[RBLNCosmosTransformer3DModelConfig]

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae Optional[RBLNAutoencoderKLWanConfig]

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.

None
safety_checker Optional[RBLNCosmosSafetyCheckerConfig]

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size Optional[int]

Batch size for inference, applied to all submodules.

None
height Optional[int]

Height of the generated videos.

None
width Optional[int]

Width of the generated videos.

None
num_frames Optional[int]

The number of frames in the generated video.

None
max_seq_len Optional[int]

Maximum sequence length supported by the model.

None
**kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

RBLNCosmos2TextToImagePipelineConfig

Bases: RBLNCosmos2PipelineBaseConfig

Config for Cosmos-Predict2 Text2Image Pipeline

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder Optional[RBLNT5EncoderModelConfig]

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer Optional[RBLNCosmosTransformer3DModelConfig]

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae Optional[RBLNAutoencoderKLWanConfig]

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.

None
safety_checker Optional[RBLNCosmosSafetyCheckerConfig]

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size Optional[int]

Batch size for inference, applied to all submodules.

None
height Optional[int]

Height of the generated videos.

None
width Optional[int]

Width of the generated videos.

None
num_frames Optional[int]

The number of frames in the generated video.

None
max_seq_len Optional[int]

Maximum sequence length supported by the model.

None
**kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

RBLNCosmos2VideoToWorldPipelineConfig

Bases: RBLNCosmos2PipelineBaseConfig

Config for Cosmos-Predict2 Video2World Pipeline

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder Optional[RBLNT5EncoderModelConfig]

Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.

None
transformer Optional[RBLNCosmosTransformer3DModelConfig]

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae Optional[RBLNAutoencoderKLWanConfig]

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.

None
safety_checker Optional[RBLNCosmosSafetyCheckerConfig]

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size Optional[int]

Batch size for inference, applied to all submodules.

None
height Optional[int]

Height of the generated videos.

None
width Optional[int]

Width of the generated videos.

None
num_frames Optional[int]

The number of frames in the generated video.

None
max_seq_len Optional[int]

Maximum sequence length supported by the model.

None
**kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

RBLNCosmos2_5_PredictBasePipelineConfig

Bases: RBLNModelConfig

Config for Cosmos-Predict2.5 Pipeline

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
text_encoder Optional[RBLNQwen2_5_VLForConditionalGenerationConfig]

Configuration for the text encoder component. Initialized as RBLNQwen2_5_VLForConditionalGenerationConfig if not provided.

None
transformer Optional[RBLNCosmosTransformer3DModelConfig]

Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.

None
vae Optional[RBLNAutoencoderKLWanConfig]

Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.

None
safety_checker Optional[RBLNCosmosSafetyCheckerConfig]

Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.

None
batch_size Optional[int]

Batch size for inference, applied to all submodules.

None
height Optional[int]

Height of the generated videos.

None
width Optional[int]

Width of the generated videos.

None
num_frames Optional[int]

The number of frames in the generated video.

None
max_seq_len Optional[int]

Maximum sequence length supported by the model.

None
**kwargs Any

Additional arguments passed to the parent RBLNModelConfig.

{}

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

RBLNCosmos2_5_TransferPipelineConfig

Bases: RBLNCosmos2_5_PredictBasePipelineConfig

Config for Cosmos-Transfer2.5 Pipeline (Predict2.5 base + ControlNet).

__init__(controlnet=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)

Parameters:

Name Type Description Default
controlnet Optional[RBLNCosmosControlNetModelConfig]

Configuration for the ControlNet component. Initialized as RBLNCosmosControlNetModelConfig if not provided; its latent geometry is copied from the transformer submodule at compile time.

None
batch_size Optional[int]

Batch size for inference, applied to all submodules.

None
height Optional[int]

Height of the generated videos.

None
width Optional[int]

Width of the generated videos.

None
num_frames Optional[int]

The number of frames per generated chunk.

None
max_seq_len Optional[int]

Maximum sequence length supported by the model.

None
**kwargs Any

Additional arguments passed to the parent config (text_encoder/transformer/vae/safety_checker).

{}

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

get_load_overrides()

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type Description
dict[str, Any]

Dictionary of load overrides, keyed like the rbln_config dict accepted by from_pretrained.

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod

Load a RBLNModelConfig from a path.

Parameters:

Name Type Description Default
path str

Path to the RBLNModelConfig file or directory containing the config file.

required
rbln_config dict[str, Any] | None

Additional configuration to override.

None
return_unused_kwargs bool

Whether to return unused kwargs.

False
kwargs Any

Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.

{}

Returns:

Name Type Description
RBLNModelConfig Self | tuple[Self, dict[str, Any]]

The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")