Cosmos¶

Cosmos World Foundation Models are specialized to generate videos and world states that accurately adhere to the laws of physics. Using diffusion-based technology, these models create dynamic, high-quality videos from various inputs like text, images, or videos. RBLN NPUs can accelerate Cosmos pipelines using RBLN Optimum.

Supported NPU

Diffusers pipelines run on ATOM™ only. Rebel100™ does not support them yet.

Supported Pipelines¶

RBLN Optimum supports the following Cosmos pipelines:

Pipeline	Class	Description
Cosmos-Predict1 Text2World	`RBLNCosmosTextToWorldPipeline`	Generate videos from text prompts.
Cosmos-Predict1 Video2World	`RBLNCosmosVideoToWorldPipeline`	Generate videos from an input video and a text prompt.
Cosmos-Predict2 Text2Image	`RBLNCosmos2TextToImagePipeline`	Generate images from text prompts.
Cosmos-Predict2 Video2World	`RBLNCosmos2VideoToWorldPipeline`	Generate videos from an input video and a text prompt.
Cosmos-Predict2.5	`RBLNCosmos2_5_PredictBasePipeline`	Generate videos from text, image, or video inputs through one compiled pipeline.
Cosmos-Transfer2.5	`RBLNCosmos2_5_TransferPipeline`	Generate videos that follow a spatial control input such as edge, on top of Cosmos-Predict2.5.

Important: Cosmos Safety Guardrails¶

Every Cosmos pipeline loads a guardrail model as its safety_checker submodule to check the safety of the pipeline's inputs and outputs, and you configure it alongside the other submodules as the examples below show.

The guardrail comes from the cosmos-guardrail package, which RBLN Optimum does not install for you. Install it before you load a Cosmos pipeline.

pip install "cosmos-guardrail==0.3.1" "nltk==3.9.4"

Keep nltk at 3.9.4. With nltk 3.10 the guardrail fails to load from a freshly downloaded Hugging Face cache: recent huggingface_hub releases store the guardrail's data files in the hub-wide blobs/ directory, and nltk 3.10 refuses data files whose real location is outside its allowed paths (Resource 'wordnet' not found, or PermissionError from 3.10.3 on).

NVIDIA Open Model License

In accordance with the NVIDIA Open Model License, the safety_checker submodule runs the guardrail over the input prompt and the generated video automatically inside the pipeline. Do not remove or disable it. Under the license, bypassing the guardrail or reducing its efficacy terminates your rights automatically, unless you substitute a substantially similar guardrail appropriate for your use case.

Usage Examples¶

Text-to-Video¶

import torch
from diffusers.utils import export_to_video
from optimum.rbln import (
    RBLNCosmosTextToWorldPipeline,
    RBLNCosmosTextToWorldPipelineConfig,
)

# Create a configuration object (optional, can use defaults)
config = RBLNCosmosTextToWorldPipelineConfig(
    height=704,
    width=1280,
    transformer={
        "num_devices": 4,
        "device": [0, 1, 2, 3],
    },
    text_encoder={
        "device": 4,
    },
    vae={
        "device": 4,
    },
    safety_checker={
        "qwen3guard": {"device": 3},
        "face_blur_filter": {"device": 3},
    },
)

# Load and compile the Cosmos model for RBLN NPU
pipe = RBLNCosmosTextToWorldPipeline.from_pretrained(
    "nvidia/Cosmos-1.0-Diffusion-7B-Text2World",
    torch_dtype=torch.bfloat16,
    export=True,
    rbln_config=config,
)

# Generate a Video
prompt = "A sleek, humanoid robot stands in a vast warehouse filled with neatly stacked cardboard boxes on industrial shelves. The robot's metallic body gleams under the bright, even lighting, highlighting its futuristic design and intricate joints. A glowing blue light emanates from its chest, adding a touch of advanced technology. The background is dominated by rows of boxes, suggesting a highly organized storage system. The floor is lined with wooden pallets, enhancing the industrial setting. The camera remains static, capturing the robot's poised stance amidst the orderly environment, with a shallow depth of field that keeps the focus on the robot while subtly blurring the background for a cinematic effect."
output = pipe(prompt=prompt).frames[0]

# Save the generated video
export_to_video(output, "output.mp4", fps=30)
print("Video saved as output.mp4")

Video-to-Video¶

import torch
from diffusers.utils import export_to_video, load_video
from optimum.rbln import (
    RBLNCosmosVideoToWorldPipeline,
    RBLNCosmosVideoToWorldPipelineConfig,
)

# Create a configuration object (optional, can use defaults)
config = RBLNCosmosVideoToWorldPipelineConfig(
    height=704,
    width=1280,
    transformer={
        "num_devices": 4,
        "device": [0, 1, 2, 3],
    },
    text_encoder={
        "device": 4,
    },
    vae={
        "device_map": {"encoder": 4, "decoder": 5},
    },
    safety_checker={
        "qwen3guard": {"device": 5},
        "face_blur_filter": {"device": 5},
    },
)

# Load and compile the Cosmos model for RBLN NPU
pipe = RBLNCosmosVideoToWorldPipeline.from_pretrained(
    "nvidia/Cosmos-1.0-Diffusion-7B-Video2World",
    torch_dtype=torch.bfloat16,
    export=True,
    rbln_config=config,
)

# Generate a Video
video = load_video(
    "https://github.com/nvidia-cosmos/cosmos-predict1/raw/refs/heads/main/assets/diffusion/video2world_input1.mp4"
)
prompt = "A dynamic and visually captivating video showcases a sleek, dark-colored SUV driving along a narrow dirt road that runs parallel to a vast, expansive ocean. The setting is a rugged coastal landscape, with the road cutting through dry, golden-brown grass that stretches across rolling hills. The ocean, a deep blue, extends to the horizon, providing a stunning backdrop to the scene. The SUV moves swiftly along the road, kicking up a trail of dust that lingers in the air behind it, emphasizing the speed and power of the vehicle. The camera maintains a steady tracking shot, following the SUV from a slightly elevated angle, which allows for a clear view of both the vehicle and the surrounding scenery. The lighting is natural, suggesting a time of day when the sun is high, casting minimal shadows and highlighting the textures of the grass and the glint of the ocean. The video captures the essence of freedom and adventure, with the SUV navigating the isolated road with ease, suggesting a journey or exploration theme. The consistent motion of the vehicle and the dust trail create a sense of continuity and fluidity throughout the video, making it engaging and immersive."
output = pipe(video=video, prompt=prompt).frames[0]

# Save the generated video
export_to_video(output, "output.mp4", fps=30)
print("Video saved as output.mp4")

API Reference¶

`RBLNCosmosTextToWorldPipeline` ¶

Bases: RBLNDiffusionMixin, CosmosTextToWorldPipeline

RBLN-accelerated implementation of Cosmos Text to World pipeline for text-to-video generation.

This pipeline compiles Cosmos Text to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

`from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.

This method has two distinct operating modes

When export=True: Takes a PyTorch-based diffusion model, compiles it for RBLN NPUs, and loads the compiled model
When export=False: Loads an already compiled RBLN model from model_id without recompilation

It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The model ID or path to the pretrained model to load. Can be either: A model ID from the HuggingFace Hub A local path to a saved model directory	required
`export`	`bool`	If True, takes a PyTorch model from `model_id` and compiles it for RBLN NPU execution. If False, loads an already compiled RBLN model from `model_id` without recompilation.	`False`
`safety_checker`	`RBLNCosmosSafetyChecker \| None`	Optional custom safety checker to use instead of the default one. Only used when `export=True`.	`None`
`rbln_config`	`dict[str, Any] \| None`	Configuration options for RBLN compilation. Can include settings for specific submodules such as `text_encoder`, `unet`, and `vae`. Configuration can be tailored to the specific pipeline being compiled.	`None`
`kwargs`	`Any`	Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used.	`{}`

`RBLNCosmosVideoToWorldPipeline` ¶

Bases: RBLNDiffusionMixin, CosmosVideoToWorldPipeline

RBLN-accelerated implementation of Cosmos Video to World pipeline for video-to-video generation.

This pipeline compiles Cosmos Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

`from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs)` `classmethod` ¶

Load a pretrained diffusion pipeline from a model checkpoint, with optional compilation for RBLN NPUs.

This method has two distinct operating modes

When export=True: Takes a PyTorch-based diffusion model, compiles it for RBLN NPUs, and loads the compiled model
When export=False: Loads an already compiled RBLN model from model_id without recompilation

It supports various diffusion pipelines including Stable Diffusion, Kandinsky, ControlNet, and other diffusers-based models.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The model ID or path to the pretrained model to load. Can be either: A model ID from the HuggingFace Hub A local path to a saved model directory	required
`export`	`bool`	If True, takes a PyTorch model from `model_id` and compiles it for RBLN NPU execution. If False, loads an already compiled RBLN model from `model_id` without recompilation.	`False`
`safety_checker`	`RBLNCosmosSafetyChecker \| None`	Optional custom safety checker to use instead of the default one. Only used when `export=True`.	`None`
`rbln_config`	`dict[str, Any] \| None`	Configuration options for RBLN compilation. Can include settings for specific submodules such as `text_encoder`, `unet`, and `vae`. Configuration can be tailored to the specific pipeline being compiled.	`None`
`kwargs`	`Any`	Additional arguments to pass to the underlying diffusion pipeline constructor or the RBLN compilation process. These may include parameters specific to individual submodules or the particular diffusion pipeline being used.	`{}`

`RBLNCosmos2TextToImagePipeline` ¶

Bases: RBLNDiffusionMixin, Cosmos2TextToImagePipeline

RBLN-accelerated implementation of Cosmos Predict2 Text to Image pipeline for text-to-image generation.

This pipeline compiles Cosmos Predict2 Text to Image models to run efficiently on RBLN NPUs, enabling high-performance inference for generating images that follow physical laws with enhanced visual quality.

`RBLNCosmos2VideoToWorldPipeline` ¶

Bases: RBLNDiffusionMixin, Cosmos2VideoToWorldPipeline

RBLN-accelerated implementation of Cosmos Predict2 Video to World pipeline for video-to-video generation.

This pipeline compiles Cosmos Predict2 Video to World models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

`RBLNCosmos2_5_PredictBasePipeline` ¶

Bases: RBLNDiffusionMixin, Cosmos2_5_PredictBasePipeline

RBLN-accelerated implementation of Cosmos-Predict2.5 pipeline.

This pipeline compiles Cosmos-Predict2.5 models to run efficiently on RBLN NPUs, enabling high-performance inference for generating videos that follow physical laws with enhanced visual quality.

One pipeline serves the three conditioning modes: Text2World (no visual input), Image2World (image=...) and Video2World (video=...).

`RBLNCosmos2_5_TransferPipeline` ¶

Bases: RBLNDiffusionMixin, Cosmos2_5_TransferPipeline

RBLN-accelerated implementation of the Cosmos-Transfer2.5 pipeline.

Transfer2.5 generates videos that follow a control signal (edge/depth/seg/blur maps): a 4-block ControlNet computes per-step residuals that are injected into the Predict2.5 base transformer. Long videos are generated auto-regressively in num_frames_per_chunk windows, so the compiled shapes are per chunk and the chunk loop runs on the host.

The official Hub layout keeps the ControlNet on its own revision, so a typical export is:

controlnet = CosmosControlNetModel.from_pretrained(
    "nvidia/Cosmos-Transfer2.5-2B", revision="diffusers/controlnet/general/edge"
)
pipe = RBLNCosmos2_5_TransferPipeline.from_pretrained(
    "nvidia/Cosmos-Transfer2.5-2B", revision="diffusers/general",
    controlnet=controlnet, export=True, ...
)

`RBLNCosmosPipelineBaseConfig` ¶

Bases: RBLNModelConfig

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`RBLNT5EncoderModelConfig \| None`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`RBLNCosmosTransformer3DModelConfig \| None`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`RBLNAutoencoderKLCosmosConfig \| None`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.	`None`
`safety_checker`	`RBLNCosmosSafetyCheckerConfig \| None`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`int \| None`	Batch size for inference, applied to all submodules.	`None`
`height`	`int \| None`	Height of the generated videos.	`None`
`width`	`int \| None`	Width of the generated videos.	`None`
`num_frames`	`int \| None`	The number of frames in the generated video.	`None`
`max_seq_len`	`int \| None`	Maximum sequence length supported by the model.	`None`
`kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`RBLNCosmosTextToWorldPipelineConfig` ¶

Bases: RBLNCosmosPipelineBaseConfig

Config for Cosmos-Predict1 Text2World Pipeline

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`RBLNT5EncoderModelConfig \| None`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`RBLNCosmosTransformer3DModelConfig \| None`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`RBLNAutoencoderKLCosmosConfig \| None`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.	`None`
`safety_checker`	`RBLNCosmosSafetyCheckerConfig \| None`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`int \| None`	Batch size for inference, applied to all submodules.	`None`
`height`	`int \| None`	Height of the generated videos.	`None`
`width`	`int \| None`	Width of the generated videos.	`None`
`num_frames`	`int \| None`	The number of frames in the generated video.	`None`
`max_seq_len`	`int \| None`	Maximum sequence length supported by the model.	`None`
`kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNCosmosVideoToWorldPipelineConfig` ¶

Bases: RBLNCosmosPipelineBaseConfig

Config for Cosmos-Predict1 Video2World Pipeline

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`RBLNT5EncoderModelConfig \| None`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`RBLNCosmosTransformer3DModelConfig \| None`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`RBLNAutoencoderKLCosmosConfig \| None`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLCosmosConfig if not provided.	`None`
`safety_checker`	`RBLNCosmosSafetyCheckerConfig \| None`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`int \| None`	Batch size for inference, applied to all submodules.	`None`
`height`	`int \| None`	Height of the generated videos.	`None`
`width`	`int \| None`	Width of the generated videos.	`None`
`num_frames`	`int \| None`	The number of frames in the generated video.	`None`
`max_seq_len`	`int \| None`	Maximum sequence length supported by the model.	`None`
`kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNCosmos2PipelineBaseConfig` ¶

Bases: RBLNModelConfig

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`Optional[RBLNT5EncoderModelConfig]`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`Optional[RBLNCosmosTransformer3DModelConfig]`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`Optional[RBLNAutoencoderKLWanConfig]`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.	`None`
`safety_checker`	`Optional[RBLNCosmosSafetyCheckerConfig]`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`Optional[int]`	Batch size for inference, applied to all submodules.	`None`
`height`	`Optional[int]`	Height of the generated videos.	`None`
`width`	`Optional[int]`	Width of the generated videos.	`None`
`num_frames`	`Optional[int]`	The number of frames in the generated video.	`None`
`max_seq_len`	`Optional[int]`	Maximum sequence length supported by the model.	`None`
`**kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`RBLNCosmos2TextToImagePipelineConfig` ¶

Bases: RBLNCosmos2PipelineBaseConfig

Config for Cosmos-Predict2 Text2Image Pipeline

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`Optional[RBLNT5EncoderModelConfig]`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`Optional[RBLNCosmosTransformer3DModelConfig]`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`Optional[RBLNAutoencoderKLWanConfig]`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.	`None`
`safety_checker`	`Optional[RBLNCosmosSafetyCheckerConfig]`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`Optional[int]`	Batch size for inference, applied to all submodules.	`None`
`height`	`Optional[int]`	Height of the generated videos.	`None`
`width`	`Optional[int]`	Width of the generated videos.	`None`
`num_frames`	`Optional[int]`	The number of frames in the generated video.	`None`
`max_seq_len`	`Optional[int]`	Maximum sequence length supported by the model.	`None`
`**kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNCosmos2VideoToWorldPipelineConfig` ¶

Bases: RBLNCosmos2PipelineBaseConfig

Config for Cosmos-Predict2 Video2World Pipeline

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`Optional[RBLNT5EncoderModelConfig]`	Configuration for the text encoder component. Initialized as RBLNT5EncoderModelConfig if not provided.	`None`
`transformer`	`Optional[RBLNCosmosTransformer3DModelConfig]`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`Optional[RBLNAutoencoderKLWanConfig]`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.	`None`
`safety_checker`	`Optional[RBLNCosmosSafetyCheckerConfig]`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`Optional[int]`	Batch size for inference, applied to all submodules.	`None`
`height`	`Optional[int]`	Height of the generated videos.	`None`
`width`	`Optional[int]`	Width of the generated videos.	`None`
`num_frames`	`Optional[int]`	The number of frames in the generated video.	`None`
`max_seq_len`	`Optional[int]`	Maximum sequence length supported by the model.	`None`
`**kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`RBLNCosmos2_5_PredictBasePipelineConfig` ¶

Bases: RBLNModelConfig

Config for Cosmos-Predict2.5 Pipeline

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`text_encoder`	`Optional[RBLNQwen2_5_VLForConditionalGenerationConfig]`	Configuration for the text encoder component. Initialized as RBLNQwen2_5_VLForConditionalGenerationConfig if not provided.	`None`
`transformer`	`Optional[RBLNCosmosTransformer3DModelConfig]`	Configuration for the Transformer model component. Initialized as RBLNCosmosTransformer3DModelConfig if not provided.	`None`
`vae`	`Optional[RBLNAutoencoderKLWanConfig]`	Configuration for the VAE model component. Initialized as RBLNAutoencoderKLWanConfig if not provided.	`None`
`safety_checker`	`Optional[RBLNCosmosSafetyCheckerConfig]`	Configuration for the safety checker component. Initialized as RBLNCosmosSafetyCheckerConfig if not provided.	`None`
`batch_size`	`Optional[int]`	Batch size for inference, applied to all submodules.	`None`
`height`	`Optional[int]`	Height of the generated videos.	`None`
`width`	`Optional[int]`	Width of the generated videos.	`None`
`num_frames`	`Optional[int]`	The number of frames in the generated video.	`None`
`max_seq_len`	`Optional[int]`	Maximum sequence length supported by the model.	`None`
`**kwargs`	`Any`	Additional arguments passed to the parent RBLNModelConfig.	`{}`

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`RBLNCosmos2_5_TransferPipelineConfig` ¶

Bases: RBLNCosmos2_5_PredictBasePipelineConfig

Config for Cosmos-Transfer2.5 Pipeline (Predict2.5 base + ControlNet).

`init(controlnet=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

Parameters:

Name	Type	Description	Default
`controlnet`	`Optional[RBLNCosmosControlNetModelConfig]`	Configuration for the ControlNet component. Initialized as RBLNCosmosControlNetModelConfig if not provided; its latent geometry is copied from the transformer submodule at compile time.	`None`
`batch_size`	`Optional[int]`	Batch size for inference, applied to all submodules.	`None`
`height`	`Optional[int]`	Height of the generated videos.	`None`
`width`	`Optional[int]`	Width of the generated videos.	`None`
`num_frames`	`Optional[int]`	The number of frames per generated chunk.	`None`
`max_seq_len`	`Optional[int]`	Maximum sequence length supported by the model.	`None`
`**kwargs`	`Any`	Additional arguments passed to the parent config (text_encoder/transformer/vae/safety_checker).	`{}`

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

Build the config of a submodule from the parent's runtime options and kwargs.

Returns the submodule config instance when its class is known (an instance was passed, or cls_name was given), otherwise the merged kwargs dict for the model class to resolve later.

`get_load_overrides()` ¶

Extract the options that must cross the load boundary, for passing to from_pretrained.

Returns the explicitly-set runtime options (device, device_map, create_runtimes, activate_profiler, timeout) and the subclass-specific non-save attributes (load-behavior flags such as _load_visual_runtime, which are never serialized so they can only travel with the caller), recursively including submodules. Compile-time attributes are not included: when loading a compiled model they come from the artifact's rbln_config.json, and this dict is merged on top of it.

Returns:

Type	Description
`dict[str, Any]`	Dictionary of load overrides, keyed like the `rbln_config` dict accepted by `from_pretrained`.

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

Load a RBLNModelConfig from a path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the RBLNModelConfig file or directory containing the config file.	required
`rbln_config`	`dict[str, Any] \| None`	Additional configuration to override.	`None`
`return_unused_kwargs`	`bool`	Whether to return unused kwargs.	`False`
`kwargs`	`Any`	Additional keyword arguments to override configuration values. Keys starting with 'rbln_' will have the prefix removed and be used to update the configuration.	`{}`

Returns:

Name	Type	Description
`RBLNModelConfig`	`Self \| tuple[Self, dict[str, Any]]`	The loaded configuration instance.

Note

This method loads the configuration from the specified path and applies any provided overrides. If the loaded configuration class doesn't match the expected class, a warning will be logged.

Examples:

config = RBLNResNetForImageClassificationConfig.from_pretrained("/path/to/model")

Cosmos¶

Supported Pipelines¶

Important: Cosmos Safety Guardrails¶

Usage Examples¶

Text-to-Video¶

Video-to-Video¶

API Reference¶

RBLNCosmosTextToWorldPipeline ¶

from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod ¶

RBLNCosmosVideoToWorldPipeline ¶

from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs) classmethod ¶

RBLNCosmos2TextToImagePipeline ¶

RBLNCosmos2VideoToWorldPipeline ¶

RBLNCosmos2_5_PredictBasePipeline ¶

RBLNCosmos2_5_TransferPipeline ¶

RBLNCosmosPipelineBaseConfig ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

get_load_overrides() ¶

RBLNCosmosTextToWorldPipelineConfig ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

get_load_overrides() ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNCosmosVideoToWorldPipelineConfig ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

get_load_overrides() ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNCosmos2PipelineBaseConfig ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

get_load_overrides() ¶

RBLNCosmos2TextToImagePipelineConfig ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

get_load_overrides() ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNCosmos2VideoToWorldPipelineConfig ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

get_load_overrides() ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

RBLNCosmos2_5_PredictBasePipelineConfig ¶

__init__(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

get_load_overrides() ¶

RBLNCosmos2_5_TransferPipelineConfig ¶

__init__(controlnet=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs) ¶

initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs) ¶

get_load_overrides() ¶

from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs) classmethod ¶

`RBLNCosmosTextToWorldPipeline` ¶

`from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs)` `classmethod` ¶

`RBLNCosmosVideoToWorldPipeline` ¶

`from_pretrained(model_id, *, export=False, safety_checker=None, rbln_config=None, **kwargs)` `classmethod` ¶

`RBLNCosmos2TextToImagePipeline` ¶

`RBLNCosmos2VideoToWorldPipeline` ¶

`RBLNCosmos2_5_PredictBasePipeline` ¶

`RBLNCosmos2_5_TransferPipeline` ¶

`RBLNCosmosPipelineBaseConfig` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`get_load_overrides()` ¶

`RBLNCosmosTextToWorldPipelineConfig` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`get_load_overrides()` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNCosmosVideoToWorldPipelineConfig` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`get_load_overrides()` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNCosmos2PipelineBaseConfig` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`get_load_overrides()` ¶

`RBLNCosmos2TextToImagePipelineConfig` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`get_load_overrides()` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNCosmos2VideoToWorldPipelineConfig` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`get_load_overrides()` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`RBLNCosmos2_5_PredictBasePipelineConfig` ¶

`init(text_encoder=None, transformer=None, vae=None, safety_checker=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`get_load_overrides()` ¶

`RBLNCosmos2_5_TransferPipelineConfig` ¶

`init(controlnet=None, *, batch_size=None, height=None, width=None, num_frames=None, max_seq_len=None, **kwargs)` ¶

`initialize_submodule_config(submodule_config=None, force_kwargs=False, **kwargs)` ¶

`get_load_overrides()` ¶

`from_pretrained(path, rbln_config=None, return_unused_kwargs=False, **kwargs)` `classmethod` ¶