Skip to content

Model Parallelism

Overview

RBLN vLLM provides two approaches to model parallelism.

vLLM-level parallelism
The framework distributes the model across worker processes using standard vLLM parallelism arguments.
Rebellions Scalable Design (RSD)
The RBLN Compiler partitions the model across devices (User-level RSD) or chiplets within a device (Internal RSD).

See Parallelism in the Feature Matrix for feature availability.

vLLM-level parallelism

Model implementation support

vLLM-level parallelism is supported only under --model-impl vllm.

vLLM distributes model execution across worker processes and coordinates communication through rbln-ccl.

A rank identifies a worker process participating in parallel execution.

Type Argument Behavior
Data parallelism (DP) --data-parallel-size Replicates the model to serve requests independently
Tensor parallelism (TP) --tensor-parallel-size Distributes computation within layers across ranks
Pipeline parallelism (PP) --pipeline-parallel-size Partitions layers into sequential stages
Expert parallelism (EP) --enable-expert-parallel Distributes a mixture-of-experts layer's experts across ranks

Tensor parallelism

TP distributes computation within a layer across ranks. The ranks exchange intermediate results through rbln-ccl.

1
2
3
vllm serve <MODEL_ID> \
  --model-impl vllm \
  --tensor-parallel-size 4

Pipeline parallelism

PP partitions model layers into sequential stages, one per rank.

Each pipeline stage supports up to max_num_seqs // pipeline_parallel_size sequences. Set max_num_seqs to a multiple of pipeline_parallel_size, with a minimum of pipeline_parallel_size, to avoid unused capacity. The engine logs a warning if max_num_seqs is not a multiple of pipeline_parallel_size.

1
2
3
4
vllm serve <MODEL_ID> \
  --model-impl vllm \
  --pipeline-parallel-size 2 \
  --max-num-seqs 8

Data parallelism

DP replicates the model across ranks. Each replica schedules requests independently.

Under DP, max_num_batched_tokens must be divisible by max_num_seqs. Keep --rbln-use-moe-tokens-mask enabled to track padded tokens. It is enabled by default.

1
2
3
4
5
vllm serve <MODEL_ID> \
  --model-impl vllm \
  --data-parallel-size 4 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 1024

Expert parallelism

EP distributes MoE experts across DP ranks.

As with DP, keep --rbln-use-moe-tokens-mask enabled to track padding.

1
2
3
4
5
6
vllm serve <MOE_MODEL_ID> \
  --model-impl vllm \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --max-num-seqs 8 \
  --max-num-batched-tokens 1024

Rebellions Scalable Design (RSD)

RSD is a compiler-driven approach to model partitioning. The RBLN Compiler partitions the computation graph and embeds the required communication operations in the compiled binaries.

The partitioning scope depends on the hardware and model implementation. Under transformers on ATOM™, User-level RSD partitions the model across a user-specified number of chips, with each chip counted as one NPU device. On Rebel100™, both model implementations use Internal RSD to automatically partition the model across the four interconnected chiplets within one NPU card. See the RSD overview for background.

Legend

  • ✅: Supported
  • ❌: Not currently supported
  • —: Not applicable
RSD type Item ATOM™ Rebel100™
Model implementation transformers transformers / vllm
User-level RSD Support ✅ ❌
Partitioning scope Across chips —
Internal RSD Support — ✅
Partitioning scope — Across chiplets within a card

User-level RSD

ATOM™ + transformers

--rbln-num-devices-per-local-rank sets the number of ATOM™ chips used by one engine process.

1
2
3
vllm serve <MODEL_ID> \
  --model-impl transformers \
  --rbln-num-devices-per-local-rank 4

This option counts chips, not cards. Each RBLN-CA22 card has one chip, while each RBLN-CA25 card has four.

Internal RSD

Rebel100™ + transformers / vllm

Under both model implementations, the compiler automatically partitions the model across the four chiplets within each Rebel100™ card. No user configuration is required for chiplet partitioning.