Feature Matrix¶

Overview¶

RBLN vLLM runs a model through one of two model implementations, and that choice decides which features are available. The implementation names are values of the --model-impl option.

vllm model implementation: The model code comes from vLLM itself, compiled through the torch.compile backend. Selected with --model-impl vllm. Runs on Rebel100™ only.
transformers (optimum) model implementation: The model code comes from optimum-rbln, which provides a model class per supported architecture (e.g. RBLNLlamaForCausalLM).

The external API is the same either way: model takes a model ID or a directory, and a model with no compiled artifact for the current configuration is compiled on the spot. The implementations use different model code and compilation workflows.

The tables below list the features RBLN vLLM adds or constrains on RBLN NPUs, grouped into execution and scheduling, caching and KV transfer, disaggregated serving, parallelism, and extensions.

Note

The table shows feature support for this release. Upstream vLLM features not listed here behave as documented by vLLM.

Legend

Symbol	Meaning
✅	Supported
🚧	Available, still under active development
❌	Not currently supported

Execution and scheduling¶

Feature	`vllm`	`transformers` (`optimum`)	Details
Naive Attention / Flash Attention	✅	✅	See Attention Modes for kernel selection and device constraints.
Chunked prefill	✅	✅	Always enabled. It cannot be turned off.
Dynamic Batch Sizes	✅	✅	Uses multiple decode batch sizes to reduce padding. See Dynamic Batch Sizes for configuration.
RBLN Sampler	✅	✅	Runs greedy and top-p/top-k sampling on the NPU. See RBLN Sampler for configuration.
Structured output	✅	✅	Supports JSON schema, regex, grammar, choice, and structural tags.
Asynchronous scheduling	✅	❌	Under `transformers`, the engine disables it and runs synchronously.
Speculative decoding	✅	❌	Proposes multiple tokens and verifies them in a single target-model pass. See Speculative Decoding for supported methods and configuration.
LoRA adapters	❌	✅	Loads LoRA adapters for inference.

Caching and KV transfer¶

Feature	`vllm`	`transformers` (`optimum`)	Details
Prefix caching	✅	✅	Supports text and multimodal generation. See Prefix Caching for model restrictions.
Sub-block prefix caching	✅	✅	Reuses prefixes at sub-block granularity. See Sub-block prefix caching for configuration constraints.
Sub-block KV cache events	✅	❌	Reports cached sub-blocks for routing. See Prefix cache-aware routing for configuration.
KV cache offloading	✅	❌	Offloads KV cache to CPU memory or NVMe through LMCache. See KV Cache Offloading for configuration.

Disaggregated serving¶

Feature	`vllm`	`transformers` (`optimum`)	Details
Disaggregated encoder	❌	🚧	Runs a multimodal model's encoder on separate instances. See Disaggregated Encoder for configuration.
Disaggregated prefill and decode	✅	❌	Runs prefill and decode on separate instances. See Disaggregated Prefill / Decode for configuration.

Parallelism¶

Feature	`vllm`	`transformers` (`optimum`)	Details
Tensor parallelism	✅	❌	See Tensor parallelism for configuration.
Pipeline parallelism	✅	❌	Splits model layers into sequential stages. See Pipeline parallelism for configuration.
Data parallelism	✅	❌	Replicates the model to serve requests independently. See Data parallelism for configuration.

Rebellions Scalable Design (RSD)

In addition to the vLLM-level parallelism methods above, RBLN supports Rebellions Scalable Design (RSD), a parallelism feature built into the RBLN Compiler. On ATOM™, users can configure User-level RSD across chips. On Rebel100™, the compiler automatically handles Internal RSD across chiplets. See Rebellions Scalable Design (RSD) for details.

Extensions and tooling¶

Feature	`vllm`	`transformers` (`optimum`)	Details
RBLN Profiler	✅	✅	See Profiling Guide for configuration and trace visualization.
OpenAI-compatible server	✅	✅	Serves requests through an OpenAI-compatible API.