Skip to content

Qwen3.6-27B on RBLN NPU

Install the RBLN Driver on the host. An RBLN Portal account is required to install the RBLN Compiler (rebel-compiler). See the installation matrix for compatible package versions.

1
2
3
$ pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.12.0

1
2
3
4
$ pip install \
  --extra-index-url https://wheels.vllm.ai/0.30.0/cpu \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  vllm-rbln==0.12.0
Guide — overview and configuration

Overview

Qwen/Qwen3.6-27B is a natively multimodal (vision-language) model. It shares the Qwen3.5 architecture — config model_type: qwen3_5, a hybrid decoder with linear_attention layers (GatedDeltaNet: conv_state + recurrent_state) interleaved with gated full_attention (paged KV) layers — so it loads through the same RBLN class as the Qwen3.5 line. On RBLN NPUs, the RBLN vLLM plugin compiles the language model and vision encoder on first run, then serves them over vLLM's OpenAI-compatible API. Include images using image_url content items in chat completion requests.

Configuration

--rbln-num-devices-per-local-rank sets how many NPU devices one engine process uses, and the RBLN Compiler partitions the model for those devices with Rebellions Scalable Design (RSD). See Model Parallelism for how RSD applies on each NPU. The vision encoder is configured through --additional-config (optimum_overrides.visual). Changing these settings recompiles the model on the next start. Compile Cache lists every setting that does.

Qwen3.6 currently ships only the 27B dense variant on HuggingFace (no smaller dense sizes; the 35B-A3B MoE variant is out of scope for this recipe). See the Configuration Guide for all engine arguments, and Fine-Grained Device Assignment for placing the vision encoder and the language model on different devices.

Notes and limitations

  • The sequence length, NPU count, and vision settings are fixed at compilation; changing any of these settings triggers recompilation at the next startup.
  • --additional-config carries the vision encoder settings (optimum_overrides.visual). For multi-image or video workloads, adjust the encoder's max_seq_len, and see Fine-Grained Device Assignment to place the encoder on its own devices.
  • Attention is fixed to flash_attn — the hybrid backbone's multi-window prefill is mis-lowered by the eager op; flash_attn is correct (see vllm-rbln#795). Do not override.
  • Unverified configuration. Parameters mirror the reference compile configuration for this model line; optimum-rbln#631 introduces RBLNQwen3_5ForConditionalGeneration and vllm-rbln#795 wires qwen3_5 into the transformers model implementation (Qwen3.6 uses the same qwen3_5 model registration).