Skip to content

Qwen3.8-27B on RBLN NPU

Install the RBLN Driver on the host. An RBLN Portal account is required to install the RBLN Compiler (rebel-compiler). See the installation matrix for compatible package versions.

1
2
3
$ pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.12.0

1
2
3
4
$ pip install \
  --extra-index-url https://wheels.vllm.ai/0.30.0/cpu \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  vllm-rbln==0.12.0
Guide — overview and configuration

Overview

Qwen/Qwen3.8-27B is a natively multimodal (vision-language) model. It shares the Qwen3.5 architecture — config model_type: qwen3_5, a hybrid decoder with linear_attention layers (GatedDeltaNet: conv_state + recurrent_state) interleaved with gated full_attention (paged KV) layers — so it loads through the same RBLN class as the Qwen3.5 line. On RBLN NPUs, the RBLN vLLM plugin compiles the language model and vision encoder on first run, then serves them over vLLM's OpenAI-compatible API. Include images using image_url content items in chat completion requests.

Configuration

--rbln-num-devices-per-local-rank sets how many NPU devices one engine process uses, and the RBLN Compiler partitions the model for those devices with Rebellions Scalable Design (RSD). See Model Parallelism for how RSD applies on each NPU. The vision encoder is configured through --additional-config (optimum_overrides.visual). Changing these settings recompiles the model on the next start. Compile Cache lists every setting that does.

Qwen3.8 ships one dense vision-language size on HuggingFace, 27B, as BF16 and FP8 checkpoints; this recipe targets the BF16 checkpoint (the FP8 checkpoint, the Flash-Next variant, and the 2.4T-A95B text MoE variant are out of scope for this recipe). See the Configuration Guide for all engine arguments, and Fine-Grained Device Assignment for placing the vision encoder and the language model on different devices.

Notes and limitations

  • The sequence length, NPU count, and vision settings are fixed at compilation; changing any of these settings triggers recompilation at the next startup.
  • --additional-config carries the vision encoder settings (optimum_overrides.visual). For multi-image or video workloads, adjust the encoder's max_seq_len, and see Fine-Grained Device Assignment to place the encoder on its own devices.
  • Qwen3.8 loads through the same qwen3_5 registration as the Qwen3.5 line (RBLNQwen3_5ForConditionalGeneration). Parameters follow the reference compile configuration for this model line.