Skip to content

MiniMax-M2.7 on RBLN NPU

Install the RBLN Driver on the host. An RBLN Portal account is required to install the RBLN Compiler (rebel-compiler). See the installation matrix for compatible package versions.

1
2
3
$ pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.12.0

The KV offload (LMCache) and prefill/decode-disaggregated commands need the kv_connectors extra, which installs lmcache-rbln and nixl-rbln. Without it those servers fail at startup.

1
2
3
4
5
$ pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  --extra-index-url https://wheels.vllm.ai/0.30.0/cpu \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  "vllm-rbln[kv_connectors]==0.12.0"

Suffix decoding (the Suffix option under Speculative Decoding) needs arctic-inference, which vllm-rbln does not install by default. Without it that server fails at startup with an ImportError.

1
$ pip install arctic-inference==0.1.1
Guide — overview and configuration

Overview

MiniMaxAI/MiniMax-M2.7 is a Mixture-of-Experts, decoder-only model. On RBLN-CR13 it is served (fp8) through the RBLN vLLM plugin, aggregated or with prefill and decode disaggregated.

Configuration

Verified on RBLN-CR13. Select a configuration in the recipe above; each server uses 4 devices.

  • Data + Expert Parallel: --data-parallel-size sets the CR13 devices that each hold a model replica, and --enable-expert-parallel shards the experts across them.
  • Pipeline Parallel: --pipeline-parallel-size splits the layers across the devices.
  • Prefill/Decode disaggregated: a prefill server and a decode server run on separate nodes behind the llm-d router, whose helm command follows the two servers in the recipe. For how to deploy the llm-d router and the serving stack on Kubernetes, see Deployment on Kubernetes in the Coding Assistant reference design.

Every command runs on the vllm model implementation, selected with --model-impl vllm.

See the Configuration Guide for all engine arguments.

Notes and limitations

  • Verified on RBLN-CR13; the exact serve flags may change.
  • The batch size, sequence length, and device count are fixed at compilation; changing any of these settings triggers recompilation at the next startup.