Validation and Limits¶

Overview¶

This page records how far the MiniMax-M2.7 reference design has been validated and how those results are reproduced. It holds two records:

Target Configuration validation. A four-node, 32-NPU stack was deployed with the versions in Software Stack and Versions. The validation workload ran in three P/D layouts (4P4D, 5P3D, 3P5D) across concurrency levels.
Feature and accuracy validation. Four serving configurations were deployed on four NPUs per instance. Deployment startup, response generation, determinism, cache reuse, and accuracy were checked at every combination of a configuration with features, and absolute accuracy was measured with gsm8k and mmlu_pro from lm-evaluation-harness. The versions are in Versions used.

What backs a Validated badge depends on the item. An item that depends on the node count or the P/D layout rests on the Target Configuration validation only. An item decided within a single instance can rest on either record.

Target Configuration validation¶

Validated (2026-10-13).

Environment¶

Item	Value
Hardware	4 RebelServer™ nodes, 32 RBLN-CR13 cards
Software	The versions in Software Stack and Versions, on RBLN Driver 3.5.1 and Kubernetes 1.36.4
GatewayClass provider	Istio 1.30.0
Workload	ISL 51,200 tokens, OSL 1,024 tokens, 80% of each prompt (40,960 tokens) a prefix shared by every request

Serving configuration exercised¶

Item	Value
P/D layouts	4P4D, 5P3D, 3P5D
Prefill	PP4, `--max-num-seqs 4`
Decode	DP4 with expert parallel. `--max-num-seqs` `1` at concurrency 1 to 16, then `2`, `4`, and `8` at concurrency 32, 64, and 128
KV cache	`--block-size 4096`, `--max-model-len 204800`, `--max-num-batched-tokens 512`, `--kv-cache-dtype fp8`
Prefix caching	Automatic Prefix Caching (APC) with sub-block prefix caching (`VLLM_RBLN_SUB_BLOCK_CACHE=1`, on by default)
Speculative decoding	EAGLE-3 (`num_speculative_tokens: 3`) in all three layouts. In 4P4D, also suffix decoding at concurrency 1 to 64 and n-gram (`num_speculative_tokens: 3`, `prompt_lookup_max: 5`, `prompt_lookup_min: 2`) at concurrency 1 to 32
KV transfer	NIXL only (`RblnNixlConnector`), without LMCache
Routing	Prefix-cache-aware routing for the agent workload, with a router configuration that differs from the canonical one on Serving Configuration › Router: `prefix-cache-scorer` 3.0, `kv-cache-utilization-scorer` 2.0, `queue-scorer` 2.0, and a manual `lruCapacityPerServer` of 8400. Round-robin across instances for the P/D layout comparison, so that each layout received an even load
Concurrency	1, 2, 4, 8, 16, 32, 64, 128, with 10 × concurrency requests per point

A second run on the same three layouts turned off prefix caching and speculative decoding and covered concurrency 1 to 64. It is the basis for the layout guidance without prefix caching in Hardware › P:D ratio.

The 4P4D layout also served an agent workload: mini-swe-agent worked on the 64 SWE-bench Lite problems with the longest inputs, with 64 agents running at once and prefix-cache-aware routing. Each problem takes many turns of tool call round trips, so this run exercises the tool-call and reasoning arguments in Serving Configuration.

Checks¶

Every point passed the checks below, in all three layouts.

Check	Pass condition
Deployment startup	Every prefill and decode instance becomes Ready
Request completion	Every request of the point completes
Prefix reuse	The prefix-cache hit rate reaches the bound the workload allows. With EAGLE-3 the bound is one block below the shared prefix, because the draft head discards the last cached block
Scheduling	No request is preempted
Load spread	With round-robin routing, the request counts of the instances differ by at most one
Speculative decoding	The EAGLE-3 acceptance rate stays within one band (46% to 50%) across layouts and concurrency

The Unverified parts of the canonical configuration that this run did not exercise are listed in Coding Assistant on Rebel100™ › Validation scope and limits.

Feature and accuracy validation¶

The results below were obtained with the versions in the table below, on four NPUs per instance. pdd-pp4-dp4ep-throughput uses the same parallelism as the Target Configuration.

Versions used¶

The feature and accuracy validation ran on the combination below. Every number in this section is based on this combination.

Component	Version
rbln-serve image	0.12.0-ubi10.2-py3.12
RBLN Driver	3.5.1
RBLN Compiler	0.12.0
torch	2.11.0+cpu
torch-rbln	0.12.0
optimum-rbln	0.12.0
lmcache	0.5.4
lmcache-rbln	0.12.0
nixl	1.1.0+cpu
nixl-rbln	0.12.0
vllm	0.30.0+cpu
vllm-rbln	0.12.0

Serving configurations¶

Name	Topology	Prefill	Decode	NPU
agg-dp4ep-latency	AGG	-	DP4 + EP, `--max-num-seqs 1`	4
agg-pp4-throughput	AGG	-	PP4, EP off, `--max-num-seqs 4`	4
pdd-dp4ep-latency	Prefill/decode disaggregated	DP4 + EP, `--max-num-seqs 1`	DP4 + EP, `--max-num-seqs 1`	4 + 4
pdd-pp4-dp4ep-throughput	Prefill/decode disaggregated	PP4, EP off, `--max-num-seqs 4`	DP4 + EP, `--max-num-seqs 4`	4 + 4

--kv-cache-dtype=fp8, --max-model-len=204800, --block-size=4096

Features¶

The axes are described in Prefix Caching, KV Cache Offloading, and Speculative Decoding.

Axis	Mode	Description
`prefix_caching`	`off`	APC off
`prefix_caching`	`on`	APC at `--block-size` granularity
`prefix_caching`	`on_sub_block`	Prefix caching at prefill chunk (`--max-num-batched-tokens 512`) sub-block granularity
`lmcache`	`off`	KV offload off
`lmcache`	`cpu`	Host memory tier
`lmcache`	`rds`	Rebellions Direct Storage (RDS): direct transfer from NPU vmem to NVMe
`spec_decode`	`off`	Speculative decoding off
`spec_decode`	`ngram`	Prompt-lookup drafting
`spec_decode`	`suffix`	Suffix tree drafting over the request and the cache
`spec_decode`	`eagle3`	Separate draft head
`pd_chunk`	`off`	P/D KV transfer sends the last block in one transfer
`pd_chunk`	`on`	`chunk_mode` sends the last block in chunks (P/D split configurations only)
`sampler`	`greedy`	temperature 0.0
`sampler`	`random`	temperature 1.0, top_p 0.95, top_k 40
`sampler`	`penalties`	Presence, frequency, and repetition penalties applied
structured output	off	No grammar
structured output	JSON schema	Schema attached to the request
structured output	regex, choice, grammar	Regular expression, choice list, or grammar attached to the request
tool calling	off	No tools
tool calling	named	The request names the tool to call
tool calling	auto	The model decides whether to call a tool
tool calling	multi_turn	Tool result fed back into the conversation for a final response

Accuracy¶

Task	Rows	Score	Standard error
gsm8k	1,319 (full)	0.9530 (1,257/1,319)	0.0058
mmlu_pro	280 (14 subtasks × 20)	0.7429 (208/280)	0.026

Validation items¶

The items below were checked at every coordinate formed by combining the serving configurations and features above.

Check	Pass condition
Deployment startup	The rollout completes and `/health` responds
Response generation	Tokens are generated for five prompts that differ in length and character
Determinism	The same request sent twice returns matching token ids
Cache reuse	The reuse counter of an enabled cache axis is non-zero
Absolute accuracy	The coordinate with every axis off scores 0.9 or higher on GSM8K 5-shot
Accuracy after a feature is applied	Within 0.1 of the score of the all-axes-off coordinate on the same serving configuration
Grammar conformance	The response to a request carrying a JSON schema, regular expression, choice list, or grammar satisfies that constraint
Tool calling	The response to a named or auto request carries a tool call that matches the requested tool and argument format, and the final response to a multi_turn request reflects the tool result
Sampling path	A temperature 1.0 request exceeds the GSM8K lower bound of 0.5
Benchmark accuracy	Absolute gsm8k and mmlu_pro scores are recorded with lm-evaluation-harness

Reproduction method¶

The configuration below is pdd-pp4-dp4ep-throughput with prefix_caching on_sub_block, lmcache rds, and spec_decode eagle3 applied. This is the combination that the MiniMax-M2.7 recipe recommends. Prefill and decode each run on four RBLN-CR13 cards.

Recipe configuration, not the canonical configuration

These commands come from the MiniMax-M2.7 recipe, which was verified on 2026-10-07 with vllm-rbln 0.12.0 and RBLN Driver 3.5.1. Decode --max-num-seqs differs from the canonical configuration on Serving Configuration: these commands set it to 4, and the canonical configuration sets it to 1. Replace <POD_IP> with the IP address of the Pod that runs the server, and <EPP_HOST> with the host of the Endpoint Picker (EPP) of the llm-d router that receives the KV cache events.

Prefill¶

--gpu-memory-utilization 0.8 leaves NPU device memory outside the KV cache for RDS to use. KV Cache Offloading describes what RDS allocates there.

LMCACHE_CHUNK_SIZE=4096 \
LMCACHE_ENABLE_KV_EVENTS=True \
LMCACHE_EXTRA_CONFIG='{
  "storage_plugin.RDSBackend.module_path": "lmcache_rbln.storage_plugin.rds_backend",
  "storage_plugin.RDSBackend.class_name": "RDSBackend"
}' \
LMCACHE_LOCAL_CPU=False \
LMCACHE_MAX_LOCAL_CPU_SIZE=0 \
LMCACHE_PRE_CACHING_HASH_ALGORITHM=sha256_cbor_64bit \
LMCACHE_STORAGE_PLUGINS=RDSBackend \
PYTHONHASHSEED=42 \
VLLM_ENGINE_READY_TIMEOUT_S=9600 \
VLLM_NIXL_SIDE_CHANNEL_HOST='<POD_IP>' \
VLLM_NIXL_SIDE_CHANNEL_PORT=5559 \
vllm serve MiniMaxAI/MiniMax-M2.7 \
    --block-size 4096 \
    --enable-auto-tool-choice \
    --gpu-memory-utilization 0.8 \
    --kv-cache-dtype fp8 \
    --kv-events-config '{
      "enable_kv_cache_events": true,
      "publisher": "zmq",
      "endpoint": "tcp://<EPP_HOST>:5557",
      "topic": "kv@<POD_IP>:8000@MiniMaxAI/MiniMax-M2.7"
    }' \
    --kv-transfer-config '{
      "kv_connector": "MultiConnector",
      "kv_role": "kv_both",
      "kv_connector_extra_config": {
        "connectors": [
          {
            "kv_connector": "RblnNixlConnector",
            "kv_role": "kv_both",
            "kv_buffer_device": "rbln",
            "kv_connector_extra_config": {
              "name": "nixl"
            }
          },
          {
            "kv_connector": "RBLNLMCacheConnectorV1",
            "kv_connector_module_path": "lmcache_rbln.integration.vllm.connector",
            "kv_role": "kv_both",
            "kv_connector_extra_config": {
              "name": "lmcache"
            }
          }
        ]
      }
    }' \
    --max-model-len 204800 \
    --max-num-batched-tokens 512 \
    --max-num-seqs 4 \
    --model-impl vllm \
    --pipeline-parallel-size 4 \
    --port 8000 \
    --prefix-caching-hash-algo sha256_cbor \
    --reasoning-parser minimax_m2 \
    --safetensors-load-strategy prefetch \
    --speculative-config '{
      "method": "eagle3",
      "model": "asherszhang/MiniMax-M2.7-EAGLE3-draft-vocab200k",
      "num_speculative_tokens": 3
    }' \
    --tool-call-parser minimax_m2 \
    --trust-remote-code

Decode¶

Decode runs each of the four DP ranks as a separate process. The command below is for rank 0. The table after the command lists the values that change for each rank.

RBLN_VISIBLE_DEVICES=0 \
VLLM_ENGINE_READY_TIMEOUT_S=9600 \
VLLM_NIXL_SIDE_CHANNEL_HOST='<POD_IP>' \
VLLM_NIXL_SIDE_CHANNEL_PORT=5659 \
vllm serve MiniMaxAI/MiniMax-M2.7 \
    --block-size 4096 \
    --data-parallel-address '<POD_IP>' \
    --data-parallel-rank 0 \
    --data-parallel-rpc-port 5555 \
    --data-parallel-size 4 \
    --enable-auto-tool-choice \
    --enable-expert-parallel \
    --kv-cache-dtype fp8 \
    --kv-events-config '{
      "enable_kv_cache_events": true,
      "publisher": "zmq",
      "endpoint": "tcp://<EPP_HOST>:5557",
      "topic": "kv@<POD_IP>:8200@MiniMaxAI/MiniMax-M2.7"
    }' \
    --kv-transfer-config '{
      "kv_connector": "RblnNixlConnector",
      "kv_role": "kv_both",
      "kv_buffer_device": "rbln",
      "kv_connector_extra_config": {
        "name": "nixl"
      }
    }' \
    --max-model-len 204800 \
    --max-num-batched-tokens 512 \
    --max-num-seqs 4 \
    --model-impl vllm \
    --port 8200 \
    --prefix-caching-hash-algo sha256_cbor \
    --reasoning-parser minimax_m2 \
    --safetensors-load-strategy prefetch \
    --speculative-config '{
      "method": "eagle3",
      "model": "asherszhang/MiniMax-M2.7-EAGLE3-draft-vocab200k",
      "num_speculative_tokens": 3
    }' \
    --tool-call-parser minimax_m2 \
    --trust-remote-code

`--data-parallel-rank`	`RBLN_VISIBLE_DEVICES`	`VLLM_NIXL_SIDE_CHANNEL_PORT`	`--port`	`--kv-events-config` `endpoint`	`--kv-events-config` `topic`
`0`	`0`	`5659`	`8200`	`tcp://<EPP_HOST>:5557`	`kv@<POD_IP>:8200@MiniMaxAI/MiniMax-M2.7`
`1`	`1`	`5660`	`8201`	`tcp://<EPP_HOST>:5556`	`kv@<POD_IP>:8201@MiniMaxAI/MiniMax-M2.7`
`2`	`2`	`5661`	`8202`	`tcp://<EPP_HOST>:5555`	`kv@<POD_IP>:8202@MiniMaxAI/MiniMax-M2.7`
`3`	`3`	`5662`	`8203`	`tcp://<EPP_HOST>:5554`	`kv@<POD_IP>:8203@MiniMaxAI/MiniMax-M2.7`

lm-eval¶

Set GATEWAY to the OpenAI-compatible endpoint in front of the deployment above and OUT to the directory that receives the results, then run the two tasks in sequence. The --model_args value is double-quoted so that the shell expands the variable.

export GATEWAY=http://<gateway>:80
export OUT=<output-dir>

python -m lm_eval --model local-chat-completions \
  --model_args "base_url=${GATEWAY}/v1/chat/completions,model=MiniMaxAI/MiniMax-M2.7,num_concurrent=16,timeout=6000" \
  --tasks gsm8k \
  --apply_chat_template --fewshot_as_multiturn true \
  --gen_kwargs max_gen_toks=65536,seed=None,temperature=1.0,top_p=0.95 \
  --output_path "${OUT}"

python -m lm_eval --model local-chat-completions \
  --model_args "base_url=${GATEWAY}/v1/chat/completions,model=MiniMaxAI/MiniMax-M2.7,num_concurrent=16,timeout=6000" \
  --tasks mmlu_pro --limit 20 \
  --apply_chat_template --fewshot_as_multiturn true \
  --gen_kwargs max_gen_toks=65536,seed=None,temperature=1.0,top_p=0.95 \
  --output_path "${OUT}"

The harness is lm-eval[api]==0.4.12. seed=None is stated explicitly because a request that carries a seed rewinds the CPU generator offset in RBLN vLLM and halts the engine. 0.4.12 converts the string none to null, so this removes the default value of 1234.

These commands run generate_until tasks only. A loglikelihood task (e.g. a multiple-choice task) needs the logprobs of the prompt tokens, which the chat completions API does not return. Support for prompt_logprobs in RBLN vLLM was not validated.

Limitations¶

EAGLE-3 was validated only with the draft head asherszhang/MiniMax-M2.7-EAGLE3-draft-vocab200k, a head for MiniMax-M2.7.

Coding Assistant on Rebel100™ › Validation approach - the validation workload and how to benchmark the design
Serving Configuration - the canonical configuration
Deployment on Kubernetes - deploying the stack whose endpoint the accuracy commands call