whisper-base on RBLN NPU¶
{"attn": "eager", "axes": [{"axis": "lmcache", "options": [{"label": "Off", "value": "off"}, {"label": "LMCache (CPU)", "value": "cpu"}, {"label": "LMCache (RDS)", "value": "rds"}], "title": "KV Offload"}, {"axis": "kv_connector", "options": [{"label": "Off", "value": "off"}, {"label": "LMCache (CPU)", "value": "lmcache_cpu"}, {"label": "LMCache (RDS)", "value": "lmcache_rds"}], "title": "KV Offload"}, {"axis": "prefix_caching", "options": [{"label": "On", "value": "on"}, {"label": "On (sub-block)", "value": "on_sub_block"}, {"label": "Off", "value": "off"}], "title": "Prefix Caching"}, {"axis": "spec_decode", "options": [{"label": "Off", "value": "off"}, {"label": "N-gram", "value": "ngram"}, {"label": "EAGLE-3", "value": "eagle3"}, {"label": "Suffix", "value": "suffix"}, {"label": "MTP", "value": "mtp"}], "title": "Speculative Decoding"}, {"axis": "on_device_sampler", "options": [{"label": "On", "value": "on"}, {"label": "Off", "value": "off"}], "title": "On-device Sampler"}, {"axis": "structured_output", "options": [{"label": "On", "value": "on"}, {"label": "Off", "value": "off"}], "title": "Structured Output"}], "badges": ["Speech-to-Text", {"hl": true, "text": "vllm-rbln"}, "Audio"], "client": "curl http://localhost:8000/v1/audio/transcriptions \\\n -H \"Content-Type: multipart/form-data\" \\\n -F file=@audio.wav \\\n -F model=whisper-base", "features": {"api": true, "mm": true, "text": false}, "hf": "https://huggingface.co/openai/whisper-base", "installLine": "pip install rebel-compiler \"vllm-rbln[audio]\"", "model": "openai/whisper-base", "name": "whisper-base", "provider": "openai", "strategies": [{"label": "RSD", "nodes": null, "options": [{"attn": null, "features": {}, "label": null, "modelImpl": "transformers", "note": null, "npus": 1, "roles": [{"cmd": "vllm serve openai/whisper-base \\\n --model-impl transformers \\\n --rbln-num-devices-per-local-rank 1 \\\n --served-model-name whisper-base", "role": "main"}], "skus": ["RBLN-CR13"], "target": null, "verified": "2026-10-16"}, {"attn": null, "features": {}, "label": null, "modelImpl": "transformers", "note": null, "npus": 1, "roles": [{"cmd": "vllm serve openai/whisper-base \\\n --model-impl transformers \\\n --rbln-num-devices-per-local-rank 1 \\\n --served-model-name whisper-base", "role": "main"}], "skus": ["RBLN-CA22", "RBLN-CA25"], "target": null, "verified": "2026-10-16"}], "slug": "rsd", "tag": "Aggregated", "variant": "BF16"}], "text": {"en": {"strategyNote": "Rebellions Scalable Design (RSD): the RBLN Compiler partitions the model across NPU devices, or across the chiplets within one device. See \u003ca href=\"../features/model-parallelism.html#rebellions-scalable-design-rsd\"\u003eModel Parallelism\u003c/a\u003e for how RSD applies on each NPU, and \u003cstrong\u003eConfiguration\u003c/strong\u003e in the guide for the engine arguments.", "sub": "whisper-base \u2014 speech transcription model served on RBLN NPUs with RBLN vLLM over the OpenAI-compatible API.", "tag": "The vllm serve command compiles the model on first run and starts an OpenAI-compatible server."}, "ko": {"strategyNote": "Rebellions Scalable Design(RSD): RBLN \ucef4\ud30c\uc77c\ub7ec\uac00 \ubaa8\ub378\uc744 NPU \ub514\ubc14\uc774\uc2a4 \uac04 \ub610\ub294 \ub514\ubc14\uc774\uc2a4 \ud558\ub098\uc758 \uce69\ub81b \uac04\uc5d0 \ubd84\ud560\ud569\ub2c8\ub2e4. NPU\ubcc4 RSD \uc801\uc6a9 \ubc29\uc2dd\uc740 \u003ca href=\"../features/model-parallelism.html#rebellions-scalable-design-rsd\"\u003e\ubaa8\ub378 \ubcd1\ub82c\ud654\u003c/a\u003e\uc5d0\uc11c, \uc5d4\uc9c4 \uc778\uc790\ub294 \uac00\uc774\ub4dc\uc758 \u003cstrong\u003eConfiguration\u003c/strong\u003e\uc5d0\uc11c \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.", "sub": "whisper-base \u2014 \uc74c\uc131 \uc778\uc2dd \ubaa8\ub378, RBLN vLLM\uc73c\ub85c RBLN NPU\uc5d0\uc11c OpenAI \ud638\ud658 API\ub85c \uc11c\ube59\ud569\ub2c8\ub2e4.", "tag": "vllm serve \uba85\ub839\uc740 \ucd5c\ucd08 \uc2e4\ud589 \uc2dc \ubaa8\ub378\uc744 \ucef4\ud30c\uc77c\ud558\uace0 OpenAI \ud638\ud658 \uc11c\ubc84\ub97c \uc2dc\uc791\ud569\ub2c8\ub2e4."}}, "variant": "BF16"} Install the RBLN Driver on the host. An RBLN Portal account is required to install the RBLN Compiler (rebel-compiler). See the installation matrix for compatible package versions.
Speech transcription requires the audio extra, which installs the decoders used to process uploaded audio. Without it, the server starts, but the first transcription request fails.
Guide — overview and configuration
Overview¶
openai/whisper-base is a speech transcription model. On RBLN NPUs, the RBLN vLLM plugin compiles it on first run, then serves it through vLLM's /v1/audio/transcriptions endpoint.
Configuration¶
--rbln-num-devices-per-local-rank sets how many NPU devices one engine process uses, and the RBLN Compiler partitions the model for those devices with Rebellions Scalable Design (RSD). See Model Parallelism for how RSD applies on each NPU. The vision encoder is configured through --additional-config (optimum_overrides.visual). Changing these settings recompiles the model on the next start. Compile Cache lists every setting that does.
See the Configuration Guide for all engine arguments.
Notes and limitations¶
- The compiled artifact is cached on first run; changing the NPU count or engine arguments recompiles on the next start.