Recipes¶
Configurations for serving models on RBLN NPUs.
vllm serve launches a model as an OpenAI-compatible inference server. On RBLN NPUs, it also accepts options that control how the model is compiled.
Each recipe is a verified configuration for a single model: the packages to install, the settings it is served with, and an example client request.
Find your recipe¶
- Choose your NPU and find your model.
- Open the recipe to see the required packages, hardware, and configuration.
- Run the recipe, then send a request to verify that the server responds.
Recipes are verified configurations, not fixed requirements. Use a recipe directly or adjust its configuration for your deployment. New variants are added as they are tested.
- For supported model architectures, see RBLN vLLM.
- For available configuration options, see the Configuration Guide.
- For model-level feature support, see the Feature Matrix.
Supported models¶
| Model | Task | Architecture |
|---|---|---|
| A.X-4.0-Light | Text generation | Dense |
| A.X-K2 | Text generation | MoE |
| blip2-opt-2.7b | Multimodal | Dense |
| blip2-opt-6.7b | Multimodal | Dense |
| Cosmos-Reason1-7B | Multimodal | Dense |
| DeepSeek-R1-Distill-Llama-8B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-1.5B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-14B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-32B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-7B | Text generation | Dense |
| DeepSeek-V3.2 | Text generation | MoE |
| EXAONE-3.0-7.8B-Instruct | Text generation | Dense |
| EXAONE-3.5-2.4B-Instruct | Text generation | Dense |
| EXAONE-3.5-32B-Instruct | Text generation | Dense |
| EXAONE-3.5-7.8B-Instruct | Text generation | Dense |
| gemma-2-9b-it | Text generation | Dense |
| gemma-2b-it | Text generation | Dense |
| gemma-3-12b-it | Multimodal | Dense |
| gemma-3-27b-it | Multimodal | Dense |
| gemma-3-4b-it | Multimodal | Dense |
| gemma-4-26B-A4b-it | Multimodal | MoE |
| gemma-4-31B-it | Multimodal | Dense |
| gemma-7b-it | Text generation | Dense |
| gpt-oss-120b | Text generation | MoE |
| gpt-oss-20b | Text generation | MoE |
| gpt2-large | Text generation | Dense |
| gpt2-medium | Text generation | Dense |
| gpt2-xl | Text generation | Dense |
| gpt2 | Text generation | Dense |
| Idefics3-8B-Llama3 | Multimodal | Dense |
| KONI-Llama3.1-8B-Instruct-20241024 | Text generation | Dense |
| Llama-2-13b-chat-hf | Text generation | Dense |
| Llama-2-7b-chat-hf | Text generation | Dense |
| Meta-Llama-3-8B-Instruct | Text generation | Dense |
| Llama-3.1-8B-Instruct | Text generation | Dense |
| Llama-3.2-3B-Instruct | Text generation | Dense |
| llava-1.5-7b-hf | Multimodal | Dense |
| llava-v1.6-mistral-7b-hf | Multimodal | Dense |
| Midm-2.0-Base-Instruct | Text generation | Dense |
| Midm-2.0-Mini-Instruct | Text generation | Dense |
| MiniMax-M2.7 | Text generation | MoE |
| Mistral-7B-Instruct-v0.3 | Text generation | Dense |
| opt-2.7b | Text generation | Dense |
| opt-6.7b | Text generation | Dense |
| paligemma-3b-mix-224 | Multimodal | Dense |
| paligemma2-3b-mix-224 | Multimodal | Dense |
| phi-2 | Text generation | Dense |
| pixtral-12b | Multimodal | Dense |
| Qwen2-7B-Instruct | Text generation | Dense |
| Qwen2-VL-7B-Instruct | Multimodal | Dense |
| Qwen2.5-0.5B-Instruct | Text generation | Dense |
| Qwen2.5-1.5B-Instruct | Text generation | Dense |
| Qwen2.5-14B-Instruct | Text generation | Dense |
| Qwen2.5-32B-Instruct | Text generation | Dense |
| Qwen2.5-3B-Instruct | Text generation | Dense |
| Qwen2.5-7B-Instruct | Text generation | Dense |
| Qwen2.5-VL-7B-Instruct | Multimodal | Dense |
| Qwen3-0.6B | Text generation | Dense |
| Qwen3-1.7B | Text generation | Dense |
| Qwen3-30B-A3B | Text generation | MoE |
| Qwen3-32B | Text generation | Dense |
| Qwen3-4B | Text generation | Dense |
| Qwen3-8B | Text generation | Dense |
| Qwen3-VL-2B-Instruct | Multimodal | Dense |
| Qwen3-VL-30B-A3B-Instruct | Multimodal | MoE |
| Qwen3-VL-4B-Instruct | Multimodal | Dense |
| Qwen3-VL-8B-Instruct | Multimodal | Dense |
| Qwen3.5-0.8B | Multimodal | Dense |
| Qwen3.5-27B | Multimodal | Dense |
| Qwen3.5-2B | Multimodal | Dense |
| Qwen3.5-4B | Multimodal | Dense |
| Qwen3.5-9B | Multimodal | Dense |
| Qwen3.6-27B | Multimodal | Dense |
| Qwen3.8-27B | Multimodal | Dense |
| salamandra-7b-instruct | Text generation | Dense |
| SOLAR-10.7B-Instruct-v1.0 | Text generation | Dense |
| whisper-base | Automatic Speech Recognition | — |
| whisper-large-v3-turbo | Automatic Speech Recognition | — |
| whisper-large-v3 | Automatic Speech Recognition | — |
| whisper-medium | Automatic Speech Recognition | — |
| whisper-small | Automatic Speech Recognition | — |
| whisper-tiny | Automatic Speech Recognition | — |
| Model | Task | Architecture |
|---|---|---|
| A.X-4.0-Light | Text generation | Dense |
| bge-base-en-v1.5 | Embedding | — |
| bge-large-en-v1.5 | Embedding | — |
| bge-m3 | Embedding | — |
| bge-reranker-base | Reranking | — |
| bge-reranker-large | Reranking | — |
| bge-reranker-v2-m3 | Reranking | — |
| bge-small-en-v1.5 | Embedding | — |
| blip2-opt-2.7b | Multimodal | Dense |
| blip2-opt-6.7b | Multimodal | Dense |
| Cosmos-Reason1-7B | Multimodal | Dense |
| DeepSeek-R1-Distill-Llama-70B | Text generation | Dense |
| DeepSeek-R1-Distill-Llama-8B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-1.5B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-14B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-32B | Text generation | Dense |
| DeepSeek-R1-Distill-Qwen-7B | Text generation | Dense |
| EXAONE-3.0-7.8B-Instruct | Text generation | Dense |
| EXAONE-3.5-2.4B-Instruct | Text generation | Dense |
| EXAONE-3.5-32B-Instruct | Text generation | Dense |
| EXAONE-3.5-7.8B-Instruct | Text generation | Dense |
| EXAONE-4.5-33B | Multimodal | Dense |
| gemma-2-2b-it | Text generation | Dense |
| gemma-2-9b-it | Text generation | Dense |
| gemma-2b-it | Text generation | Dense |
| gemma-3-12b-it | Multimodal | Dense |
| gemma-3-27b-it | Multimodal | Dense |
| gemma-3-4b-it | Multimodal | Dense |
| gemma-4-26B-A4b-it | Multimodal | MoE |
| gemma-4-31B-it | Multimodal | Dense |
| gemma-7b-it | Text generation | Dense |
| gpt-oss-20b | Text generation | MoE |
| gpt2-large | Text generation | Dense |
| gpt2-medium | Text generation | Dense |
| gpt2-xl | Text generation | Dense |
| gpt2 | Text generation | Dense |
| Idefics3-8B-Llama3 | Multimodal | Dense |
| KONI-Llama3.1-8B-Instruct-20241024 | Text generation | Dense |
| Llama-2-13b-chat-hf | Text generation | Dense |
| Llama-2-7b-chat-hf | Text generation | Dense |
| Meta-Llama-3-8B-Instruct | Text generation | Dense |
| Llama-3.1-70B-Instruct | Text generation | Dense |
| Llama-3.1-8B-Instruct | Text generation | Dense |
| Llama-3.2-1B-Instruct | Text generation | Dense |
| Llama-3.2-3B-Instruct | Text generation | Dense |
| Llama-3.3-70B-Instruct | Text generation | Dense |
| llava-1.5-7b-hf | Multimodal | Dense |
| llava-v1.6-mistral-7b-hf | Multimodal | Dense |
| Midm-2.0-Base-Instruct | Text generation | Dense |
| Midm-2.0-Mini-Instruct | Text generation | Dense |
| Mistral-7B-Instruct-v0.3 | Text generation | Dense |
| opt-125m | Text generation | Dense |
| opt-2.7b | Text generation | Dense |
| opt-6.7b | Text generation | Dense |
| opt-66b | Text generation | Dense |
| paligemma-3b-mix-224 | Multimodal | Dense |
| paligemma2-3b-mix-224 | Multimodal | Dense |
| phi-2 | Text generation | Dense |
| pixtral-12b | Multimodal | Dense |
| Qwen2-7B-Instruct | Text generation | Dense |
| Qwen2-VL-7B-Instruct | Multimodal | Dense |
| Qwen2.5-0.5B-Instruct | Text generation | Dense |
| Qwen2.5-1.5B-Instruct | Text generation | Dense |
| Qwen2.5-14B-Instruct | Text generation | Dense |
| Qwen2.5-32B-Instruct | Text generation | Dense |
| Qwen2.5-3B-Instruct | Text generation | Dense |
| Qwen2.5-72B-Instruct | Text generation | Dense |
| Qwen2.5-7B-Instruct | Text generation | Dense |
| Qwen2.5-VL-7B-Instruct | Multimodal | Dense |
| Qwen3-0.6B | Text generation | Dense |
| Qwen3-1.7B | Text generation | Dense |
| Qwen3-30B-A3B | Text generation | MoE |
| Qwen3-32B | Text generation | Dense |
| Qwen3-4B | Text generation | Dense |
| Qwen3-8B | Text generation | Dense |
| Qwen3-Embedding-0.6B | Embedding | — |
| Qwen3-Embedding-4B | Embedding | — |
| Qwen3-Reranker-0.6B | Reranking | — |
| Qwen3-Reranker-4B | Reranking | — |
| Qwen3-VL-2B-Instruct | Multimodal | Dense |
| Qwen3-VL-30B-A3B-Instruct | Multimodal | MoE |
| Qwen3-VL-32B-Instruct | Multimodal | Dense |
| Qwen3-VL-4B-Instruct | Multimodal | Dense |
| Qwen3-VL-8B-Instruct | Multimodal | Dense |
| Qwen3.5-0.8B | Multimodal | Dense |
| Qwen3.5-27B | Multimodal | Dense |
| Qwen3.5-2B | Multimodal | Dense |
| Qwen3.5-4B | Multimodal | Dense |
| Qwen3.5-9B | Multimodal | Dense |
| Qwen3.6-27B | Multimodal | Dense |
| Qwen3.8-27B | Multimodal | Dense |
| salamandra-7b-instruct | Text generation | Dense |
| SOLAR-10.7B-Instruct-v1.0 | Text generation | Dense |
| twitter-roberta-base-emotion | Classification | — |
| whisper-base | Automatic Speech Recognition | — |
| whisper-large-v3-turbo | Automatic Speech Recognition | — |
| whisper-large-v3 | Automatic Speech Recognition | — |
| whisper-medium | Automatic Speech Recognition | — |
| whisper-small | Automatic Speech Recognition | — |
| whisper-tiny | Automatic Speech Recognition | — |