Chart configuration reference¶
This page lists the chart values to consult while configuring or changing the chart. The install procedure is in Installing the serving stack (Helm chart), not on this page.
Key chart settings¶
The following table lists the frequently used chart values. How to list every value and its default is in Full values list and typo validation scope.
| Key | Description | Default |
|---|---|---|
modelserver.model | The name of the model to serve. Change modelserver.model together with the rest of a model recipe (see Common configuration changes) | MiniMaxAI/MiniMax-M2.7 |
modelserver.image.{repository,tag,digest} | The modelserver image as repository, tag, and digest. Give a tag or a digest, not both. A change to modelserver.image replaces every modelserver Pod in the way modelserver.strategy sets | The default modelserver image |
modelserver.replicas.prefill / modelserver.replicas.decode | The P:D topology in automatic placement mode. For a role set to 0, the chart creates no Deployment and no compile-cache PersistentVolumeClaim (PVC) | 4 / 4 |
modelserver.nodePlacement | The P:D topology in exact placement mode: a list of a role and a node per Pod. To use nodePlacement, set modelserver.replicas.prefill and modelserver.replicas.decode to 0. The render fails when replicas above 0 is given together with nodePlacement, or when neither is given | [] |
modelserver.serveArgs / modelserver.serveEnv | Serve arguments and environment variables. For the values, see Serving Configuration | The MiniMax-M2.7 serve arguments |
modelserver.extraServeArgs | Per-role key for adding flags. The chart appends the extraServeArgs flags after serveArgs; if a flag is given twice, the later value applies. The three parallelism flags (--data-parallel-size, --tensor-parallel-size, --pipeline-parallel-size) in extraServeArgs fail the render; change them in modelserver.serveArgs | {} |
modelserver.extraServeEnv | Per-role key for adding environment variables. serveEnv applies to both roles, so set a variable that only one role needs in extraServeEnv. On a name collision, the extraServeEnv value wins. To change an LMCACHE_* variable of prefill, change extraServeEnv.prefill, because the same variable in serveEnv is overridden there | {prefill: {LMCACHE_*}} (seven LMCache variables) |
modelserver.removeServeArgs | Per-role key for removing flags from serveArgs; write only the flag name. A flag that is not in the serveArgs of that same role fails the render. The three parallelism flags cannot be removed | {} |
modelserver.resources | CPU and memory per modelserver Pod. For the node budget, see Resources and the node budget | 40 CPUs, 630Gi memory |
storage.{model,compileCache,modelOffload,dshm} | The volumes of the modelserver Pod (see Volumes). storage.model has no usable default and must be set. The chart defaults contain no StorageClass name for your cluster | model = no default (required), compileCache = kind: pvc + storageClass: "", modelOffload = enabled: false |
devices.claimTemplateName | The name of the ResourceClaimTemplate (RCT) the Pod references (see ResourceClaimTemplate for npu-rdma) | npu-rdma |
gateway.enabled / gateway.gatewayClassName | Whether the chart creates the Gateway, and the GatewayClass provider of that Gateway (see External exposure and the gateway Service) | true / istio |
gateway.annotations | Annotations on the Gateway (see External exposure and the gateway Service) | {} |
gateway.agentgateway.parameters.service | The agentgateway Service spec, for example its type. The chart applies gateway.agentgateway.parameters.service only with agentgateway | {} |
llm-d-router-gateway.* | Keys passed through to the llm-d router subchart: replicas, resources, and all plugin settings | Subchart defaults |
llm-d-router-gateway.router.modelServers.targetPorts | The list of ports the EPP dials. The number of ports must equal the number of decode data-parallel ranks. A different number fails the render (targetPorts has N entries but decode runs M independent engines — the count is dp, not the number of ports an engine opens. …). With pipeline parallelism, the prefill engine listens on 8000 only, and pp-port-shim in the prefill Pod listens on 8001 to 8003 and forwards them to 8000 (see Container ports) | [8000, 8001, 8002, 8003] |
monitoring.enabled | Whether the chart creates the monitoring custom resources (CRs). With monitoring.enabled: true on the default VictoriaMetrics Operator, also set monitoring.scraper.serviceAccount.{name,namespace}, or the render fails. monitoring.backend: prometheus does not need it (see Monitoring › Metric collection targets) | false |
imagePullSecrets | The list of Image Pull Secret names for pulling the modelserver image | [] |
modelserver.podOverrides.<role> | Pod spec extension. The chart merges maps deeply and replaces lists whole. For the fields podOverrides rejects, see Full values list and typo validation scope | unset |
Post-install changes, rollbacks, and removal are on the Operations page.
Common configuration changes¶
The following table shows how to apply each kind of change.
| Change | How to apply | See |
|---|---|---|
| P:D topology: scaling within the same placement mode | Run helm upgrade | Scaling and Upgrades › Change rules |
P:D topology: switching between replicas and nodePlacement | Uninstall the Helm release, then install it. A nodePlacement example is in the chart's examples/node-placement.yaml | Scaling and Upgrades › Change rules |
| Storage | Choose the volume type with the kind: key of each storage.<volume> | The table in Volumes |
| Model: a model other than MiniMax-M2.7 | Uninstall the Helm release. Then apply the recipe's RCT with kubectl apply -f, and install with the recipe's values file passed with -f. The recipes are under the chart's examples/models/<model>/, and their install commands are in examples/models/README.md | N/A |
Image: modelserver.image | Changing modelserver.image recreates the modelserver Pods. To avoid recompiling, keep the compile-cache PVC | Scaling and Upgrades › Impact by change type, Rolling updates, Cache reuse condition |
| Data plane: switching to agentgateway (Unverified) | Apply the chart's examples/gateway-agentgateway.yaml with -f | GatewayClass provider |
| EPP configuration | Set the upstream chart's values as is under the llm-d-router-gateway: key. If you change llm-d-router-gateway.router.epp.pluginsCustomConfig, restart the EPP yourself after helm upgrade (the following command) | Serving Configuration, Scaling and Upgrades › Impact by change type |
Warning
-f takes values files only. A Kubernetes manifest from examples/manifests/ (the RCT or external exposure) passed with -f fails the render. The schema rejects the manifest with additional properties ... not allowed naming apiVersion, kind, metadata, and spec. Apply manifests with kubectl apply -f.
External exposure and the gateway Service¶
The chart does not expose the stack outside the cluster. Connect the gateway Service to the ingress or load balancer your cluster operates.
The chart does not provide authentication or authorization for the serving endpoint. Handle authentication and authorization at the gateway layer or in front of the gateway.
Attaching to a Gateway the cluster manages: to use an existing Gateway instead of the chart's Gateway, set the following values. Attaching to an existing Gateway is Unverified.
The chart does not create the gateway Service. Istio creates the gateway Service from the Gateway under the name <gateway-name>-istio. With the chart's default Gateway name, the gateway Service is <release-name>-inference-gateway-istio, the name used in First inference. On a cluster that cannot assign a LoadBalancer address, switch the gateway Service type to ClusterIP through gateway.annotations.
Full values list and typo validation scope¶
The table in Key chart settings covers only the frequently used values. List every value and its default as follows.
Specifying the following fields through podOverrides fails the render. Use the key in the Use instead column; the render error names that key.
Field in podOverrides | Use instead |
|---|---|
containers | modelserver.serveEnv, modelserver.serveArgs, modelserver.resources |
initContainers | routing.proxy.image |
volumes | storage.* |
volumeMounts | storage.<name>.mountPath |
imagePullSecrets | The top-level imagePullSecrets |
metadata | modelserver.podAnnotations, modelserver.podLabels |
To add volumes, mounts, or containers, use the append keys modelserver.extraVolumes, modelserver.extraVolumeMounts, modelserver.extraContainers, and modelserver.extraInitContainers.
Warning
A typo in a structural key fails the render. The schema rejects unknown keys in the chart's own blocks, from the top level down; for example, monitoring.enabeld=true is rejected with at '/monitoring': additional properties 'enabeld' not allowed. The schema cannot check maps whose key names you choose, so a typo inside the following maps is ignored without an error:
modelserver.serveEnv,modelserver.extraServeEnv.<role>modelserver.podOverrides.<role>(Pod spec fields)gateway.annotations,gateway.agentgateway.parameters.service.{metadata,spec}storage.<volume>.selector/annotations, the keys insiderouting.proxy.resources.{requests,limits},global- everything under
llm-d-router-gateway.*(the subchart has no schema) - the items of
modelserver.extraVolumes,modelserver.extraVolumeMounts,modelserver.extraContainers, andmodelserver.extraInitContainers(Kubernetes objects passed through as is)
If a --set is not taking effect, check the applied values with helm get values. The chart's values.schema.json is the authoritative source for the full validation scope.