Compile cache¶
Goal: compile the model once on the first startup, so that later startups and replica scale-out reuse the compile results.
First startup time¶
Compilation takes most of the first startup of a modelserver Pod. The modelserver stores the compile results in the compile-cache PersistentVolumeClaim (PVC). Once the compile cache is filled, restarted Pods and Pods added by scale-out start without recompiling.
| Phase | Time | Measurement condition |
|---|---|---|
| First compile (cold) | About 11 minutes until all Pods are Ready | MiniMax-M2.7, two prefill Pods and two decode Pods, empty compile cache (measured apart from the Target Configuration validation) |
| Restart after the compile cache is filled (warm) | Model loading time only, without compilation | N/A |
The startup budget of the modelserver Pods must cover the first compile time. For the startup budget, see Startup and readiness probes.
PVC shape, capacity, and accessModes¶
The chart creates one compile-cache PVC each for prefill and decode. The replicas of the same P/D role share one PVC. The chart creates no PVC for a P/D role whose replica count is 0.
Decide the PVC size and accessModes before installing. The reference configuration uses the chart defaults as they are.
| Key | Chart default | Change after install |
|---|---|---|
storage.compileCache.size | 200Gi | Can grow only when the StorageClass allows volume expansion. helm install and helm upgrade reject a value smaller than the existing PVC; helm template and GitOps renders skip this check |
storage.compileCache.accessModes | ["ReadWriteMany"] | Cannot change. Kubernetes does not allow changing the spec of an existing PVC |
The size of the compile results depends on the model size and the parallelism. A larger pipeline-parallel (PP) degree adds compile results for each rank. If the capacity runs out, the modelserver Pod fails during compilation with Disk quota exceeded. Check the current usage from a running modelserver Pod with the following command.
With ReadWriteMany (RWX), Pods of the same P/D role on different nodes share the compile cache. If the StorageClass does not support RWX, the compile-cache PVC stays Pending with a ProvisioningFailed event. In that case, set storage.compileCache.accessModes to ["ReadWriteOnce"] and run prefill and decode with one Pod each. The ReadWriteOnce setup is Unverified.
Cache reuse condition¶
The compile cache is reused only when the RBLN Compiler major.minor version matches. The serving settings in the cache key (see Compile Cache › Cache key) and the NPU type must also match. If the RBLN Compiler version, a cache-key setting, or the NPU type differs, the modelserver cannot reuse the compile cache and compiles the model from scratch.
Swapping the modelserver image for one with a different SDK version also changes the RBLN Compiler version. If the compile cache cannot be reused, delete the compile-cache PVC and let the next startup refill the compile cache.
Cache retention across uninstall¶
helm uninstall does not delete the compile-cache PVC, because the chart sets helm.sh/resource-policy: keep on it. A reinstall with the same Helm release name reuses the remaining compile cache. To delete the compile-cache PVC, see Scaling and Upgrades › What remains.