Litellm backend cache config#29605
Conversation
[Infra] Promote internal staging to main
[Infra] Promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
…end pod mirroring litellm-gateway pod
|
Tin Chi Lo seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR adds ConfigMap-based proxy config support to the backend Helm component, mirroring the equivalent feature that already exists for the gateway.
Confidence Score: 4/5Safe to merge; the change is additive and gated behind config.create: false by default, so existing deployments are unaffected. The change faithfully ports the gateway config pattern to the backend. The one nuance is that the annotations block is now always emitted in the pod template even when both config.create is false (default) and podAnnotations is empty, producing an empty annotations: key. The gateway avoids this because its config.create defaults to true. This won't break deployments but will produce diff noise for GitOps users who upgrade from the previous chart version. helm/litellm/templates/backend/deployment.yaml — the annotations block restructuring
|
| Filename | Overview |
|---|---|
| helm/litellm/templates/backend/configmap.yaml | New ConfigMap template for backend proxy config; mirrors the existing gateway configmap exactly and is correctly gated on backend.config.create |
| helm/litellm/templates/backend/deployment.yaml | Adds checksum annotation, CONFIG_FILE_PATH env var, volumeMount, and volume for the backend config; restructures annotations block so it is always emitted even when empty (default state) |
| helm/litellm/values.yaml | Adds backend.config.create (false by default) and backend.config.proxy_config ({}) to mirror the existing gateway.config shape |
Reviews (1): Last reviewed commit: "change deployment configs to include a l..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| annotations: | ||
| {{- if .Values.backend.config.create }} | ||
| checksum/config: {{ include (print $.Template.BasePath "/backend/configmap.yaml") . | sha256sum }} | ||
| {{- end }} | ||
| {{- with .Values.backend.podAnnotations }} | ||
| {{- toYaml . | nindent 8 }} | ||
| {{- end }} | ||
| {{- end }} |
There was a problem hiding this comment.
backend.config.create is false (the default) and backend.podAnnotations is empty (also the default), the rendered pod template will contain a bare annotations: key with no children — equivalent to annotations: null. While Kubernetes accepts this, it produces unnecessary noise in GitOps diff tools (ArgoCD, Flux) for users who upgrade from the previous chart version, and it differs from the original conditional behavior. The gateway avoids this because its config.create defaults to true, ensuring the checksum annotation is always present. Wrapping the whole block in a top-level conditional keeps the annotations section absent when nothing is to be emitted.
| annotations: | |
| {{- if .Values.backend.config.create }} | |
| checksum/config: {{ include (print $.Template.BasePath "/backend/configmap.yaml") . | sha256sum }} | |
| {{- end }} | |
| {{- with .Values.backend.podAnnotations }} | |
| {{- toYaml . | nindent 8 }} | |
| {{- end }} | |
| {{- end }} | |
| {{- if or .Values.backend.config.create .Values.backend.podAnnotations }} | |
| annotations: | |
| {{- if .Values.backend.config.create }} | |
| checksum/config: {{ include (print $.Template.BasePath "/backend/configmap.yaml") . | sha256sum }} | |
| {{- end }} | |
| {{- with .Values.backend.podAnnotations }} | |
| {{- toYaml . | nindent 8 }} | |
| {{- end }} | |
| {{- end }} |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
4 participants
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🐛 Bug Fix
Changes