Skip to content

Conversation

dylantangredi-jakala
Copy link
Collaborator

@dylantangredi-jakala dylantangredi-jakala commented Oct 6, 2025

List of Changes

  • upgraded helm chart to v8.0.2
  • added rolling update strategy (maxUnavailable, maxSurge)
  • overridden deployment replicas to match HPA minReplica config

Motivation and Context

This upgrade was required to address an issue with the rollout strategy and the autoscaler behaviour (if it will be enabled on this service), where the pods would scale down to 1 during deploy instead of keeping the same number of pods and doing +1 -1 to avoid traffic congestion. Also replicas are set to match the autoscaling config during deploy, to avoid starting the service with 1 pod when there is relevant traffic on the service.
This also ensures zero-downtime deployments by keeping all existing pods running until new versions are healthy and ready, then replacing them one at a time.

How Has This Been Tested?

Temporarily set maxReplica to 3, lowered the CPU trigger to 1 to force the scale up, then triggered a manual deploy. Correct behaviour in the screenshot (no scale down and rolling update working, +1 -1 pods):

image

Complete process:

❯ helm dependency update
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://pagopa.github.io/aks-microservice-chart-blueprint" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading microservice-chart from repo https://pagopa.github.io/aks-microservice-chart-blueprint
Deleting outdated charts

VERY IMPORTANT
The scripts needs to be adapted for every service and launched on each environment before actually upgrading through helm upgrade (ref. https://github.com/pagopa/aks-microservice-chart-blueprint/blob/main/scripts/migrate_from_v7_to_v8.sh)

❯ ../migrate_from_v7_to_v8.sh apiconfig-selfcare-integration-postgresql apiconfig-selfcare-integration apiconfig
scaledobject.keda.sh/apiconfig-selfcare-integration-postgresql not labeled
scaledobject.keda.sh/apiconfig-selfcare-integration-postgresql not labeled
scaledobject.keda.sh/apiconfig-selfcare-integration-postgresql annotated
scaledobject.keda.sh/apiconfig-selfcare-integration-postgresql annotated
❯ kubectl get scaledobject apiconfig-selfcare-integration-postgresql -n apiconfig -o json | jq '{
    labels: {
      "app.kubernetes.io/instance": .metadata.labels."app.kubernetes.io/instance",
      "app.kubernetes.io/managed-by": .metadata.labels."app.kubernetes.io/managed-by"
    },
    annotations: {
      "meta.helm.sh/release-name": .metadata.annotations."meta.helm.sh/release-name",
      "meta.helm.sh/release-namespace": .metadata.annotations."meta.helm.sh/release-namespace"
    }
  }'
{
  "labels": {
    "app.kubernetes.io/instance": "apiconfig-selfcare-integration",
    "app.kubernetes.io/managed-by": "Helm"
  },
  "annotations": {
    "meta.helm.sh/release-name": "apiconfig-selfcare-integration",
    "meta.helm.sh/release-namespace": "apiconfig"
  }
}
❯ helm upgrade --namespace apiconfig \
    --install --values ./values-dev.yaml \
    --set postgresql.azure.workloadIdentityClientId=xxxxx \
    --set postgresql.podAnnotations.force-rollout="rollout-$(date +%s)" \
    --wait --timeout 15m0s \
    apiconfig-selfcare-integration .
Release "apiconfig-selfcare-integration" has been upgraded. Happy Helming!
NAME: apiconfig-selfcare-integration
LAST DEPLOYED: Wed Oct 15 12:18:51 2025
NAMESPACE: apiconfig
STATUS: deployed
REVISION: 75
TEST SUITE: None
❯ kubectl get scaledobject apiconfig-selfcare-integration-postgresql -n apiconfig -o jsonpath='{.metadata.labels.helm\.sh/blueprint-version}'
7.5.0%

NOTE: still on 7.5.0 so we delete the scaled object and redeploy

❯ kubectl delete scaledobject apiconfig-selfcare-integration-postgresql -n apiconfig
scaledobject.keda.sh "apiconfig-selfcare-integration-postgresql" deleted from apiconfig namespace
❯ helm upgrade --namespace apiconfig \
    --install --values ./values-dev.yaml \
    --set postgresql.azure.workloadIdentityClientId=xxxxx \
    --set postgresql.podAnnotations.force-rollout="rollout-$(date +%s)" \
    --wait --timeout 15m0s \
    apiconfig-selfcare-integration .
Release "apiconfig-selfcare-integration" has been upgraded. Happy Helming!
NAME: apiconfig-selfcare-integration
LAST DEPLOYED: Wed Oct 15 12:26:05 2025
NAMESPACE: apiconfig
STATUS: deployed
REVISION: 76
TEST SUITE: None
❯ kubectl get scaledobject apiconfig-selfcare-integration-postgresql -n apiconfig -o jsonpath='{.metadata.labels.helm\.sh/blueprint-version}'
8.0.2%

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as
    expected)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@dylantangredi-jakala dylantangredi-jakala self-assigned this Oct 6, 2025
@dylantangredi-jakala dylantangredi-jakala requested a review from a team as a code owner October 6, 2025 10:57
@dylantangredi-jakala dylantangredi-jakala added the minor Semantic Version: minor change versioning label Oct 6, 2025
@dylantangredi-jakala dylantangredi-jakala added skip Semantic Version: skip versioning and removed minor Semantic Version: minor change versioning labels Oct 7, 2025
@dylantangredi-jakala dylantangredi-jakala added the ignore-for-release Ignore pull request from the release label Oct 15, 2025
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
10 Security Hotspots
E Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@dylantangredi-jakala dylantangredi-jakala marked this pull request as draft October 15, 2025 10:43
@dylantangredi-jakala dylantangredi-jakala added patch Semantic Version: bug fix versioning and removed ignore-for-release Ignore pull request from the release skip Semantic Version: skip versioning labels Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch Semantic Version: bug fix versioning size/small

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant