v1.24.0

[gardener-extension-provider-azure]

🐛 Bug Fixes

• [USER] A bug has been fixed that could lead to no egress connectivity for a vmss flex (vmo) based Shoot clusters. This could happen if a cluster is using the reverse-vpn feature and has no nat-gateway configured. (gardener/gardener-extension-provider-azure#423, @dkistner)

📰 Noteworthy

• [USER] Since go1.17 both net.ParseIP and net.ParseCIDR reject leading zeros in the dot-decimal notation of IPv4 addresses. With the update to go1.17, admission-azure now rejects Shoot objects with CIDR ranges that have such leading zeros in the dot-decimal notation. Before updating to this version of admission-azure, make sure that there are no Shoot objects with leading zeros in the dot-decimal notation of an IPv4 address. For reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 (gardener/gardener-extension-provider-azure#418, @rfranzke)
• [DEVELOPER] The Golang version has been updated to 1.17.5. (gardener/gardener-extension-provider-azure#418, @rfranzke)

v1.23.1

[gardener-extension-provider-azure]

🐛 Bug Fixes

• [OPERATOR] Finalizers are now properly added to the Worker resource at the start of a restore operation. (gardener/gardener-extension-provider-azure#422, @ialidzhikov)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.23.1
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.23.1

v1.23.0

[gardener-extension-provider-azure]

⚠️ Breaking Changes

• [OPERATOR] Removed azuremachineclass cleanup logic. (gardener/gardener-extension-provider-azure#407, @kon-angelo)
• [OPERATOR] The logic to migrate the Terraform natgateway state will be removed. In case there are Infrastructure resources that are created with < v1.15 and not reconciled with >= v1.15 they need to be reconciled before a release with this change is applied. (gardener/gardener-extension-provider-azure#406, @dkistner)

✨ New Features

• [OPERATOR] The node and service controllers now have a configurable sync period (defaults to 4h), so that if an event is missed, the next reconcile will happen at most after this period. (gardener/gardener-extension-provider-azure#405, @stoyanr)
  • The node and service predicates now use an expiring cache of nodes / services, so that if an event is missed, the next event is likely to cause a reconcile after comparing with the cache.
  • The node and service controllers now also watch "owned" VirtualMachine and PublicIPAddress resources and would trigger a reconcile if a an unexpected situation is detected, such as creating or updating an object without an owner, deleting an object with an owner that is not being deleted, etc.
  • The node and service controllers have been enhanced to ensure that any owned objects are deleted if the node or service is not found.
  • The service controller has been enhanced to also delete any still existing "owned" PublicIPAddress resources that are not found in the service LoadBalancer IPs when deleting a service or ensuring that it's deleted.
• [OPERATOR] Operators can now provide a list of service principals that can be used in case the user does not provide service principal information in the secret for her/his Shoot. (gardener/gardener-extension-provider-azure#401, @dkistner)

🏃 Others

• [OPERATOR] Azure provider extension now uses non-strict decoder for InfrastructrureStatus resources. (gardener/gardener-extension-provider-azure#414, @kon-angelo)
• [DEVELOPER] The rewrite_tag filter in the logging configuration is replaced by modify one (gardener/gardener-extension-provider-azure#386, @vlvasilev)

[cloud-provider-azure]

🏃 Others

• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.20.12. (gardener/cloud-provider-azure@c36b6edcc10f)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.21.6. (gardener/cloud-provider-azure@03ebe6747186)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.22.3. (gardener/cloud-provider-azure@e083e3b71091)

[machine-controller-manager]

✨ New Features

• [USER] End User can now delete the backing machine object of the node instantly by annotating the desired node with 'node.machine.sapcloud.io/trigger-deletion-by-mcm="true"` (gardener/machine-controller-manager#648, @AxiomSamarth)
• [USER] Added *expectedNodeDetails field to the MachineClass API (gardener/machine-controller-manager#644, @AxiomSamarth)

🐛 Bug Fixes

• [OPERATOR] A bug has been fixed in the pre-delivered CRD manifests for MCM (/kubernetes/crds). It caused data to be pruned from MCM related resources and led to reconciliation issues. (gardener/machine-controller-manager#641, @timuthy)

📖 Documentation

• [DEVELOPER] make generate now generates v1 version of CRDs by default instead of v1beta1. (gardener/machine-controller-manager#640, @himanshu-kun)

🏃 Others

• [USER] Update Kubernetes dependency versions to v1.20.6 (gardener/machine-controller-manager#601, @AxiomSamarth)

[terraformer]

🐛 Bug Fixes

• [DEVELOPER] A bug has been fixed preventing to use Terraformer with a Terraform version >= 0.13. (gardener/terraformer#102, @rfranzke)

🏃 Others

• [OPERATOR] terraform has been upgraded to 0.13.7 (gardener/terraformer#105, @stoyanr)
• [OPERATOR] The following terraform provider plugins are updated: (gardener/terraformer#104, @ialidzhikov)
  • hashicorp/terraform-provider-aws: 3.54.0 -> 3.63.0
• [OPERATOR] The following terraform provider plugins are updated: (gardener/terraformer#101, @ialidzhikov)
  • hashicorp/terraform-provider-aws: 3.32.0 -> 3.54.0

📰 Noteworthy

• [DEVELOPER] The version for the equinixmetal Terraform provider plugin has been updated to 3.1.0. (gardener/terraformer#103, @rfranzke)

v1.22.0

[gardener-extension-provider-azure]

✨ New Features

• [USER] The Azure extension does now support shoot clusters with Kubernetes version 1.22. You should consider the Kubernetes release notes before upgrading to 1.22. (#368, @timuthy)

🏃 Others

• [OPERATOR] machine-controller-manager logs are exposed to the end-users (#347, @vlvasilev)
• [DEVELOPER] Missing or wrong doc comments and a few other common style errors will now be reported by the linter. (#372, @stoyanr)

[cloud-provider-azure]

✨ New Features

• [DEPENDENCY] k8s.io/legacy-cloud-providers is now updated to v1.22.0. (gardener-attic/cloud-provider-azure@c3bd645)

🏃 Others

• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.18.20. (gardener-attic/cloud-provider-azure#8, @ialidzhikov)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.19.14. (gardener-attic/cloud-provider-azure#7, @vpnachev)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.20.10. (gardener-attic/cloud-provider-azure#6, @vpnachev)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.21.4. (gardener-attic/cloud-provider-azure#5, @vpnachev)
• [DEVELOPER] The alpine version has been updated to v3.13.5. (gardener-attic/cloud-provider-azure@c3bd645)
• [DEVELOPER] The Golang version has been updated to v1.16.7. (gardener-attic/cloud-provider-azure@c3bd645)

[terraformer]

⚠️ Breaking Changes

• [DEVELOPER] Once the azurerm provider plugin is updated from v2.36.0 to v2.68.0 the skip_provider_registration flag in the provider section need to be set to true. (gardener/terraformer#99, @dkistner)

🏃 Others

• [OPERATOR] The terraform azurerm provider plugin is updated from v2.36.0 to v2.68.0. (gardener/terraformer#99, @dkistner)

v1.21.2

[cloud-provider-azure]

🏃 Others

• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.19.14. (gardener-attic/cloud-provider-azure#7, @vpnachev)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.20.10. (gardener-attic/cloud-provider-azure#6, @vpnachev)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.21.2
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.21.2

v1.21.1

[cloud-provider-azure]

🏃 Others

• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.21.4. (gardener-attic/cloud-provider-azure#5, @vpnachev)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.21.1
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.21.1

v1.21.0

[gardener-extension-provider-azure]

⚠️ Breaking Changes

• [OPERATOR] The default leader election resource lock of gardener-extension-provider-azure has been changed from configmapsleases to leases. (#334, @ialidzhikov)
  • Please make sure, that you had at least gardener-extension-provider-azure@v1.18 running before upgrading to v1.21.0, so that it has successfully required leadership with the hybrid resource lock (configmapsleases) at least once.
• [OPERATOR] Replace infrastructure's terraform helm chart with native go templates. (#295, @kon-angelo)
  • ⚠️ Before upgrading to >=v1.21.0 version of gardener/gardener-extension-provider-azure make sure to upgrade to to a version >=v1.15.2 if you are using a version older than that to avoid breaking changes due to the upgrade to Terraform azurerm v2.

✨ New Features

• [USER] The following images are updated (see CHANGELOG for more details): (#324, @tareqhs)
  • k8s.gcr.io/sig-storage/csi-snapshotter: v2.1.5 -> v3.0.3
  • k8s.gcr.io/sig-storage/snapshot-controller: v2.1.5 -> v3.0.3
• [OPERATOR] It is now possible to specify the leader election resource lock via the chart value leaderElection.resourceLock (defaults to leases). (#334, @ialidzhikov)
• [OPERATOR] The existing ValidatingWebhookConfiguration of admission-azure for Shoot validation does now validate also the Shoot secret. admission-azure does now feature also a new webhook that prevents Shoot secret to be updated with invalid keys. (#301, @vpnachev)

🐛 Bug Fixes

• [USER] Do not trigger a node rollout when switching from CRI.Name==nil to CRI.Name==docker. (#341, @BeckerMax)
• [OPERATOR] An issue causing dynamic provisioning with the gardener.cloud-fast StorageClass to fail on Kubernetes v1.21 (or any other version with CSI enabled) is now fixed. (#332, @ialidzhikov)

🏃 Others

• [USER] It is now disallowed the tenant or subscription ID to be changed for azure cloud provider secret when it is still used by at least one shoot cluster. (#329, @vpnachev)
• [USER] The following image is updated (see CHANGELOG for more details): (#327, @ialidzhikov)
  • mcr.microsoft.com/k8s/csi/azuredisk-csi: v1.1.1 -> v1.2.0
• [USER] The following image is updated (see CHANGELOG for more details): (#318, @ialidzhikov)
  • k8s.gcr.io/sig-storage/livenessprobe: v2.2.0 -> v2.3.0
• [OPERATOR] When creating or updating shoots, any Kubernetes feature gates mentioned are validated against the Kubernetes version. If any feature gates are unknown or not supported in the Kubernetes version, the validation fails. (#323, @stoyanr)
• [OPERATOR] Validation of Azure cloud provider secrets is enhanced to reject subscriptionID, tenantID, and clientID that are not valid UUIDs, and clientSecret that contain leading or trailing new lines. (#321, @stoyanr)

[cloud-provider-azure]

🏃 Others

• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.19.11. (gardener-attic/cloud-provider-azure@3fc3755)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.20.7. (gardener-attic/cloud-provider-azure@0187dc8)
• [OPERATOR] k8s.io/legacy-cloud-providers is now updated to v0.21.1. (gardener-attic/cloud-provider-azure@783a563)

[machine-controller-manager]

⚠️ Breaking Changes

• [OPERATOR] Draining of pods with PVs (Persistent Volume) now waits for re-attachment of PV on a different node when volumeAttachments support is enabled on the cluster. Else it falls back to the default PV reattachment timeout value configured. The default value is 90s and this can be overwritten via the machine-pv-reattach-timeout flag. Please enable permissions to allow listing of volumeAttachments resource while importing these changes. (gardener/machine-controller-manager#608, @prashanth26)

✨ New Features

• [USER] Increase default concurrent object syncs to 50 to allow more concurrent reconciles to occur. (gardener/machine-controller-manager#629, @prashanth26)
• [USER] Machine rollouts are now more as desired with the number of replicas always maintained to desired + maxSurge. Earlier machines in termination were left out of this calculation but now is considered with this change. (gardener/machine-controller-manager#627, @prashanth26)
• [USER] Skip node drain on ReadOnlyFileSystem condition (gardener/machine-controller-manager#605, @himanshu-kun)
• [OPERATOR] Finalizers will be added to the MachineClass which is used by at least one machine. Machines whose backing MachineClass does not have finalizers shall not be reconciled. (gardener/machine-controller-manager#593, @AxiomSamarth)
• [OPERATOR] Improved log details to include node name and provider-ID in addition to existing machine name (gardener/machine-controller-manager#607, @himanshu-kun)
• [DEVELOPER] Replace integration test with unit test to test the functionality to freeze MachineSet (gardener/machine-controller-manager#620, @AxiomSamarth)

🐛 Bug Fixes

• [OPERATOR] Avoids blocking of drain call when the buffer is full for the volumeAttachmentHandlers. (gardener/machine-controller-manager#627, @prashanth26)
• [OPERATOR] Fix panic when machineClass secretRef isn't found. (gardener/machine-controller-manager#609, @jsravn)
• [DEVELOPER] Test framework now fetches secrets from the correct (control) APIServer while running tests. (gardener/machine-controller-manager#617, @himanshu-kun)
• [DEVELOPER] Adds finalizers on machines that are adopted by the machine controller. Without this change, it causes issues while migrating machine objects between clusters. (gardener/machine-controller-manager#611, @prashanth26)

🏃 Others

• [OPERATOR] Nodes attached to the cluster without MCM support are now annotated with "node.machine.sapcloud.io/notManagedByMCM": "1". This is then ignored by the MCM for further processing. (gardener/machine-controller-manager#612, @himanshu-kun)

[machine-controller-manager-provider-azure]

🐛 Bug Fixes

• [USER] Fix an issue where the availability set information was not transported properly on the driver. It also added support for VMO machines. (gardener/machine-controller-manager-provider-azure#24, @kon-angelo)
• [OPERATOR] Checks for NICs and Disks while listing VMs for orphan resource collection. (gardener/machine-controller-manager-provider-azure#26, @AxiomSamarth)
• [DEVELOPER] A shared data structure was causing race conditions leading to VM creation with wrong configurations. (gardener/machine-controller-manager-provider-azure#31, @AxiomSamarth)
• [DEVELOPER] A shared data structure was causing race conditions leading to VM creation with wrong configurations. (gardener/machine-controller-manager-provider-azure#32, @prashanth26)

🏃 Others

• [OPERATOR] Regression: Clean up wanted test logs (gardener/machine-controller-manager-provider-azure#28, @AxiomSamarth)
• [DEVELOPER] Revendors the latest MCM libraries @ v0.40.0 (gardener/machine-controller-manager-provider-azure#37, @prashanth26)
• [DEPENDENCY] Revendors MCM dependent libraries for v0.39.0 version. (gardener/machine-controller-manager-provider-azure#30, @prashanth26)

[remedy-controller]

🐛 Bug Fixes

• [USER] It is now possible to specify the minimum frequency at which PublicIPAddress and VirtualMachine resources will be reconciled via the SyncPeriod options. By default, these are set to 10 hours and 2 hours respectively. (gardener/remedy-controller#39, @stoyanr)

[terraformer]

🏃 Others

• [OPERATOR] The following terraform provider plugins are updated: (gardener/terraformer#98, @ialidzhikov)
  • hashicorp/terraform-provider-aws: 3.18.0 -> 3.32.0
• [OPERATOR] The following terraform provider plugin is updated: (gardener/terraformer#96, @minchaow)
  • aliyun/terraform-provider-alicloud: 1.124.0 -> 1.124.2
• [OPERATOR] The following terraform provider plugin is updated: (gardener/terraformer#95, @minchaow)
  • aliyun/terraform-provider-alicloud: 1.121.2 -> 1.124.0
• [OPERATOR] The terraform version for the alicloud, all, aws, azure, gcp, openstack, slim images is updated: (gardener/terraformer#94, @ialidzhikov)
  • hashicorp/terraform: 0.12.29 -> 0.12.31

v1.20.2

[gardener-extension-provider-azure]

🐛 Bug Fixes

• [OPERATOR] An issue causing dynamic provisioning with the gardener.cloud-fast StorageClass to fail on Kubernetes v1.21 (or any other version with CSI enabled) is now fixed. (#333, @ialidzhikov)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.20.2
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.20.2

v1.20.1

[machine-controller-manager]

✨ New Features

• [USER] Skip node drain on ReadOnlyFileSystem condition (gardener/machine-controller-manager#605, @himanshu-kun)
• [OPERATOR] Improved log details to include node name and provider-ID in addition to existing machine name (gardener/machine-controller-manager#607, @himanshu-kun)

🐛 Bug Fixes

• [OPERATOR] Fix panic when machineClass secretRef isn't found. (gardener/machine-controller-manager#609, @jsravn)
• [DEVELOPER] Adds finalizers on machines that are adopted by the machine controller. Without this change, it causes issues while migrating machine objects between clusters. (gardener/machine-controller-manager#611, @prashanth26)

[machine-controller-manager-provider-azure]

🐛 Bug Fixes

• [USER] Fix an issue where the availability set information was not transported properly on the driver. It also added support for VMO machines. (gardener/machine-controller-manager-provider-azure#24, @kon-angelo)
• [OPERATOR] Checks for NICs and Disks while listing VMs for orphan resource collection. (gardener/machine-controller-manager-provider-azure#26, @AxiomSamarth)
• [DEVELOPER] A shared data structure was causing race conditions leading to VM creation with wrong configurations. (gardener/machine-controller-manager-provider-azure#32, @prashanth26)

🏃 Others

• [OPERATOR] Regression: Clean up wanted test logs (gardener/machine-controller-manager-provider-azure#28, @AxiomSamarth)
• [DEPENDENCY] Revendors MCM dependent libraries for v0.39.0 version. (gardener/machine-controller-manager-provider-azure#30, @prashanth26)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.20.1
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.20.1

v1.20.0

[gardener-extension-provider-azure]

⚠️ Breaking Changes

• [USER] The Azure extension does now support shoot clusters with Kubernetes version 1.21. You should consider the Kubernetes release notes before upgrading to 1.21. Please note that both the Azure Disk CSI driver and Azure File CSI driver will be used for 1.21 shoots. They are compatible with the legacy volume provisioners, however, you might want to update your storage classes and volume handling accordingly. Please find more information about CSI in the official Kubernetes documentation. (#280, @rfranzke)
• [USER] Extension resource configs (ControlPlaneConfigs, WorkerConfig) are now deserialized in "strict" mode. This means that deserializing resources with fields that are not allowed by the API schema will result in errors. Shoots containing such resources will fail with an appropriate error until you manually update the shoot to make sure any extension resource configs contained in it are valid. (#272, @stoyanr)

🐛 Bug Fixes

• [USER] A bug is fixed which blocked the deletion of vmo based Azure cluster in case the vmo resource on Azure cannot be created. (#276, @dkistner)
• [OPERATOR] An issue causing CSI PV to do not have set spec.csi.fsType is now fixed. The csi-provisioner is now started with --default-fstype=ext4 which is the default fstype to be used when there is no fstype specified in the StorageClass. (#299, @ialidzhikov)
• [OPERATOR] A new service allow-tcp-egress is created in the shoot cluster to configure TCP egress traffic when using the reversed cluster vpn feature. (#292, @kon-angelo)

🏃 Others

• [USER] The following images are updated: (#275, @ialidzhikov)
  • k8s.gcr.io/sig-storage/csi-snapshotter: v2.1.4 -> v2.1.5
  • k8s.gcr.io/sig-storage/snapshot-controller: v2.1.4 -> v2.1.5
  • k8s.gcr.io/sig-storage/livenessprobe: v2.0.0 -> v2.2.0
• [OPERATOR] The few CSI sidecar containers that didn't specify any resource requests and limits do now specify appropriate requests and limits. (#277, @ialidzhikov)
• [OPERATOR] ⚠️ Before upgrading your gardener/gardener-extension-provider-azure to >= v1.20.0, please upgrade your gardener/gardener component version to >= v1.14.0 to avoid breaking of clusters that are using the scale from/to zero feature (clusters that allowing scaling from/to 0 worker pools). If used with an older gardener/gardener version, this would lead to failure of clusters making use of this feature. (#257, @AxiomSamarth)

[cloud-provider-azure]

✨ New Features

• [DEPENDENCY] k8s.io/legacy-cloud-providers is now updated to v0.21.0. (gardener-attic/cloud-provider-azure@70f2ce3)

🏃 Others

• [DEVELOPER] The alpine version has been updated to v3.13.4. (gardener-attic/cloud-provider-azure@32407b4)
• [DEVELOPER] The Golang version has been updated to v1.16.3. (gardener-attic/cloud-provider-azure@32407b4)

[machine-controller-manager-provider-azure]

⚠️ Breaking Changes

• [DEVELOPER] The release tags from now are prefixed with v. (gardener/machine-controller-manager-provider-azure#18, @ialidzhikov)

🏃 Others

• [USER] Fix an issue where the availability set information was not transported properly on the driver. It also adds support for VMO machines. (gardener/machine-controller-manager-provider-azure#25, @kon-angelo)
• [OPERATOR] Regression: Clean up wanted test logs (gardener/machine-controller-manager-provider-azure#29, @AxiomSamarth)
• [OPERATOR] Bug fix: Checks for NICs and Disks while listing VMs for orphan resource collection. (gardener/machine-controller-manager-provider-azure#27, @prashanth26)

📰 Noteworthy

• [USER] Fixes regressions while fetching Azure CSI PersistentVolumes during the eviction of Pods with PersistentVolumes. (gardener/machine-controller-manager-provider-azure#23, @prashanth26)

[terraformer]

✨ New Features

• [OPERATOR] Terraformer now copies Terraform's error outputs to /terraform-termination-log to make it available in the containers termination message for better analyzing and more readable error messages (e.g. in the Shoot status). (gardener/terraformer#93, @timebertt)

🐛 Bug Fixes

• [OPERATOR] The aws provider has been downgraded from 3.32.0 to 3.18.0 due to issue with additionally required permission for the AWS accounts. (gardener/terraformer#87, @vpnachev)

🏃 Others

• [USER] Terraform provider of Alicloud is upgraded to 1.121.2. (gardener/terraformer#91, @minchaow)
• [OPERATOR] The following terraform provider plugins are updated: (gardener/terraformer#88, @ialidzhikov)
  • hashicorp/terraform-provider-google: 3.59.0 -> 3.62.0
  • hashicorp/terraform-provider-google-beta: 3.59.0 -> 3.62.0
• [DEVELOPER] The golang has been updated to 1.16.2, the alpine has been updated to 3.13.2. (gardener/terraformer#85, @vpnachev)

Docker Images

gardener-extension-provider-azure: eu.gcr.io/gardener-project/gardener/extensions/provider-azure:v1.20.0
gardener-extension-admission-azure: eu.gcr.io/gardener-project/gardener/extensions/admission-azure:v1.20.0