Skip to content

Latest commit

 

History

History
737 lines (587 loc) · 78.7 KB

CHANGELOG-1.21.md

File metadata and controls

737 lines (587 loc) · 78.7 KB

v1.21.0-beta.0

Downloads for v1.21.0-beta.0

Source Code

filename sha512 hash
kubernetes.tar.gz 69b73a03b70b0ed006e9fef3f5b9bc68f0eb8dc40db6cc04777c03a2cb83a008c783012ca186b1c48357fb192403dbcf6960f120924785e2076e215b9012d546
kubernetes-src.tar.gz 9620fb6d37634271bdd423c09f33f3bd29e74298aa82c47dffc8cb6bd2ff44fa8987a53c53bc529db4ca96ec41503aa81cc8d0c3ac106f3b06c4720de933a8e6

Client binaries

filename sha512 hash
kubernetes-client-darwin-amd64.tar.gz 2a6f3fcd6b571f5ccde56b91e6e179a01899244be496dae16a2a16e0405c9437b75c6dc853b56f9a4876a7c0a60ec624ccd28400bf8fb960258263172f6860ba
kubernetes-client-linux-386.tar.gz 78fe9ad9f9a9bc043293327223f0038a2c087ca65e87187a6dcae7a24aef9565fe498d295a4639b0b90524469a04930022fcecd815d0afc742eb87ddd8eb7ef5
kubernetes-client-linux-amd64.tar.gz c025f5e5bd132355e7dd1296cf2ec752264e7f754c4d95fc34b076bd75bef2f571d30872bcb3d138ce95c592111353d275a80eb31f82c07000874b4c56282dbd
kubernetes-client-linux-arm.tar.gz 9975cd2f08fbc202575fb15ba6fc51dab23155ca4d294ebb48516a81efa51f58bab3a87d41c865103756189b554c020371d729ad42880ba788f25047ffc46910
kubernetes-client-linux-arm64.tar.gz 56a6836e24471e42e9d9a8488453f2d55598d70c8aca0a307d5116139c930c25c469fd0d1ab5060fbe88dad75a9b5209a08dc11d644af5f3ebebfbcb6c16266c
kubernetes-client-linux-ppc64le.tar.gz b6a6cc9baad0ad85ed079ee80e6d6acc905095cfb440998bbc0f553b94fa80077bd58b8692754de477517663d51161705e6e89a1b6d04aa74819800db3517722
kubernetes-client-linux-s390x.tar.gz 7b743481b340f510bf9ae28ea8ea91150aa1e8c37fe104b66d7b3aff62f5e6db3c590d2c13d14dbb5c928de31c7613372def2496075853611d10d6b5fa5b60bd
kubernetes-client-windows-386.tar.gz df06c7a524ce84c1f8d7836aa960c550c88dbca0ec4854df4dd0a85b3c84b8ecbc41b54e8c4669ce28ac670659ff0fad795deb1bc539f3c3b3aa885381265f5a
kubernetes-client-windows-amd64.tar.gz 4568497b684564f2a94fbea6cbfd778b891231470d9a6956c3b7a3268643d13b855c0fc5ebea5f769300cc0c7719c2c331c387f468816f182f63e515adeaa7a0

Server binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz 42883cca2d312153baf693fc6024a295359a421e74fd70eefc927413be4e0353debe634e7cca6b9a8f7d8a0cee3717e03ba5d29a306e93139b1c2f3027535a6d
kubernetes-server-linux-arm.tar.gz e0042215e84c769ba4fc4d159ccf67b2c4a26206bfffb0ec5152723dc813ff9c1426aa0e9b963d7bfa2efb266ca43561b596b459152882ebb42102ccf60bd8eb
kubernetes-server-linux-arm64.tar.gz bfad29d43e14152cb9bc7c4df6aa77929c6eca64a294bb832215bdba9fa0ee2195a2b709c0267dc7426bb371b547ee80bb8461a8c678c9bffa0819aa7db96289
kubernetes-server-linux-ppc64le.tar.gz ca67674c01c6cebdc8160c85b449eab1a23bb0557418665246e0208543fa2eaaf97679685c7b49bee3a4300904c0399c3d762ae34dc3e279fd69ce792c4b07ff
kubernetes-server-linux-s390x.tar.gz 285352b628ec754b01b8ad4ef1427223a142d58ebcb46f6861df14d68643133b32330460b213b1ba5bc5362ff2b6dacd8e0c2d20cce6e760fa1954af8a60df8b

Node binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz d92d9b30e7e44134a0cd9db4c01924d365991ea16b3131200b02a82cff89c8701f618cd90e7f1c65427bd4bb5f78b10d540b2262de2c143b401fa44e5b25627b
kubernetes-node-linux-arm.tar.gz 551092f23c27fdea4bb2d0547f6075892534892a96fc2be7786f82b58c93bffdb5e1c20f8f11beb8bed46c24f36d4c18ec5ac9755435489efa28e6ae775739bd
kubernetes-node-linux-arm64.tar.gz 26ae7f4163e527349b8818ee38b9ee062314ab417f307afa49c146df8f5a2bd689509b128bd4a1efd3896fd89571149a9955ada91f8ca0c2f599cd863d613c86
kubernetes-node-linux-ppc64le.tar.gz 821fa953f6cebc69d2d481e489f3e90899813d20e2eefbabbcadd019d004108e7540f741fabe60e8e7c6adbb1053ac97898bbdddec3ca19f34a71aa3312e0d4e
kubernetes-node-linux-s390x.tar.gz 22197d4f66205d5aa9de83dfddcc4f2bb3195fd7067cdb5c21e61dbeae217bc112fb7ecff8a539579b60ad92298c2b4c87b9b7c7e6ec1ee1ffa0c6e4bc4412c1
kubernetes-node-windows-amd64.tar.gz 7e22e0d9603562a04dee16a513579f06b1ff6354d97d669bd68f8777ec7f89f6ef027fb23ab0445d7bba0bb689352f0cc748ce90e3f597c6ebe495464a96b860

Changelog since v1.21.0-alpha.3

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • The metric storage_operation_errors_total is not removed, but is marked deprecated, and the metric storage_operation_status_count is marked deprecated. In both cases the storage_operation_duration_seconds metric can be used to recover equivalent counts (using status=fail-unknown in the case of storage_operations_errors_total). (#99045, @mattcary) [SIG Instrumentation and Storage]

Changes by Kind

Deprecation

  • The batch/v2alpha1 CronJob type definitions and clients are deprecated and removed. (#96987, @soltysh) [SIG API Machinery, Apps, CLI and Testing]

API Change

  • Cluster admins can now turn off /debug/pprof and /debug/flags/v endpoint in kubelet by setting enableProfilingHandler and enableDebugFlagsHandler to false in their kubelet configuration file. enableProfilingHandler and enableDebugFlagsHandler can be set to true only when enableDebuggingHandlers is also set to true. (#98458, @SaranBalaji90) [SIG Node]
  • The BoundServiceAccountTokenVolume feature has been promoted to beta, and enabled by default.
    • This changes the tokens provided to containers at /var/run/secrets/kubernetes.io/serviceaccount/token to be time-limited, auto-refreshed, and invalidated when the containing pod is deleted.
    • Clients should reload the token from disk periodically (once per minute is recommended) to ensure they continue to use a valid token. k8s.io/client-go version v11.0.0+ and v0.15.0+ reload tokens automatically.
    • By default, injected tokens are given an extended lifetime so they remain valid even after a new refreshed token is provided. The metric serviceaccount_stale_tokens_total can be used to monitor for workloads that are depending on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If that metric indicates no existing workloads are depending on extended lifetimes, injected token lifetime can be shortened to 1 hour by starting kube-apiserver with --service-account-extend-token-expiration=false. (#95667, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]

Feature

  • A new histogram metric to track the time it took to delete a job by the ttl-after-finished controller (#98676, @ahg-g) [SIG Apps and Instrumentation]
  • AWS cloudprovider supports auto-discovering subnets without any kubernetes.io/cluster/ tags. It also supports additional service annotation service.beta.kubernetes.io/aws-load-balancer-subnets to manually configure the subnets. (#97431, @kishorj) [SIG Cloud Provider]
  • Add --permit-address-sharing flag to kube-apiserver to listen with SO_REUSEADDR. While allowing to listen on wildcard IPs like 0.0.0.0 and specific IPs in parallel, it avoid waiting for the kernel to release socket in TIME_WAIT state, and hence, considably reducing kube-apiserver restart times under certain conditions. (#93861, @sttts) [SIG API Machinery]
  • Add csi_operations_seconds metric on kubelet that exposes CSI operations duration and status for node CSI operations. (#98979, @Jiawei0227) [SIG Instrumentation and Storage]
  • Add migrated field into storage_operation_duration_seconds metric (#99050, @Jiawei0227) [SIG Apps, Instrumentation and Storage]
  • Add bash-completion for comma separated list on kubectl get (#98301, @phil9909) [SIG CLI]
  • Added support for installing arm64 node artifacts. (#99242, @liu-cong) [SIG Cloud Provider]
  • Feature gate RootCAConfigMap is graduated to GA in 1.21 and will be removed in 1.22. (#98033, @zshihang) [SIG API Machinery and Auth]
  • Kubeadm: during "init" and "join" perform preflight validation on the host / node name and throw warnings if a name is not compliant (#99194, @pacoxu) [SIG Cluster Lifecycle]
  • Kubectl: kubectl get will omit managed fields by default now. Users could set --show-managed-fields to true to show managedFields when the output format is either json or yaml. (#96878, @knight42) [SIG CLI and Testing]
  • Metrics can now be disabled explicitly via a command line flag (i.e. '--disabled-metrics=bad_metric1,bad_metric2') (#99217, @logicalhan) [SIG API Machinery, Cluster Lifecycle and Instrumentation]
  • TTLAfterFinished is now beta and enabled by default (#98678, @ahg-g) [SIG Apps and Auth]
  • The RunAsGroup feature has been promoted to GA in this release. (#94641, @krmayankk) [SIG Auth and Node]
  • Turn CronJobControllerV2 on by default. (#98878, @soltysh) [SIG Apps]
  • UDP protocol support for Agnhost connect subcommand (#98639, @knabben) [SIG Testing]
  • Upgrades IPv6Dualstack to Beta and turns it on by default. Clusters new and existing will not be affected until user starting adding secondary pod and service cidrs cli flags as described here: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/563-dual-stack (#98969, @khenidak) [SIG API Machinery, Apps, Cloud Provider, Network and Node]

Documentation

  • Fix ALPHA stability level reference link (#98641, @Jeffwan) [SIG Auth, Cloud Provider, Instrumentation and Storage]

Failing Test

  • Escape the special characters like [, ] and that exist in vsphere windows path (#98830, @liyanhui1228) [SIG Storage and Windows]
  • Kube-proxy: fix a bug on UDP NodePort Services where stale conntrack entries may blackhole the traffic directed to the NodePort. (#98305, @aojea) [SIG Network]

Bug or Regression

  • Add missing --kube-api-content-type in kubemark hollow template (#98911, @Jeffwan) [SIG Scalability and Testing]
  • Avoid duplicate error messages when runing kubectl edit quota (#98201, @pacoxu) [SIG API Machinery and Apps]
  • Cleanup subnet in frontend IP configs to prevent huge subnet request bodies in some scenarios. (#98133, @nilo19) [SIG Cloud Provider]
  • Fix errors when accessing Windows container stats for Dockershim (#98510, @jsturtevant) [SIG Node and Windows]
  • Fixes spurious errors about IPv6 in kube-proxy logs on nodes with IPv6 disabled. (#99127, @danwinship) [SIG Network and Node]
  • In the method that ensures that the docker and containerd are in the correct containers with the proper OOM score set up, fixed the bug of identifying containerd process. (#97888, @pacoxu) [SIG Node]
  • Kubelet now cleans up orphaned volume directories automatically (#95301, @lorenz) [SIG Node and Storage]
  • When dynamically provisioning Azure File volumes for a premium account, the requested size will be set to 100GB if the request is initially lower than this value to accommodate Azure File requirements. (#99122, @huffmanca) [SIG Cloud Provider and Storage]

Other (Cleanup or Flake)

  • APIs for kubelet annotations and labels from k8s.io/kubernetes/pkg/kubelet/apis are now available under k8s.io/kubelet/pkg/apis/ (#98931, @michaelbeaumont) [SIG Apps, Auth and Node]
  • Migrate pkg/kubelet/(pod, pleg) to structured logging (#98990, @gjkim42) [SIG Instrumentation and Node]
  • Migrate pkg/kubelet/nodestatus to structured logging (#99001, @QiWang19) [SIG Node]
  • Migrate pkg/kubelet/server logs to structured logging (#98643, @chenyw1990) [SIG Node]
  • Migrate proxy/winkernel/proxier.go logs to structured logging (#98001, @JornShen) [SIG Network and Windows]
  • Migrate scheduling_queue.go to structured logging (#98358, @tanjing2020) [SIG Scheduling]
  • Several flags related to the deprecated dockershim which are present in the kubelet command line are now deprecated. (#98730, @dims) [SIG Node]
  • The deprecated feature gates CSIDriverRegistry, BlockVolume and CSIBlockVolume are now unconditionally enabled and can no longer be specified in component invocations. (#98021, @gavinfish) [SIG Storage]

Dependencies

Added

Nothing has changed.

Changed

  • sigs.k8s.io/structured-merge-diff/v4: v4.0.2 → v4.0.3

Removed

Nothing has changed.

v1.21.0-alpha.3

Downloads for v1.21.0-alpha.3

Source Code

filename sha512 hash
kubernetes.tar.gz 704ec916a1dbd134c54184d2652671f80ae09274f9d23dbbed312944ebeccbc173e2e6b6949b38bdbbfdaf8aa032844deead5efeda1b3150f9751386d9184bc8
kubernetes-src.tar.gz 57db9e7560cfc9c10e7059cb5faf9c4bd5eb8f9b7964f44f000a417021cf80873184b774e7c66c80d4aba84c14080c6bc335618db3d2e5f276436ae065e25408

Client binaries

filename sha512 hash
kubernetes-client-darwin-amd64.tar.gz e2706efda92d5cf4f8b69503bb2f7703a8754407eff7f199bb77847838070e720e5f572126c14daa4c0c03b59bb1a63c1dfdeb6e936a40eff1d5497e871e3409
kubernetes-client-linux-386.tar.gz 007bb23c576356ed0890bdfd25a0f98d552599e0ffec19fb982591183c7c1f216d8a3ffa3abf15216be12ae5c4b91fdcd48a7306a2d26b007b86a6abd553fc61
kubernetes-client-linux-amd64.tar.gz 39504b0c610348beba60e8866fff265bad58034f74504951cd894c151a248db718d10f77ebc83f2c38b2d517f8513a46325b38889eefa261ca6dbffeceba50ff
kubernetes-client-linux-arm.tar.gz 30bc2c40d0c759365422ad1651a6fb35909be771f463c5b971caf401f9209525d05256ab70c807e88628dd357c2896745eecf13eda0b748464da97d0a5ef2066
kubernetes-client-linux-arm64.tar.gz 085cdf574dc8fd33ece667130b8c45830b522a07860e03a2384283b1adea73a9652ef3dfaa566e69ee00aea1a6461608814b3ce7a3f703e4a934304f7ae12f97
kubernetes-client-linux-ppc64le.tar.gz b34b845037d83ea7b3e2d80a9ede4f889b71b17b93b1445f0d936a36e98c13ed6ada125630a68d9243a5fcd311ee37cdcc0c05da484da8488ea5060bc529dbfc
kubernetes-client-linux-s390x.tar.gz c4758adc7a404b776556efaa79655db2a70777c562145d6ea6887f3335988367a0c2fcd4383e469340f2a768b22e786951de212805ca1cb91104d41c21e0c9ce
kubernetes-client-windows-386.tar.gz f51edc79702bbd1d9cb3a672852a405e11b20feeab64c5411a7e85c9af304960663eb6b23ef96e0f8c44a722fecf58cb6d700ea2c42c05b3269d8efd5ad803f2
kubernetes-client-windows-amd64.tar.gz 6a3507ce4ac40a0dc7e4720538863fa15f8faf025085a032f34b8fa0f6fa4e8c26849baf649b5b32829b9182e04f82721b13950d31cf218c35be6bf1c05d6abf

Server binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz 19181d162dfb0b30236e2bf1111000e037eece87c037ca2b24622ca94cb88db86aa4da4ca533522518b209bc9983bbfd6b880a7898e0da96b33f3f6c4690539b
kubernetes-server-linux-arm.tar.gz 42a02f9e08a78ad5da6e5fa1ab12bf1e3c967c472fdbdadbd8746586da74dc8093682ba8513ff2a5301393c47ee9021b860e88ada56b13da386ef485708e46ca
kubernetes-server-linux-arm64.tar.gz 3c8ba8eb02f70061689bd7fab7813542005efe2edc6cfc6b7aecd03ffedf0b81819ad91d69fff588e83023d595eefbfe636aa55e1856add8733bf42fff3c748f
kubernetes-server-linux-ppc64le.tar.gz cd9e6537450411c39a06fd0b5819db3d16b668d403fb3627ec32c0e32dd1c4860e942934578ca0e1d1b8e6f21f450ff81e37e0cd46ff5c5faf7847ab074aefc5
kubernetes-server-linux-s390x.tar.gz ada3f65e53bc0e0c0229694dd48c425388089d6d77111a62476d1b08f6ad1d8ab3d60b9ed7d95ac1b42c2c6be8dc0618f40679717160769743c43583d8452362

Node binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz ae0fec6aa59e49624b55d9a11c12fdf717ddfe04bdfd4f69965d03004a34e52ee4a3e83f7b61d0c6a86f43b72c99f3decb195b39ae529ef30526d18ec5f58f83
kubernetes-node-linux-arm.tar.gz 9a48c140ab53b7ed8ecec6903988a1a474efc16d2538e5974bc9a12f0c9190be78c4f9e326bf4e982d0b7045a80b99dd0fda7e9b650663be5b89bfd991596746
kubernetes-node-linux-arm64.tar.gz 6912adbc9300344bea470d6435f7b387bfce59767078c11728ce59faf47cd3f72b41b9604fcc5cda45e9816fe939fbe2fb33e52a773e6ff2dfa9a615b4df6141
kubernetes-node-linux-ppc64le.tar.gz d66dccfe3e6ed6d81567c70703f15375a53992b3a5e2814b98c32e581b861ad95912e03ed2562415d087624c008038bb4a816611fa255442ae752968ea15856b
kubernetes-node-linux-s390x.tar.gz ad8c69a28f1fbafa3f1cb54909bfd3fc22b104bed63d7ca2b296208c9d43eb5f2943a0ff267da4c185186cdd9f7f77b315cd7f5f1bf9858c0bf42eceb9ac3c58
kubernetes-node-windows-amd64.tar.gz 91d723aa848a9cb028f5bcb41090ca346fb973961521d025c4399164de2c8029b57ca2c4daca560d3c782c05265d2eb0edb0abcce6f23d3efbecf2316a54d650

Changelog since v1.21.0-alpha.2

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Newly provisioned PVs by gce-pd will no longer have the beta FailureDomain label. gce-pd volume plugin will start to have GA topology label instead. (#98700, @Jiawei0227) [SIG Cloud Provider, Storage and Testing]
  • Remove alpha CSIMigrationXXComplete flag and add alpha InTreePluginXXUnregister flag. Deprecate CSIMigrationvSphereComplete flag and it will be removed in 1.22. (#98243, @Jiawei0227) [SIG Node and Storage]

Changes by Kind

API Change

  • Adds support for portRange / EndPort in Network Policy (#97058, @rikatz) [SIG Apps and Network]
  • Fixes using server-side apply with APIService resources (#98576, @kevindelgado) [SIG API Machinery, Apps and Testing]
  • Kubernetes is now built using go1.15.7 (#98363, @cpanato) [SIG Cloud Provider, Instrumentation, Node, Release and Testing]
  • Scheduler extender filter interface now can report unresolvable failed nodes in the new field FailedAndUnresolvableNodes of ExtenderFilterResult struct. Nodes in this map will be skipped in the preemption phase. (#92866, @cofyc) [SIG Scheduling]

Feature

  • A lease can only attach up to 10k objects. (#98257, @lingsamuel) [SIG API Machinery]
  • Add ignore-errors flag for drain, support none-break drain in group (#98203, @yuzhiquan) [SIG CLI]
  • Base-images: Update to debian-iptables:buster-v1.4.0
    • Uses iptables 1.8.5
    • base-images: Update to debian-base:buster-v1.3.0
    • cluster/images/etcd: Build etcd:3.4.13-2 image
  • Export NewDebuggingRoundTripper function and DebugLevel options in the k8s.io/client-go/transport package. (#98324, @atosatto) [SIG API Machinery]
  • Kubectl wait ensures that observedGeneration >= generation if applicable (#97408, @KnicKnic) [SIG CLI]
  • Kubernetes is now built using go1.15.8 (#98834, @cpanato) [SIG Cloud Provider, Instrumentation, Release and Testing]
  • New admission controller "denyserviceexternalips" is available. Clusters which do not *need- the Service "externalIPs" feature should enable this controller and be more secure. (#97395, @thockin) [SIG API Machinery]
  • Overall, enable the feature of PreferNominatedNode will improve the performance of scheduling where preemption might frequently happen, but in theory, enable the feature of PreferNominatedNode, the pod might not be scheduled to the best candidate node in the cluster. (#93179, @chendave) [SIG Scheduling and Testing]
  • Pause image upgraded to 3.4.1 in kubelet and kubeadm for both Linux and Windows. (#98205, @pacoxu) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Testing and Windows]
  • The ServiceAccountIssuerDiscovery feature has graduated to GA, and is unconditionally enabled. The ServiceAccountIssuerDiscovery feature-gate will be removed in 1.22. (#98553, @mtaufen) [SIG API Machinery, Auth and Testing]

Documentation

  • Feat: azure file migration go beta in 1.21. Feature gates CSIMigration to Beta (on by default) and CSIMigrationAzureFile to Beta (off by default since it requires installation of the AzureFile CSI Driver) The in-tree AzureFile plugin "kubernetes.io/azure-file" is now deprecated and will be removed in 1.23. Users should enable CSIMigration + CSIMigrationAzureFile features and install the AzureFile CSI Driver (https://github.com/kubernetes-sigs/azurefile-csi-driver) to avoid disruption to existing Pod and PVC objects at that time. Users should start using the AzureFile CSI Driver directly for any new volumes. (#96293, @andyzhangx) [SIG Cloud Provider]

Failing Test

  • Kubelet: the HostPort implementation in dockershim was not taking into consideration the HostIP field, causing that the same HostPort can not be used with different IP addresses. This bug causes the conformance test "HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol" to fail. (#98755, @aojea) [SIG Cloud Provider, Network and Node]

Bug or Regression

  • Fix NPE in ephemeral storage eviction (#98261, @wzshiming) [SIG Node]
  • Fixed a bug that on k8s nodes, when the policy of INPUT chain in filter table is not ACCEPT, healthcheck nodeport would not work. Added iptables rules to allow healthcheck nodeport traffic. (#97824, @hanlins) [SIG Network]
  • Fixed kube-proxy container image architecture for non amd64 images. (#98526, @saschagrunert) [SIG API Machinery, Release and Testing]
  • Fixed provisioning of Cinder volumes migrated to CSI when StorageClass with AllowedTopologies was used. (#98311, @jsafrane) [SIG Storage]
  • Fixes a panic in the disruption budget controller for PDB objects with invalid selectors (#98750, @mortent) [SIG Apps]
  • Fixes connection errors when using --volume-host-cidr-denylist or --volume-host-allow-local-loopback (#98436, @liggitt) [SIG Network and Storage]
  • If the user specifies an invalid timeout in the request URL, the request will be aborted with an HTTP 400.
    • in cases where the client specifies a timeout in the request URL, the overall request deadline is shortened now since the deadline is setup as soon as the request is received by the apiserver. (#96901, @tkashem) [SIG API Machinery and Testing]
  • Kubeadm: Some text in the kubeadm upgrade plan output has changed. If you have scripts or other automation that parses this output, please review these changes and update your scripts to account for the new output. (#98728, @stmcginnis) [SIG Cluster Lifecycle]
  • Kubeadm: fix a bug where external credentials in an existing admin.conf prevented the CA certificate to be written in the cluster-info ConfigMap. (#98882, @kvaps) [SIG Cluster Lifecycle]
  • Kubeadm: fix bad token placeholder text in "config print *-defaults --help" (#98839, @Mattias-) [SIG Cluster Lifecycle]
  • Kubeadm: get k8s CI version markers from k8s infra bucket (#98836, @hasheddan) [SIG Cluster Lifecycle and Release]
  • Mitigate CVE-2020-8555 for kube-up using GCE by preventing local loopback folume hosts. (#97934, @mattcary) [SIG Cloud Provider and Storage]
  • Remove CSI topology from migrated in-tree gcepd volume. (#97823, @Jiawei0227) [SIG Cloud Provider and Storage]
  • Sync node status during kubelet node shutdown. Adds an pod admission handler that rejects new pods when the node is in progress of shutting down. (#98005, @wzshiming) [SIG Node]
  • Truncates a message if it hits the NoteLengthLimit when the scheduler records an event for the pod that indicates the pod has failed to schedule. (#98715, @carlory) [SIG Scheduling]
  • We will no longer automatically delete all data when a failure is detected during creation of the volume data file on a CSI volume. Now we will only remove the data file and volume path. (#96021, @huffmanca) [SIG Storage]

Other (Cleanup or Flake)

  • Fix the description of command line flags that can override --config (#98254, @changshuchao) [SIG Scheduling]
  • Migrate scheduler/taint_manager.go structured logging (#98259, @tanjing2020) [SIG Apps]
  • Migrate staging/src/k8s.io/apiserver/pkg/admission logs to structured logging (#98138, @lala123912) [SIG API Machinery]
  • Resolves flakes in the Ingress conformance tests due to conflicts with controllers updating the Ingress object (#98430, @liggitt) [SIG Network and Testing]
  • The default delegating authorization options now allow unauthenticated access to healthz, readyz, and livez. A system:masters user connecting to an authz delegator will not perform an authz check. (#98325, @deads2k) [SIG API Machinery, Auth, Cloud Provider and Scheduling]
  • The e2e suite can be instructed not to wait for pods in kube-system to be ready or for all nodes to be ready by passing --allowed-not-ready-nodes=-1 when invoking the e2e.test program. This allows callers to run subsets of the e2e suite in scenarios other than perfectly healthy clusters. (#98781, @smarterclayton) [SIG Testing]
  • The feature gates WindowsGMSA and WindowsRunAsUserName that are GA since v1.18 are now removed. (#96531, @ialidzhikov) [SIG Node and Windows]
  • The new -gce-zones flag on the e2e.test binary instructs tests that check for information about how the cluster interacts with the cloud to limit their queries to the provided zone list. If not specified, the current behavior of asking the cloud provider for all available zones in multi zone clusters is preserved. (#98787, @smarterclayton) [SIG API Machinery, Cluster Lifecycle and Testing]

Dependencies

Added

  • github.com/moby/spdystream: v0.2.0

Changed

Removed

  • github.com/docker/spdystream: 449fdfc

v1.21.0-alpha.2

Downloads for v1.21.0-alpha.2

Source Code

filename sha512 hash
kubernetes.tar.gz 6836f6c8514253fe0831fd171fc4ed92eb6d9a773491c8dc82b90d171a1b10076bd6bfaea56ec1e199c5f46c273265bdb9f174f0b2d99c5af1de4c99b862329e
kubernetes-src.tar.gz d137694804741a05ab09e5f9a418448b66aba0146c028eafce61bcd9d7c276521e345ce9223ffbc703e8172041d58dfc56a3242a4df3686f24905a4541fcd306

Client binaries

filename sha512 hash
kubernetes-client-darwin-amd64.tar.gz 9478b047a97717953f365c13a098feb7e3cb30a3df22e1b82aa945f2208dcc5cb90afc441ba059a3ae7aafb4ee000ec3a52dc65a8c043a5ac7255a391c875330
kubernetes-client-linux-386.tar.gz 44c8dd4b1ddfc256d35786c8abf45b0eb5f0794f5e310d2efc865748adddc50e8bf38aa71295ae8a82884cb65f2e0b9b0737b000f96fd8f2d5c19971d7c4d8e8
kubernetes-client-linux-amd64.tar.gz e1291989892769de6b978c17b8612b94da6f3b735a4d895100af622ca9ebb968c75548afea7ab00445869625dd0da3afec979e333afbb445805f5d31c1c13cc7
kubernetes-client-linux-arm.tar.gz 3c4bcb8cbe73822d68a2f62553a364e20bec56b638c71d0f58679b4f4b277d809142346f18506914e694f6122a3e0f767eab20b7b1c4dbb79e4c5089981ae0f1
kubernetes-client-linux-arm64.tar.gz 9389974a790268522e187f5ba5237f3ee4684118c7db76bc3d4164de71d8208702747ec333b204c7a78073ab42553cbbce13a1883fab4fec617e093b05fab332
kubernetes-client-linux-ppc64le.tar.gz 63399e53a083b5af3816c28ff162c9de6b64c75da4647f0d6bbaf97afdf896823cb1e556f2abac75c6516072293026d3ff9f30676fd75143ac6ca3f4d21f4327
kubernetes-client-linux-s390x.tar.gz 50898f197a9d923971ff9046c9f02779b57f7b3cea7da02f3ea9bab8c08d65a9c4a7531a2470fa14783460f52111a52b96ebf916c0a1d8215b4070e4e861c1b0
kubernetes-client-windows-386.tar.gz a7743e839e1aa19f5ee20b6ee5000ac8ef9e624ac5be63bb574fad6992e4b9167193ed07e03c9bc524e88bfeed66c95341a38a03bff1b10bc9910345f33019f0
kubernetes-client-windows-amd64.tar.gz 5f1d19c230bd3542866d16051808d184e9dd3e2f8c001ed4cee7b5df91f872380c2bf56a3add8c9413ead9d8c369efce2bcab4412174df9b823d3592677bf74e

Server binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz ef2cac10febde231aeb6f131e589450c560eeaab8046b49504127a091cddc17bc518c2ad56894a6a033033ab6fc6e121b1cc23691683bc36f45fe6b1dd8e0510
kubernetes-server-linux-arm.tar.gz d11c9730307f08e80b2b8a7c64c3e9a9e43c622002e377dfe3a386f4541e24adc79a199a6f280f40298bb36793194fd44ed45defe8a3ee54a9cb1386bc26e905
kubernetes-server-linux-arm64.tar.gz 28f8c32bf98ee1add7edf5d341c3bac1afc0085f90dcbbfb8b27a92087f13e2b53c327c8935ee29bf1dc3160655b32bbe3e29d5741a8124a3848a777e7d42933
kubernetes-server-linux-ppc64le.tar.gz 99ae8d44b0de3518c27fa8bbddd2ecf053dfb789fb9d65f8a4ecf4c8331cf63d2f09a41c2bcd5573247d5f66a1b2e51944379df1715017d920d521b98589508a
kubernetes-server-linux-s390x.tar.gz f8c0e954a2dfc6845614488dadeed069cc7f3f08e33c351d7a77c6ef97867af590932e8576d12998a820a0e4d35d2eee797c764e2810f09ab1e90a5acaeaad33

Node binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz c5456d50bfbe0d75fb150b3662ed7468a0abd3970792c447824f326894382c47bbd3a2cc5a290f691c8c09585ff6fe505ab86b4aff2b7e5ccee11b5e6354ae6c
kubernetes-node-linux-arm.tar.gz 335b5cd8672e053302fd94d932fb2fa2e48eeeb1799650b3f93acdfa635e03a8453637569ab710c46885c8317759f4c60aaaf24dca9817d9fa47500fe4a3ca53
kubernetes-node-linux-arm64.tar.gz 3ee87dbeed8ace9351ac89bdaf7274dd10b4faec3ceba0825f690ec7a2bb7eb7c634274a1065a0939eec8ff3e43f72385f058f4ec141841550109e775bc5eff9
kubernetes-node-linux-ppc64le.tar.gz 6956f965b8d719b164214ec9195fdb2c776b907fe6d2c524082f00c27872a73475927fd7d2a994045ce78f6ad2aa5aeaf1eb5514df1810d2cfe342fd4e5ce4a1
kubernetes-node-linux-s390x.tar.gz 3b643aa905c709c57083c28dd9e8ffd88cb64466cda1499da7fc54176b775003e08b9c7a07b0964064df67c8142f6f1e6c13bfc261bd65fb064049920bfa57d0
kubernetes-node-windows-amd64.tar.gz b2e6d6fb0091f2541f9925018c2bdbb0138a95bab06b4c6b38abf4b7144b2575422263b78fb3c6fd09e76d90a25a8d35a6d4720dc169794d42c95aa22ecc6d5f

Changelog since v1.21.0-alpha.1

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Remove storage metrics storage_operation_errors_total, since we already have storage_operation_status_count.And add new field status for storage_operation_duration_seconds, so that we can know about all status storage operation latency. (#98332, @JornShen) [SIG Instrumentation and Storage]

Changes by Kind

Deprecation

  • Remove the TokenRequest and TokenRequestProjection feature gates (#97148, @wawa0210) [SIG Node]
  • Removing experimental windows container hyper-v support with Docker (#97141, @wawa0210) [SIG Node and Windows]
  • The export query parameter (inconsistently supported by API resources and deprecated in v1.14) is fully removed. Requests setting this query parameter will now receive a 400 status response. (#98312, @deads2k) [SIG API Machinery, Auth and Testing]

API Change

  • Enable SPDY pings to keep connections alive, so that kubectl exec and kubectl port-forward won't be interrupted. (#97083, @knight42) [SIG API Machinery and CLI]

Documentation

  • Official support to build kubernetes with docker-machine / remote docker is removed. This change does not affect building kubernetes with docker locally. (#97935, @adeniyistephen) [SIG Release and Testing]
  • Set kubelet option --volume-stats-agg-period to negative value to disable volume calculations. (#96675, @pacoxu) [SIG Node]

Bug or Regression

  • Clean ReplicaSet by revision instead of creation timestamp in deployment controller (#97407, @waynepeking348) [SIG Apps]
  • Ensure that client-go's EventBroadcaster is safe (non-racy) during shutdown. (#95664, @DirectXMan12) [SIG API Machinery]
  • Fix azure file migration issue (#97877, @andyzhangx) [SIG Auth, Cloud Provider and Storage]
  • Fix kubelet from panic after getting the wrong signal (#98200, @wzshiming) [SIG Node]
  • Fix repeatedly acquire the inhibit lock (#98088, @wzshiming) [SIG Node]
  • Fixed a bug that the kubelet cannot start on BtrfS. (#98042, @gjkim42) [SIG Node]
  • Fixed an issue with garbage collection failing to clean up namespaced children of an object also referenced incorrectly by cluster-scoped children (#98068, @liggitt) [SIG API Machinery and Apps]
  • Fixed no effect namespace when exposing deployment with --dry-run=client. (#97492, @masap) [SIG CLI]
  • Fixing a bug where a failed node may not have the NoExecute taint set correctly (#96876, @howieyuen) [SIG Apps and Node]
  • Indentation of Resource Quota block in kubectl describe namespaces output gets correct. (#97946, @dty1er) [SIG CLI]
  • KUBECTL_EXTERNAL_DIFF now accepts equal sign for additional parameters. (#98158, @dougsland) [SIG CLI]
  • Kubeadm: fix a bug where "kubeadm join" would not properly handle missing names for existing etcd members. (#97372, @ihgann) [SIG Cluster Lifecycle]
  • Kubelet should ignore cgroup driver check on Windows node. (#97764, @pacoxu) [SIG Node and Windows]
  • Make podTopologyHints protected by lock (#95111, @choury) [SIG Node]
  • Readjust kubelet_containers_per_pod_count bucket (#98169, @wawa0210) [SIG Instrumentation and Node]
  • Scores from InterPodAffinity have stronger differentiation. (#98096, @leileiwan) [SIG Scheduling]
  • Specifying the KUBE_TEST_REPO environment variable when e2e tests are executed will instruct the test infrastructure to load that image from a location within the specified repo, using a predefined pattern. (#93510, @smarterclayton) [SIG Testing]
  • Static pods will be deleted gracefully. (#98103, @gjkim42) [SIG Node]
  • Use network.Interface.VirtualMachine.ID to get the binded VM Skip standalone VM when reconciling LoadBalancer (#97635, @nilo19) [SIG Cloud Provider]

Other (Cleanup or Flake)

  • Kubeadm: change the default image repository for CI images from 'gcr.io/kubernetes-ci-images' to 'gcr.io/k8s-staging-ci-images' (#97087, @SataQiu) [SIG Cluster Lifecycle]
  • Migrate generic_scheduler.go and types.go to structured logging. (#98134, @tanjing2020) [SIG Scheduling]
  • Migrate proxy/winuserspace/proxier.go logs to structured logging (#97941, @JornShen) [SIG Network]
  • Migrate staging/src/k8s.io/apiserver/pkg/audit/policy/reader.go logs to structured logging. (#98252, @lala123912) [SIG API Machinery and Auth]
  • Migrate staging\src\k8s.io\apiserver\pkg\endpoints logs to structured logging (#98093, @lala123912) [SIG API Machinery]
  • Node (#96552, @pandaamanda) [SIG Apps, Cloud Provider, Node and Scheduling]
  • The kubectl alpha debug command was scheduled to be removed in v1.21. (#98111, @pandaamanda) [SIG CLI]
  • Update cri-tools to v1.20.0 (#97967, @rajibmitra) [SIG Cloud Provider]
  • Windows nodes on GCE will take longer to start due to dependencies installed at node creation time. (#98284, @pjh) [SIG Cloud Provider]

Dependencies

Added

Nothing has changed.

Changed

Removed

Nothing has changed.

v1.21.0-alpha.1

Downloads for v1.21.0-alpha.1

Source Code

filename sha512 hash
kubernetes.tar.gz b2bacd5c3fc9f829e6269b7d2006b0c6e464ff848bb0a2a8f2fe52ad2d7c4438f099bd8be847d8d49ac6e4087f4d74d5c3a967acd798e0b0cb4d7a2bdb122997
kubernetes-src.tar.gz 518ac5acbcf23902fb1b902b69dbf3e86deca5d8a9b5f57488a15f185176d5a109558f3e4df062366af874eca1bcd61751ee8098b0beb9bcdc025d9a1c9be693

Client binaries

filename sha512 hash
kubernetes-client-darwin-amd64.tar.gz eaa7aea84a5ed954df5ec710cbeb6ec88b46465f43cb3d09aabe2f714b84a050a50bf5736089f09dbf1090f2e19b44823d656c917e3c8c877630756c3026f2b6
kubernetes-client-linux-386.tar.gz 47f74b8d46ad1779c5b0b5f15aa15d5513a504eeb6f53db4201fbe9ff8956cb986b7c1b0e9d50a99f78e9e2a7f304f3fc1cc2fa239296d9a0dd408eb6069e975
kubernetes-client-linux-amd64.tar.gz 1a148e282628b008c8abd03dd12ec177ced17584b5115d92cd33dd251e607097d42e9da8c7089bd947134b900f85eb75a4740b6a5dd580c105455b843559df39
kubernetes-client-linux-arm.tar.gz d13d2feb73bd032dc01f7e2955b98d8215a39fe1107d037a73fa1f7d06c3b93ebaa53ed4952d845c64454ef3cca533edb97132d234d50b6fb3bcbd8a8ad990eb
kubernetes-client-linux-arm64.tar.gz 8252105a17b09a78e9ad2c024e4e401a69764ac869708a071aaa06f81714c17b9e7c5b2eb8efde33f24d0b59f75c5da607d5e1e72bdf12adfbb8c829205cd1c1
kubernetes-client-linux-ppc64le.tar.gz 297a9082df4988389dc4be30eb636dff49f36f5d87047bab44745884e610f46a17ae3a08401e2cab155b7c439f38057bfd8288418215f7dd3bf6a49dbe61ea0e
kubernetes-client-linux-s390x.tar.gz 04c06490dd17cd5dccfd92bafa14acf64280ceaea370d9635f23aeb6984d1beae6d0d1d1506edc6f30f927deeb149b989d3e482b47fbe74008b371f629656e79
kubernetes-client-windows-386.tar.gz ec6e9e87a7d685f8751d7e58f24f417753cff5554a7229218cb3a08195d461b2e12409344950228e9fbbc92a8a06d35dd86242da6ff1e6652ec1fae0365a88c1
kubernetes-client-windows-amd64.tar.gz 51039e6221d3126b5d15e797002ae01d4f0b10789c5d2056532f27ef13f35c5a2e51be27764fda68e8303219963126559023aed9421313bec275c0827fbcaf8a

Server binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz 4edf820930c88716263560275e3bd7fadb8dc3700b9f8e1d266562e356e0abeb1a913f536377dab91218e3940b447d6bf1da343b85da25c2256dc4dcde5798dd
kubernetes-server-linux-arm.tar.gz b15213e53a8ab4ba512ce6ef9ad42dd197d419c61615cd23de344227fd846c90448d8f3d98e555b63ba5b565afa627cca6b7e3990ebbbba359c96f2391302df1
kubernetes-server-linux-arm64.tar.gz 5be29cca9a9358fc68351ee63e99d57dc2ffce6e42fc3345753dbbf7542ff2d770c4852424158540435fa6e097ce3afa9b13affc40c8b3b69fe8406798f8068f
kubernetes-server-linux-ppc64le.tar.gz 89fd99ab9ce85db0b94b86709932105efc883cc93959cf7ea9a39e79a4acea23064d7010eeb577450cccabe521c04b7ba47bbec212ed37edeed7cb04bad34518
kubernetes-server-linux-s390x.tar.gz 2fbc30862c77d247aa8d96ab9d1a144599505287b0033a3a2d0988958e7bb2f2e8b67f52c1fec74b4ec47d74ba22cd0f6cb5c4228acbaa72b1678d5fece0254d

Node binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz 95658d321a0a371c0900b401d1469d96915310afbc4e4b9b11f031438bb188513b57d5a60b5316c3b0c18f541cda6f0ac42f59a76495f8abc743a067115da23a
kubernetes-node-linux-arm.tar.gz f375acfb42aad6c65b833c270e7e3acfe9cd1d6b2441c33874e77faae263957f7acfe86f1b71f14298118595e4cc6952c7dea0c832f7f2e72428336f13034362
kubernetes-node-linux-arm64.tar.gz 43b4baccd58d74e7f48d096ab92f2bbbcdf47e30e7a3d2b56c6cc9f90002cfd4fefaac894f69bd5f9f4dbdb09a4749a77eb76b1b97d91746bd96fe94457879ab
kubernetes-node-linux-ppc64le.tar.gz e7962b522c6c7c14b9ee4c1d254d8bdd9846b2b33b0443fc9c4a41be6c40e5e6981798b720f0148f36263d5cc45d5a2bb1dd2f9ab2838e3d002e45b9bddeb7bf
kubernetes-node-linux-s390x.tar.gz 49ebc97f01829e65f7de15be00b882513c44782eaadd1b1825a227e3bd3c73cc6aca8345af05b303d8c43aa2cb944a069755b2709effb8cc22eae621d25d4ba5
kubernetes-node-windows-amd64.tar.gz 6e0fd7724b09e6befbcb53b33574e97f2db089f2eee4bbf391abb7f043103a5e6e32e3014c0531b88f9a3ca88887bbc68625752c44326f98dd53adb3a6d1bed8

Changelog since v1.20.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Kube-proxy's IPVS proxy mode no longer sets the net.ipv4.conf.all.route_localnet sysctl parameter. Nodes upgrading will have net.ipv4.conf.all.route_localnet set to 1 but new nodes will inherit the system default (usually 0). If you relied on any behavior requiring net.ipv4.conf.all.route_localnet, you must set ensure it is enabled as kube-proxy will no longer set it automatically. This change helps to further mitigate CVE-2020-8558. (#92938, @lbernail) [SIG Network and Release]

Changes by Kind

Deprecation

  • Deprecate the topologyKeys field in Service. This capability will be replaced with upcoming work around Topology Aware Subsetting and Service Internal Traffic Policy. (#96736, @andrewsykim) [SIG Apps]
  • Kubeadm: deprecated command "alpha selfhosting pivot" is removed now. (#97627, @knight42) [SIG Cluster Lifecycle]
  • Kubeadm: graduate the command kubeadm alpha kubeconfig user to kubeadm kubeconfig user. The kubeadm alpha kubeconfig user command is deprecated now. (#97583, @knight42) [SIG Cluster Lifecycle]
  • Kubeadm: the "kubeadm alpha certs" command is removed now, please use "kubeadm certs" instead. (#97706, @knight42) [SIG Cluster Lifecycle]
  • Remove the deprecated metrics "scheduling_algorithm_preemption_evaluation_seconds" and "binding_duration_seconds", suggest to use "scheduler_framework_extension_point_duration_seconds" instead. (#96447, @chendave) [SIG Cluster Lifecycle, Instrumentation, Scheduling and Testing]
  • The PodSecurityPolicy API is deprecated in 1.21, and will no longer be served starting in 1.25. (#97171, @deads2k) [SIG Auth and CLI]

API Change

  • Change the APIVersion proto name of BoundObjectRef from aPIVersion to apiVersion. (#97379, @kebe7jun) [SIG Auth]
  • Promote Immutable Secrets/ConfigMaps feature to Stable. This allows to set Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (#97615, @wojtek-t) [SIG Apps, Architecture, Node and Testing]

Feature

  • Add flag --lease-max-object-size and metric etcd_lease_object_counts for kube-apiserver to config and observe max objects attached to a single etcd lease. (#97480, @lingsamuel) [SIG API Machinery, Instrumentation and Scalability]
  • Add flag --lease-reuse-duration-seconds for kube-apiserver to config etcd lease reuse duration. (#97009, @lingsamuel) [SIG API Machinery and Scalability]
  • Adds the ability to pass --strict-transport-security-directives to the kube-apiserver to set the HSTS header appropriately. Be sure you understand the consequences to browsers before setting this field. (#96502, @249043822) [SIG Auth]
  • Kubeadm now includes CoreDNS v1.8.0. (#96429, @rajansandeep) [SIG Cluster Lifecycle]
  • Kubeadm: add support for certificate chain validation. When using kubeadm in external CA mode, this allows an intermediate CA to be used to sign the certificates. The intermediate CA certificate must be appended to each signed certificate for this to work correctly. (#97266, @robbiemcmichael) [SIG Cluster Lifecycle]
  • Kubeadm: amend the node kernel validation to treat CGROUP_PIDS, FAIR_GROUP_SCHED as required and CFS_BANDWIDTH, CGROUP_HUGETLB as optional (#96378, @neolit123) [SIG Cluster Lifecycle and Node]
  • The Kubernetes pause image manifest list now contains an image for Windows Server 20H2. (#97322, @claudiubelu) [SIG Windows]
  • The apimachinery util/net function used to detect the bind address ResolveBindAddress() takes into consideration global ip addresses on loopback interfaces when: - the host has default routes - there are no global IPs on those interfaces. in order to support more complex network scenarios like BGP Unnumbered RFC 5549 (#95790, @aojea) [SIG Network]

Bug or Regression

  • Changelog

    General

    • Fix priority expander falling back to a random choice even though there is a higher priority option to choose
    • Clone kubernetes/kubernetes in update-vendor.sh shallowly, instead of fetching all revisions
    • Speed up binpacking by reducing the number of PreFilter calls (call once per pod instead of #pods*#nodes times)
    • Speed up finding unneeded nodes by 5x+ in very large clusters by reducing the number of PreFilter calls
    • Expose --max-nodes-total as a metric
    • Errors in IncreaseSize changed from type apiError to cloudProviderError
    • Make build-in-docker and test-in-docker work on Linux systems with SELinux enabled
    • Fix an error where existing nodes were not considered as destinations while finding place for pods in scale-down simulations
    • Remove redundant log lines and reduce severity around parsing kubeEnv
    • Don't treat nodes created by virtual kubelet as nodes from non-autoscaled node groups
    • Remove redundant logging around calculating node utilization
    • Add configurable --network and --rm flags for docker in Makefile
    • Subtract DaemonSet pods' requests from node allocatable in the denominator while computing node utilization
    • Include taints by condition when determining if a node is unready/still starting
    • Fix update-vendor.sh to work on OSX and zsh
    • Add best-effort eviction for DaemonSet pods while scaling down non-empty nodes
    • Add build support for ARM64

    AliCloud

    • Add missing daemonsets and replicasets to ALI example cluster role

    Apache CloudStack

    • Add support for Apache CloudStack

    AWS

    • Regenerate list of EC2 instances
    • Fix pricing endpoint in AWS China Region

    Azure

    • Add optional jitter on initial VMSS VM cache refresh, keep the refreshes spread over time
    • Serve from cache for the whole period of ongoing throttling
    • Fix unwanted VMSS VMs cache invalidations
    • Enforce setting the number of retries if cloud provider backoff is enabled
    • Don't update capacity if VMSS provisioning state is updating
    • Support allocatable resources overrides via VMSS tags
    • Add missing stable labels in template nodes
    • Proactively set instance status to deleting on node deletions

    Cluster API

    • Migrate interaction with the API from using internal types to using Unstructured
    • Improve tests to work better with constrained resources
    • Add support for node autodiscovery
    • Add support for --cloud-config
    • Update group identifier to use for Cluster API annotations

    Exoscale

    • Add support for Exoscale

    GCE

    • Decrease the number of GCE Read Requests made while deleting nodes
    • Base pricing of custom instances on their instance family type
    • Add pricing information for missing machine types
    • Add pricing information for different GPU types
    • Ignore the new topology.gke.io/zone label when comparing groups
    • Add missing stable labels to template nodes

    HuaweiCloud

    • Add auto scaling group support
    • Implement node group by AS
    • Implement getting desired instance number of node group
    • Implement increasing node group size
    • Implement TemplateNodeInfo
    • Implement caching instances

    IONOS

    • Add support for IONOS

    Kubemark

    • Skip non-kubemark nodes while computing node infos for node groups.

    Magnum

    • Add Magnum support in the Cluster Autoscaler helm chart

    Packet

    • Allow empty nodepools
    • Add support for multiple nodepools
    • Add pricing support

    Image

    Image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.20.0 (#97011, @towca) [SIG Cloud Provider]

  • AcceleratorStats will be available in the Summary API of kubelet when cri_stats_provider is used. (#96873, @ruiwen-zhao) [SIG Node]

  • Add limited lines to log when having tail option (#93920, @zhouya0) [SIG Node]

  • Avoid systemd-logind loading configuration warning (#97950, @wzshiming) [SIG Node]

  • Cloud-controller-manager: routes controller should not depend on --allocate-node-cidrs (#97029, @andrewsykim) [SIG Cloud Provider and Testing]

  • Copy annotations with empty value when deployment rolls back (#94858, @waynepeking348) [SIG Apps]

  • Detach volumes from vSphere nodes not tracked by attach-detach controller (#96689, @gnufied) [SIG Cloud Provider and Storage]

  • Fix kubectl label error when local=true is set. (#97440, @pandaamanda) [SIG CLI]

  • Fix Azure file share not deleted issue when the namespace is deleted (#97417, @andyzhangx) [SIG Cloud Provider and Storage]

  • Fix CVE-2020-8555 for Gluster client connections. (#97922, @liggitt) [SIG Storage]

  • Fix counting error in service/nodeport/loadbalancer quota check (#97451, @pacoxu) [SIG API Machinery, Network and Testing]

  • Fix kubectl-convert import known versions (#97754, @wzshiming) [SIG CLI and Testing]

  • Fix missing cadvisor machine metrics. (#97006, @lingsamuel) [SIG Node]

  • Fix nil VMSS name when setting service to auto mode (#97366, @nilo19) [SIG Cloud Provider]

  • Fix the panic when kubelet registers if a node object already exists with no Status.Capacity or Status.Allocatable (#95269, @SataQiu) [SIG Node]

  • Fix the regression with the slow pods termination. Before this fix pods may take an additional time to terminate - up to one minute. Reversing the change that ensured that CNI resources cleaned up when the pod is removed on API server. (#97980, @SergeyKanzhelev) [SIG Node]

  • Fix to recover CSI volumes from certain dangling attachments (#96617, @yuga711) [SIG Apps and Storage]

  • Fix: azure file latency issue for metadata-heavy workloads (#97082, @andyzhangx) [SIG Cloud Provider and Storage]

  • Fixed Cinder volume IDs on OpenStack Train (#96673, @jsafrane) [SIG Cloud Provider]

  • Fixed FibreChannel volume plugin corrupting filesystems on detach of multipath volumes. (#97013, @jsafrane) [SIG Storage]

  • Fixed a bug in kubelet that will saturate CPU utilization after containerd got restarted. (#97174, @hanlins) [SIG Node]

  • Fixed bug in CPUManager with race on container map access (#97427, @klueska) [SIG Node]

  • Fixed cleanup of block devices when /var/lib/kubelet is a symlink. (#96889, @jsafrane) [SIG Storage]

  • GCE Internal LoadBalancer sync loop will now release the ILB IP address upon sync failure. An error in ILB forwarding rule creation will no longer leak IP addresses. (#97740, @prameshj) [SIG Cloud Provider and Network]

  • Ignore update pod with no new images in alwaysPullImages admission controller (#96668, @pacoxu) [SIG Apps, Auth and Node]

  • Kubeadm now installs version 3.4.13 of etcd when creating a cluster with v1.19 (#97244, @pacoxu) [SIG Cluster Lifecycle]

  • Kubeadm: avoid detection of the container runtime for commands that do not need it (#97625, @pacoxu) [SIG Cluster Lifecycle]

  • Kubeadm: fix a bug in the host memory detection code on 32bit Linux platforms (#97403, @abelbarrera15) [SIG Cluster Lifecycle]

  • Kubeadm: fix a bug where "kubeadm upgrade" commands can fail if CoreDNS v1.8.0 is installed. (#97919, @neolit123) [SIG Cluster Lifecycle]

  • Performance regression #97685 has been fixed. (#97860, @MikeSpreitzer) [SIG API Machinery]

  • Remove deprecated --cleanup-ipvs flag of kube-proxy, and make --cleanup flag always to flush IPVS (#97336, @maaoBit) [SIG Network]

  • The current version of the container image publicly exposed IP serving a /metrics endpoint to the Internet. The new version of the container image serves /metrics endpoint on a different port. (#97621, @vbannai) [SIG Cloud Provider]

  • Use force unmount for NFS volumes if regular mount fails after 1 minute timeout (#96844, @gnufied) [SIG Storage]

  • Users will see increase in time for deletion of pods and also guarantee that removal of pod from api server would mean deletion of all the resources from container runtime. (#92817, @kmala) [SIG Node]

  • Using exec auth plugins with kubectl no longer results in warnings about constructing many client instances from the same exec auth config. (#97857, @liggitt) [SIG API Machinery and Auth]

  • Warning about using a deprecated volume plugin is logged only once. (#96751, @jsafrane) [SIG Storage]

Other (Cleanup or Flake)

  • Bump github.com/Azure/go-autorest/autorest to v0.11.12 (#97033, @patrickshan) [SIG API Machinery, CLI, Cloud Provider and Cluster Lifecycle]
  • Delete deprecated mixed protocol annotation (#97096, @nilo19) [SIG Cloud Provider]
  • Kube-proxy: Traffic from the cluster directed to ExternalIPs is always sent directly to the Service. (#96296, @aojea) [SIG Network and Testing]
  • Kubeadm: fix a whitespace issue in the output of the "kubeadm join" command shown as the output of "kubeadm init" and "kubeadm token create --print-join-command" (#97413, @SataQiu) [SIG Cluster Lifecycle]
  • Kubeadm: improve the error messaging when the user provides an invalid discovery token CA certificate hash. (#97290, @neolit123) [SIG Cluster Lifecycle]
  • Migrate log messages in pkg/scheduler/{scheduler.go,factory.go} to structured logging (#97509, @aldudko) [SIG Scheduling]
  • Migrate proxy/iptables/proxier.go logs to structured logging (#97678, @JornShen) [SIG Network]
  • Migrate some scheduler log messages to structured logging (#97349, @aldudko) [SIG Scheduling]
  • NONE (#97167, @geegeea) [SIG Node]
  • NetworkPolicy validation framework optimizations for rapidly verifying CNI's work correctly across several pods and namespaces (#91592, @jayunit100) [SIG Network, Storage and Testing]
  • Official support to build kubernetes with docker-machine / remote docker is removed. This change does not affect building kubernetes with docker locally. (#97618, @jherrera123) [SIG Release and Testing]
  • Scheduler plugin validation now provides all errors detected instead of the first one. (#96745, @lingsamuel) [SIG Node, Scheduling and Testing]
  • Storage related e2e testsuite redesign & cleanup (#96573, @Jiawei0227) [SIG Storage and Testing]
  • The OIDC authenticator no longer waits 10 seconds before attempting to fetch the metadata required to verify tokens. (#97693, @enj) [SIG API Machinery and Auth]
  • The AttachVolumeLimit feature gate that is GA since v1.17 is now removed. (#96539, @ialidzhikov) [SIG Storage]
  • The CSINodeInfo feature gate that is GA since v1.17 is unconditionally enabled, and can no longer be specified via the --feature-gates argument. (#96561, @ialidzhikov) [SIG Apps, Auth, Scheduling, Storage and Testing]
  • The deprecated feature gates RotateKubeletClientCertificate, AttachVolumeLimit, VolumePVCDataSource and EvenPodsSpread are now unconditionally enabled and can no longer be specified in component invocations. (#97306, @gavinfish) [SIG Node, Scheduling and Storage]
  • ServiceNodeExclusion, NodeDisruptionExclusion and LegacyNodeRoleBehavior(locked to false) features have been promoted to GA. To prevent control plane nodes being added to load balancers automatically, upgrade users need to add "node.kubernetes.io/exclude-from-external-load-balancers" label to control plane nodes. (#97543, @pacoxu) [SIG API Machinery, Apps, Cloud Provider and Network]

Uncategorized

  • Adding Brazilian Portuguese translation for kubectl (#61595, @cpanato) [SIG CLI]

Dependencies

Added

Nothing has changed.

Changed

Removed

  • rsc.io/quote/v3: v3.1.0
  • rsc.io/sampler: v1.3.0