Skip to content

[BUG]: HelmDeploy and Kubernetes task on version 246 is having authorization issue on fetching credential of AKS cluster #20538

Closed
@sandywangkpmg

Description

New issue checklist

Task name

HelmDeploy and Kubernetes

Task version

[email protected], [email protected]

Issue Description

Our CICD pipelines started failing today.
Looking into logs, we found out the new version of those tasks changed the behavior of how the credential is fetched.
Version 243 and 244 worked fine.

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 22.04

Relevant log output

Version 246:
2024-10-11T02:43:13.1416028Z ##[section]Starting: Test Release to AUS Cluster
2024-10-11T02:43:13.3470414Z ##[section]Starting: Initialize job
2024-10-11T02:43:13.3473344Z Agent name: 'origi116f000055'
2024-10-11T02:43:13.3474017Z Agent machine name: 'origi116f000055'
2024-10-11T02:43:13.3474379Z Current agent version: '3.245.0'
2024-10-11T02:43:13.3495401Z Current image version: 'dev'
2024-10-11T02:43:13.4925235Z Agent running as: 'AzDevOps'
2024-10-11T02:43:13.5306692Z Prepare build directory.
2024-10-11T02:43:13.5631883Z Set build variables.
2024-10-11T02:43:13.5664337Z Download all required tasks.
2024-10-11T02:43:14.0853666Z Downloading task: KubeloginInstaller (0.246.1)
2024-10-11T02:43:15.1123585Z Downloading task: HelmDeploy (0.246.1)
2024-10-11T02:43:16.9117680Z Downloading task: Kubernetes (1.246.1)

2024-10-11T02:43:25.0960847Z ==============================================================================
2024-10-11T02:43:25.0960983Z Task : Package and deploy Helm charts
2024-10-11T02:43:25.0961079Z Description : Deploy, configure, update a Kubernetes cluster in Azure Container Service by running helm commands
2024-10-11T02:43:25.0961215Z Version : 0.246.1
2024-10-11T02:43:25.0961284Z Author : Microsoft Corporation
2024-10-11T02:43:25.0961370Z Help : https://aka.ms/azpipes-helm-tsg
2024-10-11T02:43:25.0961454Z ==============================================================================
2024-10-11T02:43:26.2077844Z ##[error]Error: Cannot fetch the credentials for the cluster xxx-aks-cluster-xxx. Reason The client 'xxxx' with object id 'xxxx' does not have authorization to perform action 'Microsoft.ContainerService/managedClusters/listClusterUserCredential/action' over scope '/subscriptions/xxxx/resourceGroups/xxxx-rg-aks-xxx-australiaeast/providers/Microsoft.ContainerService/managedClusters/xxx-aks-cluster-xxx' or the scope is invalid. If access was recently granted, please refresh your credentials. (CODE: 403).

2024-10-11T02:44:48.2534837Z ==============================================================================
2024-10-11T02:44:48.2534996Z Task : Kubectl
2024-10-11T02:44:48.2535070Z Description : Deploy, configure, update a Kubernetes cluster in Azure Container Service by running kubectl commands
2024-10-11T02:44:48.2535210Z Version : 1.246.1
2024-10-11T02:44:48.2535290Z Author : Microsoft Corporation
2024-10-11T02:44:48.2535369Z Help : https://aka.ms/azpipes-kubectl-tsg
2024-10-11T02:44:48.2535456Z ==============================================================================
2024-10-11T02:44:49.2676982Z ##[error]Cannot fetch the credentials for the cluster the credentials for the cluster xxx-aks-cluster-xxx. Reason The client 'xxxx' with object id 'xxxx' does not have authorization to perform action 'Microsoft.ContainerService/managedClusters/listClusterUserCredential/action' over scope '/subscriptions/xxxx/resourceGroups/xxxx-rg-aks-xxx-australiaeast/providers/Microsoft.ContainerService/managedClusters/xxx-aks-cluster-xxx' or the scope is invalid. If access was recently granted, please refresh your credentials. (CODE: 403).

Version 243 and 244:
2024-10-10T04:12:42.0832255Z Set build variables.
2024-10-10T04:12:42.0866289Z Download all required tasks.
2024-10-10T04:12:42.4874746Z Downloading task: KubeloginInstaller (0.246.1)
2024-10-10T04:12:43.3707776Z Downloading task: HelmDeploy (0.243.11)
2024-10-10T04:12:44.4708599Z Downloading task: Kubernetes (1.244.0)

2024-10-10T04:12:56.1610194Z ##[section]Starting: helm upgrade - xxx installer job
2024-10-10T04:12:56.1616811Z ==============================================================================
2024-10-10T04:12:56.1616931Z Task : Package and deploy Helm charts
2024-10-10T04:12:56.1617036Z Description : Deploy, configure, update a Kubernetes cluster in Azure Container Service by running helm commands
2024-10-10T04:12:56.1617170Z Version : 0.243.11
2024-10-10T04:12:56.1617238Z Author : Microsoft Corporation
2024-10-10T04:12:56.1617330Z Help : https://aka.ms/azpipes-helm-tsg
2024-10-10T04:12:56.1617413Z ==============================================================================
2024-10-10T04:12:57.2954266Z [command]/agent/_work/_temp/c23852c7-09cb-4746-a619-d6ef9b202ad9/bin/linux_amd64/kubelogin convert-kubeconfig -l spn --client-id *** --client-secret *** --tenant-id xxxx
2024-10-10T04:12:57.3143400Z
2024-10-10T04:12:57.3561994Z [command]/usr/local/bin/helm upgrade --namespace xxx --install --wait --timeout 12000s --version xxx --set namespace=xxx --set xxxx
2024-10-10T04:12:58.2617299Z Release "xxx-xxx-installer" does not exist. Installing it now.
2024-10-10T04:12:58.2617876Z NAME: xxx-xxx-installer
2024-10-10T04:12:58.2618200Z LAST DEPLOYED: Thu Oct 10 04:12:58 2024
2024-10-10T04:12:58.2618489Z NAMESPACE: xxx
2024-10-10T04:12:58.2618709Z STATUS: deployed
2024-10-10T04:12:58.2618925Z REVISION: 1
2024-10-10T04:12:58.2619138Z TEST SUITE: None
2024-10-10T04:12:58.2619371Z NOTES:
2024-10-10T04:12:58.2619697Z Note
....
2024-10-10T04:13:00.8062025Z ##[section]Finishing: helm upgrade - xxx installer job

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION] 

Repro steps

No response

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions