Test and validation for each CSP of Kubernetes provisioning

[seokho-son]

What would you like to be enhanced
: Testing and validation for Kubernetes provisioning on each CSP

Why is this needed
: To understand and share the current implementation status of the managed Kubernetes service

Proposed solution
: Create a checklist and mark each item once testing has been completed

• AWS
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• Azure
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• GCP
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• Alibaba
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• Tencent
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• IBM
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• NCP
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• NHN
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• KT
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access
• Open Stack
  • Implementation stability:
    • NA,
    • Alpha,
    • Beta,
    • GA
  • Cluster/NodeGroup: provisioning
  • Cluster/NodeGroup: get info
  • Cluster/NodeGroup: lifecycle control
  • Cluster/NodeGroup: termination
  • Cluster/NodeGroup: access
    • by Headlamp
    • kubectl Nginx deploy/access

[seokho-son]

Hi @hanizang77 (CC @powerkimhub @yunkon-kim),

As the maintainer, I would like to check the implementation status of Kubernetes provisioning for all CSPs.

Please thoroughly test all cases and update the checklist items accordingly.

I know your are working on this issue, but please set a target date for completing the testing in advance.

[seokho-son]

@hanizang77

please note that I have changed checkbox items.

  - [ ] Cluster/NodeGroup: access 
    - [ ] by [Headlamp](https://headlamp.dev/)
    - [ ] kubectl Nginx deploy/access

[hanizang77]

Current Status Check & Key Issues

The checklist proposed by @seokho-son is meaningful for re-checking the real-time status of each CSP.
While this is a great template, I would like to share some specific issues I've been personally tracking in more detail.

🔗 Kubeconfig & Cluster Access

• AWS, GCP: Fetching kubeconfig and accessing clusters was previously verified, but we need to re-confirm whether it still works correctly.
• NCP: The same functionality needs to be implemented and is planned for upcoming work.

🔄 Lifecycle Management

• While working on NCP, issues related to lifecycle management (e. g., managing the state of resources being deleted) were discovered.
• This issue is not limited to NCP but appears to be commonly linked across multiple CSPs.

🔑 Credential & Permission Issues

• NHN: Currently experiencing errors when issuing credentials and configuring permissions.

---

I will work on these items systematically once my current report writing is completed.
I'll keep this issue updated with any progress or new findings.

[hanizang77]

Test Results: AWS EKS & GCP GKE (2025-12-05)

AWS EKS (ap-northeast-2)

• K8s Version: 1.31
• Cluster: k8scluster01 ✅
• NodeGroup: k8sng01 (t3a.xlarge, 1 node) ✅
• kubectl get nodes: Ready ✅
• Nginx deployment: Running ✅
• LoadBalancer External IP: Assigned ✅
• Internal curl test: Welcome to nginx! ✅

GCP GKE (asia-east1)

• K8s Version: 1.33.5-gke.1125000
• Cluster: k8scluster01 ✅
• NodeGroup: k8sng01 (e2-standard-4, 1 node) ✅
• kubectl get nodes: Ready ✅
• Nginx deployment: Running ✅
• LoadBalancer External IP: 34.80.90.94 ✅
• Internal curl test: Welcome to nginx! ✅

Both CSPs passed the "kubectl Nginx deploy/access" test successfully.

[hanizang77]

Test Results: Azure AKS & Tencent TKE (2025-12-11)

Azure AKS (koreacentral)

• K8s Version: 1.30.6
• Cluster: k8scluster01 ✅
• NodeGroup: k8sng01 (Standard_B2s, 2 nodes) ✅
• kubectl get nodes: Ready ✅
• Nginx deployment: Running ✅
• LoadBalancer External IP: Assigned ✅
• External curl test: Welcome to nginx! ✅

Tencent TKE (ap-seoul)

• K8s Version: 1.30.0
• Cluster: k8scluster01 ✅
• NodeGroup: k8sng01 (S5.MEDIUM4, 2 nodes) ✅
• kubectl get nodes: Ready ✅
• Nginx deployment: Running ✅
• LoadBalancer External IP: lb-otmyqm55-izjya5k8tq2weo3d.clb.kr-tencentclb.com ✅
• External curl test: Welcome to nginx! ✅

Both CSPs passed the "kubectl Nginx deploy/access" test successfully.

---

⚠️ Known Issue: Tencent Role Configuration

During testing, we encountered IAM role configuration issues with Tencent Cloud:

Problem 1 - TKE Service Role: When creating a TKE cluster via CB-Tumblebug, the cluster creation failed due to missing TKE_QCSRole service role.

Root Cause: Tencent Cloud requires the TKE_QCSRole service role to be pre-configured in the account before cluster creation. This role grants TKE permission to manage cloud resources (VPC, CLB, CVM, CBS, etc.) on behalf of the user.

Required Permissions:

• QcloudAccessForTKERole - Base TKE permissions
• QcloudAccessForTKERoleInOpsManagement - TKE operations management permissions

Solution:

1. Via Tencent Cloud Console:
   • CAM → Roles → Create Role → Tencent Cloud Service → Select "TKE"
   • Manually attach the following policies: QcloudAccessForTKERole, QcloudAccessForTKERoleInOpsManagement
2. Or use Tencent Cloud API/CLI to create the service role and attach both policies

Reference: Tencent TKE Service Authorization

---

Problem 2 - Auto Scaling Role: When adding a NodeGroup with Auto Scaling enabled via CB-Tumblebug, the request failed with HTTP 500. The error message from CB-Spider was:

{
  "message": "Failed to Add Node Group : [TencentCloudSDKError] Code=FailedOperation.AsCommon, Message=AS_COMMON_ERROR(err:[TencentCloudSDKError] Code=UnauthorizedOperation.AutoScalingRoleUnauthorized, Message=当前账号未授权服务角色 AS_QCSRole，请前往弹性伸缩控制台进行授权。, RequestId=...), RequestId=... (from cb-spider:1024/spider/cluster/.../nodegroup (500 Internal Server Error))"
}

Error Translation: "The current account has not authorized the service role AS_QCSRole. Please go to the Auto Scaling console for authorization."

Root Cause: Tencent Cloud requires the AS_QCSRole (Auto Scaling service role) to be pre-configured for Auto Scaling Group (ASG) operations. This role allows TKE to automatically scale worker nodes.

Required Permissions:

• QcloudAccessForASRole - Auto Scaling permissions for CVM, VPC, CLB, CBS operations

Solution:

1. Via Tencent Cloud Console:
   • CAM → Roles → Create Role → Tencent Cloud Service → Select "Auto Scaling"
   • Manually attach the QcloudAccessForASRole policy
2. Or use Tencent Cloud API/CLI to create the service role and attach the policy

Reference: Tencent Auto Scaling Service Authorization

---

These are one-time setup requirements per Tencent Cloud account and not specific to CB-Tumblebug.