Skip to content

Commit a06eee5

Browse files
wenxuan0923Copilot
wenxuan0923
and
Copilot
authored
Align FlexNode config with RP bootstrap data (#195)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1 parent b5c416e commit a06eee5

32 files changed

Lines changed: 1630 additions & 377 deletions

‎.env.example‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ E2E_RESOURCE_GROUP=rg-aksflexnode-e2e-tests
3434
# Azure region for E2E test resources
3535
E2E_LOCATION=westus2
3636

37+
# Target AKS agent pool name written to E2E node configs
38+
E2E_TARGET_AGENT_POOL_NAME=aksflexnodes
39+
3740
# ============================================================================
3841
# Required: Runner VM Configuration
3942
# ============================================================================

‎README.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ On your workstation, save the config helper script, setup node RBAC permissions,
3636
RESOURCE_GROUP="<resource-group>"
3737
CLUSTER_NAME="<cluster-name>"
3838
SUBSCRIPTION_ID="<subscription-id>"
39+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
3940

4041
curl -fsSLo ./aks-flex-config https://raw.githubusercontent.com/Azure/AKSFlexNode/main/scripts/aks-flex-config
4142
chmod +x ./aks-flex-config
@@ -49,6 +50,7 @@ chmod +x ./aks-flex-config
4950
--resource-group "$RESOURCE_GROUP" \
5051
--cluster-name "$CLUSTER_NAME" \
5152
--subscription "$SUBSCRIPTION_ID" \
53+
--agent-pool-name "$AGENT_POOL_NAME" \
5254
--bootstrap-token \
5355
--output ./aks-flex-node-config.json
5456
```
@@ -65,7 +67,8 @@ The rendered config should look like this. Comments are shown here only to expla
6567
"azure": {
6668
"subscriptionId": "<subscription-id>", // Azure subscription that owns the AKS cluster.
6769
"tenantId": "<tenant-id>", // Microsoft Entra tenant for the subscription.
68-
"cloud": "AzurePublicCloud", // Azure cloud environment.
70+
"resourceManagerEndpoint": "https://management.azure.com", // Azure Resource Manager endpoint for ARM calls.
71+
"targetAgentPoolName": "<agent-pool-name>", // AKS agent pool used for FlexNode machine registration.
6972
"bootstrapToken": {
7073
"token": "<token-id>.<token-secret>" // Kubernetes bootstrap token created by generate-node-config.
7174
},
@@ -77,15 +80,18 @@ The rendered config should look like this. Comments are shown here only to expla
7780
},
7881
"node": {
7982
"kubelet": {
80-
"serverURL": "https://<aks-api-server>", // AKS API server endpoint.
83+
"clusterFQDN": "<aks-api-server-fqdn>", // AKS API server FQDN.
8184
"caCertData": "<base64-ca-data>" // Cluster CA bundle from kubeconfig.
8285
}
8386
},
87+
"networking": {
88+
"dnsServiceIP": "<cluster-dns-service-ip>" // Cluster DNS service IP from the AKS network profile.
89+
},
8490
"agent": {
8591
"logLevel": "info", // Agent log verbosity.
8692
"logDir": "/var/log/aks-flex-node" // Host log directory.
8793
},
88-
"kubernetes": { "version": "<aks-kubernetes-version>" } // Kubelet version to install.
94+
"components": { "kubernetes": "<aks-kubernetes-version>" } // Kubelet version to install.
8995
}
9096
```
9197

‎docs/labs/aks-private-cluster-cilium.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,7 @@ AKS_REGION="<aks-region>"
134134
VM_REGION="<vm-region>"
135135
AKS_VNET="<aks-vnet-name>"
136136
FLEX_VNET="<flex-vnet-name>"
137+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
137138

138139
az account set --subscription "$SUBSCRIPTION_ID"
139140

@@ -340,6 +341,7 @@ chmod +x ./aks-flex-config
340341
--resource-group "$AKS_RG" \
341342
--cluster-name "$CLUSTER_NAME" \
342343
--subscription "$SUBSCRIPTION_ID" \
344+
--agent-pool-name "$AGENT_POOL_NAME" \
343345
--bootstrap-token \
344346
--output ./aks-flex-node-config.json
345347
```
@@ -351,7 +353,7 @@ TOKEN_ID=$(python3 -c 'import json; print(json.load(open("./aks-flex-node-config
351353
kubectl get secret -n kube-system "bootstrap-token-${TOKEN_ID}"
352354
```
353355

354-
For private clusters, if your workstation cannot reach the private API endpoint, run the RBAC/token creation from the admin VM and render the config from `az aks show` plus `az aks get-credentials --admin --file <path>`. The `kubernetes.version` value must be the full patch version, such as `1.34.7`, not the major/minor alias such as `1.34`; `aks-flex-node` uses this value to download Kubernetes binaries.
356+
For private clusters, if your workstation cannot reach the private API endpoint, run the RBAC/token creation from the admin VM and render the config from `az aks show` plus `az aks get-credentials --admin --file <path>`. The `components.kubernetes` value must be the full patch version, such as `1.34.7`, not the major/minor alias such as `1.34`; `aks-flex-node` uses this value to download Kubernetes binaries.
355357

356358
```bash
357359
KUBERNETES_VERSION=$(az aks show \
@@ -365,14 +367,16 @@ The config must contain:
365367

366368
```json
367369
{
368-
"kubernetes": {
369-
"version": "<full-kubernetes-version>"
370+
"components": {
371+
"kubernetes": "<full-kubernetes-version>"
372+
},
373+
"networking": {
374+
"dnsServiceIP": "10.84.0.10"
370375
},
371376
"node": {
372377
"kubelet": {
373-
"serverURL": "https://<private-aks-fqdn>:443",
378+
"clusterFQDN": "<private-aks-fqdn>",
374379
"caCertData": "<base64-ca-data>",
375-
"dnsServiceIP": "10.84.0.10",
376380
"nodeIP": "<flex-vm-private-ip>"
377381
}
378382
},

‎docs/labs/aks-private-cluster-unbounded-net.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ AKS_REGION="<aks-region>"
142142
VM_REGION="<vm-region>"
143143
AKS_VNET="<aks-vnet-name>"
144144
FLEX_VNET="<flex-vnet-name>"
145+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
145146

146147
az account set --subscription "$SUBSCRIPTION_ID"
147148

@@ -395,6 +396,7 @@ chmod +x ./aks-flex-config
395396
--resource-group "$AKS_RG" \
396397
--cluster-name "$CLUSTER_NAME" \
397398
--subscription "$SUBSCRIPTION_ID" \
399+
--agent-pool-name "$AGENT_POOL_NAME" \
398400
--bootstrap-token \
399401
--output ./aks-flex-node-config.json
400402
```
@@ -406,7 +408,7 @@ TOKEN_ID=$(python3 -c 'import json; print(json.load(open("./aks-flex-node-config
406408
kubectl get secret -n kube-system "bootstrap-token-${TOKEN_ID}"
407409
```
408410

409-
For private clusters, if your workstation cannot reach the private API endpoint, run the RBAC/token creation from the admin VM and render the config from `az aks show` plus `az aks get-credentials --admin --file <path>`. The `kubernetes.version` value must be the full patch version, such as `1.34.7`, not the major/minor alias such as `1.34`; `aks-flex-node` uses this value to download Kubernetes binaries.
411+
For private clusters, if your workstation cannot reach the private API endpoint, run the RBAC/token creation from the admin VM and render the config from `az aks show` plus `az aks get-credentials --admin --file <path>`. The `components.kubernetes` value must be the full patch version, such as `1.34.7`, not the major/minor alias such as `1.34`; `aks-flex-node` uses this value to download Kubernetes binaries.
410412

411413
```bash
412414
KUBERNETES_VERSION=$(az aks show \
@@ -420,14 +422,16 @@ The config must contain:
420422

421423
```json
422424
{
423-
"kubernetes": {
424-
"version": "<full-kubernetes-version>"
425+
"components": {
426+
"kubernetes": "<full-kubernetes-version>"
427+
},
428+
"networking": {
429+
"dnsServiceIP": "10.74.0.10"
425430
},
426431
"node": {
427432
"kubelet": {
428-
"serverURL": "https://<private-aks-fqdn>:443",
433+
"clusterFQDN": "<private-aks-fqdn>",
429434
"caCertData": "<base64-ca-data>",
430-
"dnsServiceIP": "10.74.0.10",
431435
"nodeIP": "<flex-vm-private-ip>"
432436
}
433437
},

‎docs/labs/aks-public-cluster-unbounded-net-wireguard.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ AKS_REGION="eastus2"
9393
VM_REGION="southcentralus"
9494
AKS_VNET="aks-public-unbounded-vnet"
9595
FLEX_VNET="flex-public-unbounded-vnet"
96+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
9697

9798
az account set --subscription "$SUBSCRIPTION_ID"
9899

@@ -440,6 +441,7 @@ chmod +x ./aks-flex-config
440441
--resource-group "$AKS_RG" \
441442
--cluster-name "$CLUSTER_NAME" \
442443
--subscription "$SUBSCRIPTION_ID" \
444+
--agent-pool-name "$AGENT_POOL_NAME" \
443445
--bootstrap-token \
444446
--output ./aks-flex-node-config.json
445447
```
@@ -457,7 +459,7 @@ jq \
457459
--arg nodeIP "$VM_PRIVATE_IP" \
458460
--arg kubernetesVersion "$KUBERNETES_VERSION" \
459461
'.node.kubelet.nodeIP = $nodeIP
460-
| .kubernetes.version = $kubernetesVersion' \
462+
| .components.kubernetes = $kubernetesVersion' \
461463
./aks-flex-node-config.json > ./aks-flex-node-config.json.tmp
462464
mv ./aks-flex-node-config.json.tmp ./aks-flex-node-config.json
463465
```

‎docs/labs/gpu-node-setup.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,7 @@ On your workstation, use `aks-flex-config` to create the bootstrap RBAC and rend
118118
RESOURCE_GROUP="<aks-resource-group>"
119119
CLUSTER_NAME="<aks-cluster-name>"
120120
SUBSCRIPTION_ID="<subscription-id>"
121+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
121122

122123
curl -fsSLo ./aks-flex-config https://raw.githubusercontent.com/Azure/AKSFlexNode/main/scripts/aks-flex-config
123124
chmod +x ./aks-flex-config
@@ -131,6 +132,7 @@ chmod +x ./aks-flex-config
131132
--resource-group "$RESOURCE_GROUP" \
132133
--cluster-name "$CLUSTER_NAME" \
133134
--subscription "$SUBSCRIPTION_ID" \
135+
--agent-pool-name "$AGENT_POOL_NAME" \
134136
--bootstrap-token \
135137
--output ./aks-flex-node-config.json
136138
```

‎docs/usages/aks-flex-config.md‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ Most commands use the same AKS cluster selectors:
2626
RESOURCE_GROUP="<resource-group>"
2727
CLUSTER_NAME="<cluster-name>"
2828
SUBSCRIPTION_ID="<subscription-id>"
29+
AGENT_POOL_NAME="${AGENT_POOL_NAME:-aksflexnodes}"
2930
```
3031

3132
| Flag | Required | Description |
@@ -52,6 +53,11 @@ This applies the bootstrap-related `ClusterRoleBinding` objects for the `system:
5253
`generate-node-config` fetches AKS metadata and renders a config file. It requires exactly one auth mode.
5354

5455
Use `--output <path>` to write a config file with mode `0600`. If omitted, the config is written to stdout.
56+
The helper writes `azure.resourceManagerEndpoint` with the public ARM endpoint and writes the selected agent pool name to `azure.targetAgentPoolName`. If `--agent-pool-name` is omitted, it defaults to the historical `aksflexnodes` pool name.
57+
58+
| Flag | Required | Description |
59+
|------|----------|-------------|
60+
| `--agent-pool-name` | no | FlexNode agent pool name written into the generated config. Defaults to `aksflexnodes`. |
5561

5662
### Bootstrap Token
5763

@@ -60,11 +66,12 @@ Use `--output <path>` to write a config file with mode `0600`. If omitted, the c
6066
--resource-group "$RESOURCE_GROUP" \
6167
--cluster-name "$CLUSTER_NAME" \
6268
--subscription "$SUBSCRIPTION_ID" \
69+
--agent-pool-name "$AGENT_POOL_NAME" \
6370
--bootstrap-token \
6471
--output ./aks-flex-node-config.json
6572
```
6673

67-
Bootstrap-token mode creates a Kubernetes bootstrap token `Secret`, reads the AKS API server and CA data from kubeconfig, and includes those values in the generated config.
74+
Bootstrap-token mode creates a Kubernetes bootstrap token `Secret`, reads the AKS API server and CA data from kubeconfig, and includes those values plus the AKS DNS service IP in the generated config.
6875

6976
### Managed Identity
7077

@@ -73,6 +80,7 @@ Bootstrap-token mode creates a Kubernetes bootstrap token `Secret`, reads the AK
7380
--resource-group "$RESOURCE_GROUP" \
7481
--cluster-name "$CLUSTER_NAME" \
7582
--subscription "$SUBSCRIPTION_ID" \
83+
--agent-pool-name "$AGENT_POOL_NAME" \
7684
--identity \
7785
--output ./aks-flex-node-config.json
7886
```
@@ -84,6 +92,7 @@ For user-assigned managed identity, pass the client ID with `--username`:
8492
--resource-group "$RESOURCE_GROUP" \
8593
--cluster-name "$CLUSTER_NAME" \
8694
--subscription "$SUBSCRIPTION_ID" \
95+
--agent-pool-name "$AGENT_POOL_NAME" \
8796
--identity \
8897
--username "<managed-identity-client-id>" \
8998
--output ./aks-flex-node-config.json
@@ -98,6 +107,7 @@ Service principal flags follow the `az login --service-principal` convention:
98107
--resource-group "$RESOURCE_GROUP" \
99108
--cluster-name "$CLUSTER_NAME" \
100109
--subscription "$SUBSCRIPTION_ID" \
110+
--agent-pool-name "$AGENT_POOL_NAME" \
101111
--service-principal \
102112
--username "<client-id>" \
103113
--password "<client-secret>" \
@@ -114,6 +124,7 @@ Service principal flags follow the `az login --service-principal` convention:
114124
--resource-group "$RESOURCE_GROUP" \
115125
--cluster-name "$CLUSTER_NAME" \
116126
--subscription "$SUBSCRIPTION_ID" \
127+
--agent-pool-name "$AGENT_POOL_NAME" \
117128
--arc \
118129
--arc-machine-name "<arc-machine-name>" \
119130
--arc-resource-group "<arc-resource-group>" \

0 commit comments

Comments
 (0)