Skip to content

Commit a9cb5f7

Browse files
Kelsey Hightowerkelseyhightower
Kelsey Hightower
authored and
kelseyhightower
committed
Remove cloud provider and move to ARM64
1 parent 79a3f79 commit a9cb5f7

37 files changed

+1255
-2109
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,4 @@ service-account.csr
4848
service-account.pem
4949
service-account-csr.json
5050
*.swp
51+
.idea/

‎README.md

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
# Kubernetes The Hard Way
22

3-
This tutorial walks you through setting up Kubernetes the hard way. This guide is not for people looking for a fully automated command to bring up a Kubernetes cluster. If that's you then check out [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine), or the [Getting Started Guides](https://kubernetes.io/docs/setup).
4-
5-
Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.
3+
This tutorial walks you through setting up Kubernetes the hard way. This guide is not for someone looking for a fully automated tool to bring up a Kubernetes cluster. Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.
64

75
> The results of this tutorial should not be viewed as production ready, and may receive limited support from the community, but don't let that stop you from learning!
86
@@ -13,24 +11,25 @@ Kubernetes The Hard Way is optimized for learning, which means taking the long r
1311

1412
## Target Audience
1513

16-
The target audience for this tutorial is someone planning to support a production Kubernetes cluster and wants to understand how everything fits together.
14+
The target audience for this tutorial is someone who wants to understand the fundamentals of Kubernetes and how the core components fit together.
1715

1816
## Cluster Details
1917

20-
Kubernetes The Hard Way guides you through bootstrapping a highly available Kubernetes cluster with end-to-end encryption between components and RBAC authentication.
18+
Kubernetes The Hard Way guides you through bootstrapping a basic Kubernetes cluster with all control plane components running on a single node, and two worker nodes, which is enough to learn the core concepts.
19+
20+
Component versions:
2121

22-
* [kubernetes](https://github.com/kubernetes/kubernetes) v1.21.0
23-
* [containerd](https://github.com/containerd/containerd) v1.4.4
24-
* [coredns](https://github.com/coredns/coredns) v1.8.3
25-
* [cni](https://github.com/containernetworking/cni) v0.9.1
26-
* [etcd](https://github.com/etcd-io/etcd) v3.4.15
22+
* [kubernetes](https://github.com/kubernetes/kubernetes) v1.28.x
23+
* [containerd](https://github.com/containerd/containerd) v1.7.x
24+
* [cni](https://github.com/containernetworking/cni) v1.3.x
25+
* [etcd](https://github.com/etcd-io/etcd) v3.4.x
2726

2827
## Labs
2928

30-
This tutorial assumes you have access to the [Google Cloud Platform](https://cloud.google.com). While GCP is used for basic infrastructure requirements the lessons learned in this tutorial can be applied to other platforms.
29+
This tutorial requires four (4) ARM64 based virtual or physical machines connected to the same network. While ARM64 based machines are used for the tutorial, the lessons learned can be applied to other platforms.
3130

3231
* [Prerequisites](docs/01-prerequisites.md)
33-
* [Installing the Client Tools](docs/02-client-tools.md)
32+
* [Setting up the Jumpbox](docs/02-jumpbox.md)
3433
* [Provisioning Compute Resources](docs/03-compute-resources.md)
3534
* [Provisioning the CA and Generating TLS Certificates](docs/04-certificate-authority.md)
3635
* [Generating Kubernetes Configuration Files for Authentication](docs/05-kubernetes-configuration-files.md)
@@ -40,6 +39,5 @@ This tutorial assumes you have access to the [Google Cloud Platform](https://clo
4039
* [Bootstrapping the Kubernetes Worker Nodes](docs/09-bootstrapping-kubernetes-workers.md)
4140
* [Configuring kubectl for Remote Access](docs/10-configuring-kubectl.md)
4241
* [Provisioning Pod Network Routes](docs/11-pod-network-routes.md)
43-
* [Deploying the DNS Cluster Add-on](docs/12-dns-addon.md)
44-
* [Smoke Test](docs/13-smoke-test.md)
45-
* [Cleaning Up](docs/14-cleanup.md)
42+
* [Smoke Test](docs/12-smoke-test.md)
43+
* [Cleaning Up](docs/13-cleanup.md)

‎ca.conf

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
[req]
2+
distinguished_name = req_distinguished_name
3+
prompt = no
4+
x509_extensions = ca_x509_extensions
5+
6+
[ca_x509_extensions]
7+
basicConstraints = CA:TRUE
8+
keyUsage = cRLSign, keyCertSign
9+
10+
[req_distinguished_name]
11+
C = US
12+
ST = Washington
13+
L = Seattle
14+
CN = CA
15+
16+
[admin]
17+
distinguished_name = admin_distinguished_name
18+
prompt = no
19+
req_extensions = default_req_extensions
20+
21+
[admin_distinguished_name]
22+
CN = admin
23+
O = system:masters
24+
25+
# Service Accounts
26+
#
27+
# The Kubernetes Controller Manager leverages a key pair to generate
28+
# and sign service account tokens as described in the
29+
# [managing service accounts](https://kubernetes.io/docs/admin/service-accounts-admin/)
30+
# documentation.
31+
32+
[service-accounts]
33+
distinguished_name = service-accounts_distinguished_name
34+
prompt = no
35+
req_extensions = default_req_extensions
36+
37+
[service-accounts_distinguished_name]
38+
CN = service-accounts
39+
40+
# Worker Nodes
41+
#
42+
# Kubernetes uses a [special-purpose authorization mode](https://kubernetes.io/docs/admin/authorization/node/)
43+
# called Node Authorizer, that specifically authorizes API requests made
44+
# by [Kubelets](https://kubernetes.io/docs/concepts/overview/components/#kubelet).
45+
# In order to be authorized by the Node Authorizer, Kubelets must use a credential
46+
# that identifies them as being in the `system:nodes` group, with a username
47+
# of `system:node:<nodeName>`.
48+
49+
[node-0]
50+
distinguished_name = node-0_distinguished_name
51+
prompt = no
52+
req_extensions = node-0_req_extensions
53+
54+
[node-0_req_extensions]
55+
basicConstraints = CA:FALSE
56+
extendedKeyUsage = clientAuth, serverAuth
57+
keyUsage = critical, digitalSignature, keyEncipherment
58+
nsCertType = client
59+
nsComment = "Node-0 Certificate"
60+
subjectAltName = DNS:node-0, IP:127.0.0.1
61+
subjectKeyIdentifier = hash
62+
63+
[node-0_distinguished_name]
64+
CN = system:node:node-0
65+
O = system:nodes
66+
C = US
67+
ST = Washington
68+
L = Seattle
69+
70+
[node-1]
71+
distinguished_name = node-1_distinguished_name
72+
prompt = no
73+
req_extensions = node-1_req_extensions
74+
75+
[node-1_req_extensions]
76+
basicConstraints = CA:FALSE
77+
extendedKeyUsage = clientAuth, serverAuth
78+
keyUsage = critical, digitalSignature, keyEncipherment
79+
nsCertType = client
80+
nsComment = "Node-1 Certificate"
81+
subjectAltName = DNS:node-1, IP:127.0.0.1
82+
subjectKeyIdentifier = hash
83+
84+
[node-1_distinguished_name]
85+
CN = system:node:node-1
86+
O = system:nodes
87+
C = US
88+
ST = Washington
89+
L = Seattle
90+
91+
92+
# Kube Proxy Section
93+
[kube-proxy]
94+
distinguished_name = kube-proxy_distinguished_name
95+
prompt = no
96+
req_extensions = kube-proxy_req_extensions
97+
98+
[kube-proxy_req_extensions]
99+
basicConstraints = CA:FALSE
100+
extendedKeyUsage = clientAuth, serverAuth
101+
keyUsage = critical, digitalSignature, keyEncipherment
102+
nsCertType = client
103+
nsComment = "Kube Proxy Certificate"
104+
subjectAltName = DNS:kube-proxy, IP:127.0.0.1
105+
subjectKeyIdentifier = hash
106+
107+
[kube-proxy_distinguished_name]
108+
CN = system:kube-proxy
109+
O = system:node-proxier
110+
C = US
111+
ST = Washington
112+
L = Seattle
113+
114+
115+
# Controller Manager
116+
[kube-controller-manager]
117+
distinguished_name = kube-controller-manager_distinguished_name
118+
prompt = no
119+
req_extensions = kube-controller-manager_req_extensions
120+
121+
[kube-controller-manager_req_extensions]
122+
basicConstraints = CA:FALSE
123+
extendedKeyUsage = clientAuth, serverAuth
124+
keyUsage = critical, digitalSignature, keyEncipherment
125+
nsCertType = client
126+
nsComment = "Kube Controller Manager Certificate"
127+
subjectAltName = DNS:kube-proxy, IP:127.0.0.1
128+
subjectKeyIdentifier = hash
129+
130+
[kube-controller-manager_distinguished_name]
131+
CN = system:kube-controller-manager
132+
O = system:kube-controller-manager
133+
C = US
134+
ST = Washington
135+
L = Seattle
136+
137+
138+
# Scheduler
139+
[kube-scheduler]
140+
distinguished_name = kube-scheduler_distinguished_name
141+
prompt = no
142+
req_extensions = kube-scheduler_req_extensions
143+
144+
[kube-scheduler_req_extensions]
145+
basicConstraints = CA:FALSE
146+
extendedKeyUsage = clientAuth, serverAuth
147+
keyUsage = critical, digitalSignature, keyEncipherment
148+
nsCertType = client
149+
nsComment = "Kube Scheduler Certificate"
150+
subjectAltName = DNS:kube-scheduler, IP:127.0.0.1
151+
subjectKeyIdentifier = hash
152+
153+
[kube-scheduler_distinguished_name]
154+
CN = system:kube-scheduler
155+
O = system:system:kube-scheduler
156+
C = US
157+
ST = Washington
158+
L = Seattle
159+
160+
161+
# API Server
162+
#
163+
# The Kubernetes API server is automatically assigned the `kubernetes`
164+
# internal dns name, which will be linked to the first IP address (`10.32.0.1`)
165+
# from the address range (`10.32.0.0/24`) reserved for internal cluster
166+
# services.
167+
168+
[kube-api-server]
169+
distinguished_name = kube-api-server_distinguished_name
170+
prompt = no
171+
req_extensions = kube-api-server_req_extensions
172+
173+
[kube-api-server_req_extensions]
174+
basicConstraints = CA:FALSE
175+
extendedKeyUsage = clientAuth, serverAuth
176+
keyUsage = critical, digitalSignature, keyEncipherment
177+
nsCertType = client
178+
nsComment = "Kube Scheduler Certificate"
179+
subjectAltName = @kube-api-server_alt_names
180+
subjectKeyIdentifier = hash
181+
182+
[kube-api-server_alt_names]
183+
IP.0 = 127.0.0.1
184+
IP.1 = 10.32.0.1
185+
DNS.0 = kubernetes
186+
DNS.1 = kubernetes.default
187+
DNS.2 = kubernetes.default.svc
188+
DNS.3 = kubernetes.default.svc.cluster
189+
DNS.4 = kubernetes.svc.cluster.local
190+
DNS.5 = server.kubernetes.local
191+
DNS.6 = api-server.kubernetes.local
192+
193+
[kube-api-server_distinguished_name]
194+
CN = kubernetes
195+
C = US
196+
ST = Washington
197+
L = Seattle
198+
199+
200+
[default_req_extensions]
201+
basicConstraints = CA:FALSE
202+
extendedKeyUsage = clientAuth
203+
keyUsage = critical, digitalSignature, keyEncipherment
204+
nsCertType = client
205+
nsComment = "Admin Client Certificate"
206+
subjectKeyIdentifier = hash

‎configs/10-bridge.conf

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"cniVersion": "1.0.0",
3+
"name": "bridge",
4+
"type": "bridge",
5+
"bridge": "cni0",
6+
"isGateway": true,
7+
"ipMasq": true,
8+
"ipam": {
9+
"type": "host-local",
10+
"ranges": [
11+
[{"subnet": "SUBNET"}]
12+
],
13+
"routes": [{"dst": "0.0.0.0/0"}]
14+
}
15+
}

‎configs/99-loopback.conf

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"cniVersion": "1.1.0",
3+
"name": "lo",
4+
"type": "loopback"
5+
}

‎configs/containerd-config.toml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
version = 2
2+
3+
[plugins."io.containerd.grpc.v1.cri"]
4+
[plugins."io.containerd.grpc.v1.cri".containerd]
5+
snapshotter = "overlayfs"
6+
default_runtime_name = "runc"
7+
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
8+
runtime_type = "io.containerd.runc.v2"
9+
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
10+
SystemdCgroup = true
11+
[plugins."io.containerd.grpc.v1.cri".cni]
12+
bin_dir = "/opt/cni/bin"
13+
conf_dir = "/etc/cni/net.d"

‎configs/kube-apiserver-to-kubelet.yaml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
apiVersion: rbac.authorization.k8s.io/v1
2+
kind: ClusterRole
3+
metadata:
4+
annotations:
5+
rbac.authorization.kubernetes.io/autoupdate: "true"
6+
labels:
7+
kubernetes.io/bootstrapping: rbac-defaults
8+
name: system:kube-apiserver-to-kubelet
9+
rules:
10+
- apiGroups:
11+
- ""
12+
resources:
13+
- nodes/proxy
14+
- nodes/stats
15+
- nodes/log
16+
- nodes/spec
17+
- nodes/metrics
18+
verbs:
19+
- "*"
20+
---
21+
apiVersion: rbac.authorization.k8s.io/v1
22+
kind: ClusterRoleBinding
23+
metadata:
24+
name: system:kube-apiserver
25+
namespace: ""
26+
roleRef:
27+
apiGroup: rbac.authorization.k8s.io
28+
kind: ClusterRole
29+
name: system:kube-apiserver-to-kubelet
30+
subjects:
31+
- apiGroup: rbac.authorization.k8s.io
32+
kind: User
33+
name: kubernetes

‎configs/kube-proxy-config.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
kind: KubeProxyConfiguration
2+
apiVersion: kubeproxy.config.k8s.io/v1alpha1
3+
clientConnection:
4+
kubeconfig: "/var/lib/kube-proxy/kubeconfig"
5+
mode: "iptables"
6+
clusterCIDR: "10.200.0.0/16"

‎configs/kube-scheduler.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
apiVersion: kubescheduler.config.k8s.io/v1
2+
kind: KubeSchedulerConfiguration
3+
clientConnection:
4+
kubeconfig: "/var/lib/kubernetes/kube-scheduler.kubeconfig"
5+
leaderElection:
6+
leaderElect: true

‎configs/kubelet-config.yaml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
kind: KubeletConfiguration
2+
apiVersion: kubelet.config.k8s.io/v1beta1
3+
authentication:
4+
anonymous:
5+
enabled: false
6+
webhook:
7+
enabled: true
8+
x509:
9+
clientCAFile: "/var/lib/kubelet/ca.crt"
10+
authorization:
11+
mode: Webhook
12+
clusterDomain: "cluster.local"
13+
clusterDNS:
14+
- "10.32.0.10"
15+
cgroupDriver: systemd
16+
containerRuntimeEndpoint: "unix:///var/run/containerd/containerd.sock"
17+
podCIDR: "SUBNET"
18+
resolvConf: "/etc/resolv.conf"
19+
runtimeRequestTimeout: "15m"
20+
tlsCertFile: "/var/lib/kubelet/kubelet.crt"
21+
tlsPrivateKeyFile: "/var/lib/kubelet/kubelet.key"

0 commit comments

Comments
 (0)