Skip to content

Commit 7c08a78

Browse files
deveshdamadeveshdama
authored
Merge pull request #4511 from deveshdama/master
Adding example for cert-manager letsencrypt for Istio-based service mesh
2 parents fcf8d44 + b524e18 commit 7c08a78

File tree

6 files changed

+377
-0
lines changed

6 files changed

+377
-0
lines changed

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/certificate.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: cert-manager.io/v1
2+
kind: Certificate
3+
metadata:
4+
name: bookinfo-letsencrypt-certs
5+
namespace: aks-istio-ingress
6+
spec:
7+
secretName: bookinfo-certs # letsencrypt certificate secret name, should match the credentialName in the gateway resource
8+
issuerRef:
9+
name: letsencrypt-prod
10+
kind: ClusterIssuer
11+
commonName: test.dev.azureservicemesh.io
12+
dnsNames:
13+
- test.dev.azureservicemesh.io # this should match a host name in the gateway resource

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/cluster-issuer.yaml

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
apiVersion: networking.k8s.io/v1
2+
kind: IngressClass
3+
metadata:
4+
name: istio
5+
spec:
6+
controller: istio.io/ingress-controller
7+
8+
---
9+
10+
apiVersion: cert-manager.io/v1
11+
kind: ClusterIssuer
12+
metadata:
13+
name: letsencrypt-prod
14+
spec:
15+
acme:
16+
server: https://acme-v02.api.letsencrypt.org/directory # the ACME server production URL
17+
email: [email protected] # email address used for ACME registration
18+
privateKeySecretRef: # name of a secret used to store the ACME account private key
19+
name: letsencrypt-prod
20+
solvers:
21+
- http01: # enable HTTP-01 challenge provider
22+
ingress:
23+
ingressClassName: istio

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/configmap.yaml

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: istio-shared-configmap-<asm-revision>
5+
namespace: aks-istio-system
6+
data:
7+
mesh: |-
8+
accessLogFile: /dev/stdout
9+
ingressService: aks-istio-ingressgateway-external
10+
ingressSelector: aks-istio-ingressgateway-external

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/gateway.yaml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
apiVersion: networking.istio.io/v1alpha3
2+
kind: Gateway
3+
metadata:
4+
name: bookinfo-gateway
5+
namespace: default
6+
spec:
7+
selector:
8+
istio: aks-istio-ingressgateway-external
9+
servers:
10+
- port:
11+
number: 80
12+
name: http
13+
protocol: HTTP
14+
hosts:
15+
- "test.dev.azureservicemesh.io" # this should match your DNS name
16+
- port:
17+
number: 443
18+
name: https-bookinfo
19+
protocol: HTTPS
20+
tls:
21+
mode: SIMPLE
22+
credentialName: "bookinfo-certs" # this should match the Certificate secretName
23+
hosts:
24+
- "test.dev.azureservicemesh.io" # this should match a DNS name in the Certificate

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/readme.md

+278
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
# Cert-Manager Let's Encrypt Integration with Istio-based service mesh add-on
2+
3+
This document contains instructions on how to integrate Istio-based service mesh add-on for AKS with cert-manager and obtain let's encrypt certificates for setting up secure ingress gateways.
4+
5+
## Objectives
6+
* Deploy bookinfo demo app, expose a secure HTTPS service using simple TLS.
7+
* Demonstrate HTTPS connections for Istio-based service mesh workloads using cert-manager and let's encrypt as the certificate authority.
8+
9+
> [!Note]
10+
> Kubernetes ingress for Istio-based service mesh is an `allowed` feature. More details on configuration options [here](https://learn.microsoft.com/en-us/azure/aks/istio-support-policy#allowed-supported-and-blocked-customizations)
11+
> [cert-manager](https://cert-manager.io/) and [let's encrypt](https://letsencrypt.org/) are not supported by Microsoft.
12+
13+
## Before you begin
14+
* [Install](https://learn.microsoft.com/en-us/azure/aks/istio-deploy-addon#install-istio-add-on) Istio-based service mesh add-on on your cluster.
15+
```shell
16+
az aks mesh enable -g <rg-name> -n <cluster-name>
17+
```
18+
* [Enable external ingressgateway](https://learn.microsoft.com/en-us/azure/aks/istio-deploy-ingress#enable-external-ingress-gateway)
19+
```shell
20+
az aks mesh enable-ingress-gateway -g <rg-name> -n <cluster-name> --ingress-gateway-type external
21+
```
22+
* [Enable sidecar injection](https://learn.microsoft.com/en-us/azure/aks/istio-deploy-addon#enable-sidecar-injection) on the default namespace.
23+
```shell
24+
revision=$(az aks show --resource-group <rg-name> --name <cluster-name> --query 'serviceMeshProfile.istio.revisions[0]' -o tsv)
25+
kubectl label namespace default istio.io/rev=$revision
26+
```
27+
28+
## Steps
29+
### 1. Setup DNS record
30+
Set up a DNS record for the `EXTERNAL-IP` address of the external ingressgateway service with your cloud provider.
31+
In this example, we [set up the DNS record](https://learn.microsoft.com/en-us/azure/dns/dns-operations-recordsets-portal) for `4.153.8.39` with `test.dev.azureservicemesh.io` on azure portal.
32+
33+
Run the following command to retrieve the external IP address of the ingress gateway:
34+
```shell
35+
kubectl get svc -n aks-istio-ingress
36+
```
37+
38+
```console
39+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
40+
aks-istio-ingressgateway-external LoadBalancer 10.0.59.31 4.153.8.39 15021:30786/TCP,80:30626/TCP,443:30236/TCP 8m44s
41+
```
42+
43+
Verify/wait until `dig +short A test.dev.azureservicemesh.io` returns the configured IP address that is `4.153.8.39` in this example.
44+
45+
### 2. Install demo app
46+
```shell
47+
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.21/samples/bookinfo/platform/kube/bookinfo.yaml
48+
```
49+
50+
Confirm several deployments and services are created on your cluster. For example:
51+
```console
52+
service/details created
53+
serviceaccount/bookinfo-details created
54+
deployment.apps/details-v1 created
55+
service/ratings created
56+
serviceaccount/bookinfo-ratings created
57+
deployment.apps/ratings-v1 created
58+
service/reviews created
59+
serviceaccount/bookinfo-reviews created
60+
deployment.apps/reviews-v1 created
61+
deployment.apps/reviews-v2 created
62+
deployment.apps/reviews-v3 created
63+
service/productpage created
64+
serviceaccount/bookinfo-productpage created
65+
deployment.apps/productpage-v1 created
66+
```
67+
```shell
68+
kubectl get pods
69+
```
70+
Ensure each pod has 2/2 containers in the `Ready` state
71+
```console
72+
NAME READY STATUS RESTARTS AGE
73+
details-v1-558b8b4b76-2llld 2/2 Running 0 2m41s
74+
productpage-v1-6987489c74-lpkgl 2/2 Running 0 2m40s
75+
ratings-v1-7dc98c7588-vzftc 2/2 Running 0 2m41s
76+
reviews-v1-7f99cc4496-gdxfn 2/2 Running 0 2m41s
77+
reviews-v2-7d79d5bd5d-8zzqd 2/2 Running 0 2m41s
78+
reviews-v3-7dbcdcbc56-m8dph 2/2 Running 0 2m41s
79+
```
80+
### 3. Configure ingress gateway and virtual service
81+
Before deploying the `virtualservice` and `gateway` resources, make sure to update the host name to match your own DNS name.
82+
```shell
83+
kubectl apply -f gateway.yaml
84+
kubectl apply -f virtualservice.yaml
85+
```
86+
87+
> [!NOTE]
88+
> In the gateway definition, `credentialName` must match the `secretName` in the `certificate` resource which will be created later in the example and selector must refer to the external ingress gateway by its label, in which the key of the label is `istio` and the value is `aks-istio-ingressgateway-external`.
89+
90+
### Validate http request to productpage service
91+
92+
Send an HTTP request to access the productpage service
93+
```shell
94+
curl -sS http://test.dev.azureservicemesh.io/productpage | grep -o "<title>.*</title>"
95+
```
96+
97+
this should print `<title>Simple Bookstore App</title>`
98+
99+
### 4. Create the shared configmap
100+
Create a ConfigMap with the name `istio-shared-configmap-<asm-revision>` in the `aks-istio-system` namespace to set `ingressService` and `ingressSelector`. For example, if your cluster is running asm-1-21 revision of mesh, then the ConfigMap needs to be named as istio-shared-configmap-asm-1-21. Mesh configuration has to be provided within the data section under mesh.
101+
102+
```shell
103+
cat configmap.yaml | sed -e "s/<asm-revision>/${revision}/" | kubectl apply -f -
104+
```
105+
106+
### 5. Install cert-manager
107+
```shell
108+
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.2/cert-manager.yaml
109+
```
110+
this installs cert-manager in its own namespace `cert-manager`
111+
112+
#### Validate cert-manager pods
113+
```shell
114+
kubectl get po -n cert-manager
115+
```
116+
```console
117+
NAME READY STATUS RESTARTS AGE
118+
cert-manager-6fd987499c-xmf44 1/1 Running 0 72s
119+
cert-manager-cainjector-5b94bd6f-jw644 1/1 Running 0 72s
120+
cert-manager-webhook-575479ff47-d87pf 1/1 Running 0 72s
121+
```
122+
123+
### 6. Setup cluster-issuer and Certificate resources
124+
Set your email address in `cluster-issuer.yaml` that you'd like to register with ACME server.
125+
```shell
126+
EMAIL=email
127+
cat cluster-issuer.yaml | sed -e "s/[email protected]/${EMAIL}/" | kubectl apply -f -
128+
kubectl apply -f certificate.yaml
129+
```
130+
This should create k8s secret `bookinfo-certs` in `aks-istio-ingress` namespace as requested by the certificate resource created above.
131+
132+
```shell
133+
kubectl get secret -n aks-istio-ingress
134+
```
135+
136+
```console
137+
NAME TYPE DATA AGE
138+
bookinfo-certs kubernetes.io/tls 2 70s
139+
sh.helm.release.v1.asm-igx-aks-istio-ingressgateway-external.v112 helm.sh/release.v1 1 3m19s
140+
sh.helm.release.v1.asm-igx-aks-istio-ingressgateway-external.v113 helm.sh/release.v1 1 80s
141+
```
142+
143+
### Validate https request to productpage service
144+
Verify that the productpage can be accessed via the HTTPS endpoint
145+
146+
```shell
147+
curl -sS https://test.dev.azureservicemesh.io/productpage | grep -o "<title>.*</title>"
148+
```
149+
this should print `<title>Simple Bookstore App</title>`
150+
151+
## Troubleshooting
152+
1. Verify that the DNS record has propagated
153+
```dig +short A test.dev.azureservicemesh.io``` should return the configured ip address ```4.153.8.39``` in the example.
154+
155+
2. Check certificate resource readiness.
156+
```shell
157+
kubectl get certificate -n aks-istio-ingress
158+
```
159+
```console
160+
NAME READY SECRET AGE
161+
bookinfo-letsencrypt-certs False bookinfo-certs 2m5s
162+
```
163+
In case certificate resource is not ready as above, check the resources provisioned by cert-manager in order to solve the ACME challenge. Below is the chain of resources created by cert-manager.
164+
165+
```shell
166+
kubectl describe certificate bookinfo-letsencrypt-certs -n aks-istio-ingress
167+
```
168+
```console
169+
Name: bookinfo-letsencrypt-certs
170+
Namespace: aks-istio-ingress
171+
Labels: <none>
172+
Annotations: <none>
173+
API Version: cert-manager.io/v1
174+
Kind: Certificate
175+
...
176+
Events:
177+
Type Reason Age From Message
178+
---- ------ ---- ---- -------
179+
Normal Issuing 14m cert-manager-certificates-trigger Issuing certificate as Secret does not exist
180+
Normal Generated 14m cert-manager-certificates-key-manager Stored new private key in temporary Secret resource "bookinfo-letsencrypt-certs-hpjst"
181+
Normal Requested 14m cert-manager-certificates-request-manager Created new CertificateRequest resource "bookinfo-letsencrypt-certs-1"
182+
```
183+
184+
3. Check certificaterequest resource readiness.
185+
```shell
186+
kubectl describe certificaterequest bookinfo-letsencrypt-certs-1 -n aks-istio-ingress
187+
```
188+
```console
189+
Name: bookinfo-letsencrypt-certs-1
190+
Namespace: aks-istio-ingress
191+
Labels: <none>
192+
Annotations: cert-manager.io/certificate-name: bookinfo-letsencrypt-certs
193+
cert-manager.io/certificate-revision: 1
194+
cert-manager.io/private-key-secret-name: bookinfo-letsencrypt-certs-hpjst
195+
API Version: cert-manager.io/v1
196+
Kind: CertificateRequest
197+
...
198+
Events:
199+
Type Reason Age From Message
200+
---- ------ ---- ---- -------
201+
Normal WaitingForApproval 6m57s cert-manager-certificaterequests-issuer-selfsigned Not signing CertificateRequest until it is Approved
202+
Normal WaitingForApproval 6m57s cert-manager-certificaterequests-issuer-vault Not signing CertificateRequest until it is Approved
203+
Normal WaitingForApproval 6m57s cert-manager-certificaterequests-issuer-venafi Not signing CertificateRequest until it is Approved
204+
Normal WaitingForApproval 6m57s cert-manager-certificaterequests-issuer-acme Not signing CertificateRequest until it is Approved
205+
Normal WaitingForApproval 6m57s cert-manager-certificaterequests-issuer-ca Not signing CertificateRequest until it is Approved
206+
Normal cert-manager.io 6m57s cert-manager-certificaterequests-approver Certificate request has been approved by cert-manager.io
207+
Normal OrderCreated 6m57s cert-manager-certificaterequests-issuer-acme Created Order resource aks-istio-ingress/bookinfo-letsencrypt-certs-1-507153367
208+
```
209+
210+
4. Check order resource readiness.
211+
```shell
212+
kubectl describe order bookinfo-letsencrypt-certs-1-507153367 -n aks-istio-ingress
213+
```
214+
```console
215+
Name: bookinfo-letsencrypt-certs-1-507153367
216+
Namespace: aks-istio-ingress
217+
Labels: <none>
218+
Annotations: cert-manager.io/certificate-name: bookinfo-letsencrypt-certs
219+
cert-manager.io/certificate-revision: 1
220+
cert-manager.io/private-key-secret-name: bookinfo-letsencrypt-certs-hpjst
221+
...
222+
Events:
223+
Type Reason Age From Message
224+
---- ------ ---- ---- -------
225+
Normal Created 11m cert-manager-orders Created Challenge resource "bookinfo-letsencrypt-certs-1-507153367-692923294" for domain "test.dev.azureservicemesh.io"
226+
```
227+
228+
5. Check challenge resource readiness.
229+
```shell
230+
kubectl describe challenge bookinfo-letsencrypt-certs-1-507153367-692923294 -n aks-istio-ingress
231+
```
232+
```console
233+
Name: bookinfo-letsencrypt-certs-1-507153367-692923294
234+
Namespace: aks-istio-ingress
235+
Labels: <none>
236+
Annotations: <none>
237+
API Version: acme.cert-manager.io/v1
238+
Kind: Challenge
239+
...
240+
Status:
241+
Presented: true
242+
Processing: true
243+
Reason: Waiting for HTTP-01 challenge propagation: wrong status code '404', expected '200'
244+
State: pending
245+
Events:
246+
Type Reason Age From Message
247+
---- ------ ---- ---- -------
248+
Normal Started 16m cert-manager-challenges Challenge scheduled for processing
249+
Normal Presented 15m cert-manager-challenges Presented challenge using HTTP-01 challenge mechanism
250+
```
251+
Status: You can refer to the 'Reason' field to further diagnose the root cause.
252+
253+
In the above example, HTTP-01 challenge propagation failed as `test.dev.azureservicemesh.io/.well-known/acme-challenge/hGsAXbL_uHSCL2tAvkh34d0AUmwuCfAge-ThVX0QfIA` could not be resolved and resulted in `404` instead of `200`.
254+
255+
6. Check ingress resource.
256+
cert-manager configures an ingress resource, sets up a node port service automatically to route the ACME challenge requests.
257+
```shell
258+
kubectl describe ing -n aks-istio-ingress
259+
```
260+
```console
261+
Name: cm-acme-http-solver-2lczl
262+
Labels: acme.cert-manager.io/http-domain=2715880231
263+
acme.cert-manager.io/http-token=1047727680
264+
acme.cert-manager.io/http01-solver=true
265+
Namespace: aks-istio-ingress
266+
Address:
267+
Ingress Class: istio
268+
Default backend: <default>
269+
Rules:
270+
Host Path Backends
271+
---- ---- --------
272+
test.dev.azureservicemesh.io
273+
/.well-known/acme-challenge/hGsAXbL_uHSCL2tAvkh34d0AUmwuCfAge-ThVX0QfIA cm-acme-http-solver-sddqg:8089 (10.244.0.24:8089)
274+
Annotations: nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
275+
Events: <none>
276+
```
277+
278+
Once the certificate has been served, all temporary resources are cleaned up automatically.

Diff for: ‎examples/istio-based-service-mesh/security/cert-manager-letsencrypt/virtualservice.yaml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
apiVersion: networking.istio.io/v1alpha3
2+
kind: VirtualService
3+
metadata:
4+
name: bookinfo-vs
5+
namespace: default
6+
spec:
7+
hosts:
8+
- test.dev.azureservicemesh.io # this should match your DNS name
9+
gateways:
10+
- bookinfo-gateway
11+
http:
12+
- match:
13+
- uri:
14+
exact: /
15+
- uri:
16+
exact: /productpage
17+
- uri:
18+
prefix: /static
19+
- uri:
20+
exact: /login
21+
- uri:
22+
exact: /logout
23+
- uri:
24+
prefix: /api/v1/products
25+
route:
26+
- destination:
27+
host: productpage
28+
port:
29+
number: 9080

0 commit comments

Comments
 (0)