Skip to content

Commit f518034

Browse files
rajsinghtechrajsinghtech
committed
Add BYOD GatewayAPI Tailscale post
Add new blog post content about integrating BYOD (Bring Your Own Device) with Gateway API and Tailscale. This post will cover implementation details and configuration for this networking setup.
1 parent 2741a1d commit f518034

File tree

1 file changed

+302
-0
lines changed
  • content/post/byod-gatewayapi-tailscale

1 file changed

+302
-0
lines changed

‎content/post/byod-gatewayapi-tailscale/index.md‎

Lines changed: 302 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,302 @@
1+
---
2+
title: "BYO Domain Gateway API Tailscale Operator"
3+
date: 2025-05-21 00:00:00+0000
4+
draft: false
5+
tags: ["Kubernetes", "Tailscale", "GatewayAPI", "Networking", "DevOps"]
6+
categories: ["Kubernetes", "Tailscale"]
7+
---
8+
9+
While Tailscale excels at making services accessible via its managed `*.ts.net` domain names, using your own custom domain (like `hello.example.com`) for services exposed through the Tailscale Kubernetes operator requires a different approach. Tailscale itself doesn't manage DNS records or issue TLS certificates for domains it doesn't control. This guide presents a robust solution to this by integrating the Kubernetes Gateway API (specifically with Envoy Gateway) with ExternalDNS and CertManager. This combination allows you to seamlessly use your custom domains for services on your tailnet, complete with automated DNS and TLS management.
10+
11+
To showcase this, we will utilize (EnvoyGateway)[https://gateway.envoyproxy.io/docs/] in conjunction with Tailscale Operator.
12+
This setup allows ExternalDNS to manage records for your custom domain in an internal DNS server, which Tailscale's MagicDNS then uses to resolve these names across your tailnet.
13+
14+
## Prerequisites
15+
16+
Ensure you have the following components set up and configured:
17+
18+
* **Kubernetes Cluster:** A running Kubernetes cluster.
19+
* **Tailscale Account:** With administrative privileges for DNS configuration.
20+
* **Tailscale Kubernetes Operator:** Installed in your cluster. ([Official Guide](https://tailscale.com/kb/1185/kubernetes/)).
21+
* **Envoy Gateway:** Installed as your Gateway API implementation. ([Documentation](https://gateway.envoyproxy.io/docs/)).
22+
* **ExternalDNS:** Deployed and configured to manage records in your internal DNS provider. This creates DNS entries for services like `hello.example.com`. ([GitHub Repository](https://github.com/kubernetes-sigs/external-dns)).
23+
* **CertManager:** Installed to automate TLS certificate issuance and renewal. ([Documentation](https://cert-manager.io/docs/)).
24+
* **Internal DNS Server:** Such as CoreDNS, BIND, or a cloud provider's private DNS (e.g., AWS Route 53 private hosted zones, Google Cloud DNS private zones).
25+
* Must be accessible from your Tailscale nodes (e.g., via a Tailscale subnet router).
26+
* ExternalDNS requires permissions to manage records in this server for your domain (`example.com`).
27+
28+
## Configuring Operator Hostname and API Server Proxy
29+
30+
When installing or upgrading the Tailscale Kubernetes operator, configuring it with OAuth credentials (client ID and client secret) is crucial. These allow the operator to authenticate with your Tailscale account, enabling it to manage resources like exposing services and updating device information in your tailnet.
31+
32+
Use the following Helm command, replacing placeholders with your Tailscale OAuth client ID and secret:
33+
34+
```bash
35+
helm upgrade --install tailscale-operator tailscale/tailscale-operator \\
36+
--namespace=tailscale \\
37+
--create-namespace \\
38+
--set-string oauth.clientId=<oauth_client_id> \\
39+
--set-string oauth.clientSecret=<oauth_client_secret> \\
40+
--wait
41+
```
42+
43+
## Configuring Tailscale MagicDNS for Your Custom Domain
44+
45+
To make `hello.example.com` (and other services on `example.com`) resolvable within your tailnet to the Envoy Gateway, configure Tailscale's MagicDNS with a split DNS setup. This directs DNS queries for `*.example.com` from Tailscale devices to your internal DNS server, which ExternalDNS keeps updated.
46+
47+
1. **Identify Your Internal DNS Server:**
48+
You need an internal DNS server that ExternalDNS is configured to manage, holding records for `example.com`. Ensure it's accessible from Tailscale nodes (potentially via a [subnet router](https://tailscale.com/kb/1019/subnets/)). Note its IP address.
49+
50+
2. **Configure Tailscale DNS for Split DNS:**
51+
* Log in to your [Tailscale admin console](https://login.tailscale.com/admin/dns).
52+
* Navigate to **DNS**.
53+
* In "Nameservers," click "Add Nameserver" and select **Custom**.
54+
* Input the IP address of your internal DNS server.
55+
* Enable **Restrict to search domain** (or similar).
56+
* For the search domain, enter your custom domain (e.g., `example.com`). This tells Tailscale to use your internal DNS server *only* for this domain.
57+
* Save changes.
58+
59+
With this setup, when a Tailscale device accesses `hello.example.com`, MagicDNS forwards the DNS query to your internal DNS server. ExternalDNS, in your Kubernetes cluster, maintains the DNS record (e.g., an A or CNAME for `hello.example.com`) in this server, pointing to the Tailscale IP or hostname of your Envoy Gateway service (e.g., `common-envoy-ts.your-tailnet-name.ts.net`).
60+
61+
## Create Gateway
62+
63+
Define the Kubernetes Gateway API resources to configure an Envoy-based gateway, integrate it with Tailscale, and serve traffic for your custom domain.
64+
65+
### 1. Define EnvoyProxy and GatewayClass
66+
67+
The `EnvoyProxy` resource configures the underlying Envoy proxy instances:
68+
* `tailscale.com/hostname: common-envoy-ts` (in `provider.kubernetes.envoyService.annotations`): Instructs the Tailscale operator to assign this DNS name (e.g., `common-envoy-ts.your-tailnet.ts.net`) to Envoy's LoadBalancer service.
69+
* `loadBalancerClass: tailscale` (in `provider.kubernetes.envoyService`): Tells Kubernetes to use Tailscale for provisioning Envoy's LoadBalancer service.
70+
71+
The `GatewayClass` links this `EnvoyProxy` configuration to the Gateway API.
72+
73+
``` yaml
74+
apiVersion: gateway.envoyproxy.io/v1alpha1
75+
kind: EnvoyProxy
76+
metadata:
77+
name: ts
78+
namespace: tailscale
79+
spec:
80+
provider:
81+
type: Kubernetes
82+
kubernetes:
83+
envoyService:
84+
annotations:
85+
tailscale.com/hostname: common-envoy-ts
86+
type: LoadBalancer
87+
loadBalancerClass: tailscale
88+
---
89+
apiVersion: gateway.networking.k8s.io/v1
90+
kind: GatewayClass
91+
metadata:
92+
name: ts
93+
spec:
94+
controllerName: gateway.envoyproxy.io/gatewayclass-controller
95+
parametersRef:
96+
group: gateway.envoyproxy.io
97+
kind: EnvoyProxy
98+
name: ts
99+
namespace: tailscale
100+
```
101+
102+
### 2. Define the Gateway
103+
104+
The `Gateway` resource represents an instance of your gateway listening for traffic:
105+
106+
* `external-dns: example` (label): Can be used by ExternalDNS to identify Gateways to process (value is customizable).
107+
* `cert-manager.io/cluster-issuer: example-issuer` (annotation): Tells CertManager to use this ClusterIssuer for TLS certificates.
108+
* `gatewayClassName: ts`: Links to the `GatewayClass` defined above.
109+
* `listeners`: Defines how the gateway listens:
110+
* `protocol: HTTPS`, `port: 443`: Listens for HTTPS on port 443.
111+
* `hostname: "hello.example.com"`: Handles requests only for this hostname.
112+
* `tls.mode: Terminate`: Gateway terminates TLS.
113+
* `tls.certificateRefs`: Points to the Kubernetes Secret (e.g., `hello-example-https-tls`) where CertManager stores the certificate.
114+
115+
``` yaml
116+
---
117+
apiVersion: gateway.networking.k8s.io/v1
118+
kind: Gateway
119+
metadata:
120+
name: ts
121+
namespace: tailscale
122+
labels:
123+
external-dns: example
124+
annotations:
125+
cert-manager.io/cluster-issuer: example-issuer
126+
spec:
127+
gatewayClassName: ts
128+
listeners:
129+
- name: hello-example-https
130+
protocol: HTTPS
131+
port: 443
132+
hostname: "hello.example.com"
133+
allowedRoutes:
134+
namespaces:
135+
from: All
136+
tls:
137+
mode: Terminate
138+
certificateRefs:
139+
- group: ''
140+
kind: Secret
141+
name: hello-example-https-tls
142+
```
143+
144+
## Define an HTTPRoute to Direct Traffic
145+
146+
The `HTTPRoute` resource defines how HTTP requests are routed from the Gateway to backend services:
147+
148+
* `parentRefs`: Links this `HTTPRoute` to our `Gateway` (`ts` in `tailscale` namespace).
149+
* `hostnames`: Specifies this route applies to requests for `hello.example.com`.
150+
* `rules`: Defines routing logic:
151+
* `backendRefs`: Forwards traffic to the `hello-world` Kubernetes Service (port `80`).
152+
* `matches`: Matches requests where the path starts with `/` (all requests for the hostname).
153+
154+
``` YAML
155+
apiVersion: gateway.networking.k8s.io/v1
156+
kind: HTTPRoute
157+
metadata:
158+
name: hello-ts
159+
spec:
160+
parentRefs:
161+
- group: gateway.networking.k8s.io
162+
kind: Gateway
163+
name: ts
164+
namespace: tailscale
165+
hostnames:
166+
- "hello.example.com"
167+
rules:
168+
- backendRefs:
169+
- group: ""
170+
kind: Service
171+
name: hello-world
172+
port: 80
173+
weight: 1
174+
matches:
175+
- path:
176+
type: PathPrefix
177+
value: /
178+
```
179+
180+
This setup enables ExternalDNS to populate a DNS record for `hello.example.com` in your internal DNS server (used by Tailscale's split DNS). CertManager procures an HTTPS certificate, and the HTTPRoute directs traffic for `hello.example.com` to your `hello-world` service via the Envoy Gateway.
181+
182+
## Example: Using Pi-hole as Internal DNS with ExternalDNS
183+
184+
Pi-hole running in Kubernetes can serve as the internal DNS for this setup, providing ad blocking and local DNS resolution. ExternalDNS can be configured to use Pi-hole as a provider. This example assumes `helm` is installed.
185+
186+
### 1. Add Helm Repositories
187+
188+
```bash
189+
helm repo add pihole https://mojo2600.github.io/pihole-kubernetes/
190+
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns
191+
helm repo update
192+
```
193+
194+
### 2. Create Pi-hole Admin Secret
195+
196+
Pi-hole and ExternalDNS need an admin password. Create a Kubernetes secret (replace `YOUR_PIHOLE_PASSWORD`):
197+
198+
```bash
199+
kubectl create secret generic pihole-admin-secret --from-literal=password='YOUR_PIHOLE_PASSWORD' -n pihole-ns # Ensure pihole-ns exists or use --create-namespace
200+
```
201+
202+
### 3. Install Pi-hole
203+
204+
Create `pihole-values.yaml`. This exposes Pi-hole's DNS (port 53) as a Tailscale LoadBalancer service with a specific Tailscale FQDN.
205+
206+
```yaml
207+
# pihole-values.yaml
208+
admin:
209+
existingSecret: "pihole-admin-secret"
210+
passwordKey: password
211+
212+
serviceDns:
213+
type: LoadBalancer
214+
loadBalancerClass: tailscale
215+
port: 53
216+
annotations:
217+
# USER: Replace with your desired Tailscale FQDN for Pi-hole DNS
218+
"tailscale.com/tailnet-fqdn": "pihole-dns.your-tailnet-name.ts.net"
219+
220+
serviceWeb:
221+
type: ClusterIP # Keep web UI internal
222+
223+
ingressWeb:
224+
enabled: false # Disable Pi-hole's ingress if managing access differently
225+
226+
podDnsConfig:
227+
enabled: false # Avoid conflicts with cluster DNS
228+
```
229+
230+
Install Pi-hole (ensure `pihole-ns` namespace exists or use `--create-namespace`):
231+
232+
```bash
233+
helm install pihole pihole/pihole --version 2.31.0 \
234+
-n pihole-ns --create-namespace \
235+
-f pihole-values.yaml
236+
```
237+
Pi-hole's DNS service will be available at the FQDN specified (e.g., `pihole-dns.your-tailnet-name.ts.net`).
238+
239+
### 4. Install ExternalDNS for Pi-hole
240+
241+
Create `external-dns-pihole-values.yaml` to configure ExternalDNS for your Pi-hole deployment.
242+
243+
```yaml
244+
# external-dns-pihole-values.yaml
245+
fullnameOverride: external-dns-pihole
246+
247+
logLevel: debug # Or info
248+
provider: pihole
249+
250+
env:
251+
- name: EXTERNAL_DNS_PIHOLE_PASSWORD
252+
valueFrom:
253+
secretKeyRef:
254+
name: pihole-admin-secret # Match secret name for Pi-hole
255+
key: password # Match key in secret
256+
257+
extraArgs:
258+
# USER: Adjust Pi-hole server URL if service name/namespace differs.
259+
# Points to Pi-hole web admin (port 80 internally).
260+
# Assumes Pi-hole in 'pihole-ns', release 'pihole' (service: 'pihole-web').
261+
- --pihole-server=http://pihole-web.pihole-ns
262+
# USER: Customize label selector to match your Gateway resources.
263+
- --gateway-label-filter=external-dns==example # Matches label in main guide's Gateway
264+
265+
policy: sync # Or "upsert-only"
266+
267+
sources:
268+
- gateway-httproute # For hostnames in HTTPRoutes attached to labeled Gateways
269+
# - service # To create DNS for annotated K8s services
270+
# - ingress # If using Ingress resources
271+
272+
# USER: Define domain(s) for ExternalDNS to manage in Pi-hole (e.g., "example.com").
273+
domainFilters:
274+
- "example.com"
275+
# - "another.internal.domain"
276+
277+
# USER: Customize for TXT record identification.
278+
txtOwnerId: "my-k8s-cluster-pihole"
279+
txtPrefix: "k8s-edns-"
280+
```
281+
282+
Install ExternalDNS (ensure `external-dns-ns` namespace exists or use `--create-namespace`):
283+
284+
```bash
285+
# Check for the latest stable ExternalDNS chart version
286+
helm install external-dns external-dns/external-dns \
287+
-n external-dns-ns --create-namespace \
288+
-f external-dns-pihole-values.yaml
289+
```
290+
291+
### 5. Configure Tailscale Split DNS for Pi-hole
292+
293+
Tell your Tailscale network to use Pi-hole for resolving your custom domain(s) (e.g., `example.com`):
294+
295+
1. Go to **DNS** in your [Tailscale admin console](https://login.tailscale.com/admin/dns).
296+
2. Under "Nameservers," add a **Custom** nameserver.
297+
3. Enter Pi-hole's Tailscale IP or FQDN (e.g., `pihole-dns.your-tailnet-name.ts.net`).
298+
4. Enable **Restrict to search domain**.
299+
5. For the search domain, enter the domain(s) ExternalDNS manages via Pi-hole (e.g., `example.com`).
300+
6. Save changes.
301+
302+
Now, when your Gateway (e.g., for `hello.example.com`) is created with the label `external-dns: example`, ExternalDNS will add its DNS record to Pi-hole. Tailscale clients querying `hello.example.com` will use Pi-hole, which resolves it to your Envoy Gateway's Tailscale IP (`common-envoy-ts`).

0 commit comments

Comments
 (0)