Skip to content

DNS integration #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dovholuknf merged 1 commit into from
Aug 13, 2026
Merged

DNS integration #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 67 additions & 2 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# Architecture

## Overview

![Full-tunnel travel router overview](travel-router-overview.svg)

The full-tunnel travel router carries all client traffic -- except the OpenZiti control/data underlay and the local
uplink -- through the overlay to a chosen exit, with a hard rule that the internet and DNS must never stop. Three
on-device pieces enforce that: `ziti-edge-tunnel` (ZET) is the dataplane; `ziti-guard` is a continuous watchdog that
falls open to direct internet if the tunnel stops carrying data; and the boot guard refuses to start the tunnel when
the controller is unreachable. DNS is wired in additively so general resolution never depends on the tunnel.

## Components

```
Expand Down Expand Up @@ -28,6 +38,27 @@ Each daemon owns its own UCI file and init script. The LuCI app drives ZET only;
the router is configured out-of-band today (enroll via `ziti router enroll`),
with a future LuCI tab planned.

## Resilience + DNS pieces

Three helpers wrap ZET so full-tunnel mode can never black-hole the wifi, all living in
`/usr/libexec/ziti-boot-guard` (busybox sh) plus one extra procd service:

- **Boot guard (`ziti-boot-guard preflight`)** -- runs inside the ZET init before starting the daemon. It refreshes
the `/etc/hosts` controller pins (every `ztAPI` + `ztAPIs[]` across enabled identities, re-resolved via direct DNS,
in a marker-delimited block) and confirms a controller is reachable over the current uplink. If not, it falls open
(restore `lan->wan`, flush any stray `/1` routes) and does NOT start ZET -- so the `/1` intercept never installs when
the tunnel cannot work. `refresh-hosts` also runs on enroll.
- **Continuous watchdog (`ziti-guard` service -> `ziti-boot-guard watchdog`)** -- a SEPARATE procd service (so its own
fall-open, which stops ZET, cannot kill it). It acts whenever a wildcard `/1` intercept is actually live on `ziti0`
(route-based, not a UCI flag), probing egress through the tunnel every `watchdog_interval`; after `watchdog_fails`
consecutive failures it runs `fall_open` (stop ZET, flush `0.0.0.0/1`+`128.0.0.0/1`, restore `lan->wan`) and STAYS
open. This covers the mid-session case an exit dying while the tunnel is up.
- **DNS integration (`ziti-boot-guard dns-sync`)** -- programs dnsmasq to forward chosen domains to ZET's embedded
resolver while leaving dnsmasq's own default upstream untouched (see the DNS design decision below).

`fall_open` is the single owner of every failure transition. Failure state is surfaced to LuCI status
(`guard_state`, `boot_failures`).

## UCI schema

`/etc/config/ziti` (ZET):
Expand All @@ -36,6 +67,17 @@ with a future LuCI tab planned.
config ziti 'main'
option enabled '1'
option log_level 'INFO' # ERROR | WARN | INFO | DEBUG | TRACE | VERBOSE
# resilience watchdog (all optional; guard defaults apply if unset)
option boot_verify '1'
option max_boot_failures '3'
list watchdog_probes '1.1.1.1'
option watchdog_interval '10'
option watchdog_fails '3'
# DNS integration (optional; off unless ziti_dns_domains is set)
list ziti_dns_domains 'ziti'
list ziti_dns_domains 'parkplace-via-dhcp'
option dns_upstream '192.168.1.5' # home resolver, reached over the tunnel
option dns_resolver_ip '100.64.0.2' # ZET's resolver (the .2 of the DNS range)

config identity
option name 'home'
Expand Down Expand Up @@ -73,10 +115,13 @@ config router 'main'
/usr/bin/ziti-edge-tunnel ZET binary
/usr/bin/ziti router multi-call binary
/usr/bin/ziti-router symlink -> ziti
/etc/init.d/ziti-edge-tunnel procd init for ZET
/etc/init.d/ziti-edge-tunnel procd init for ZET (preflight-gates, runs dns-sync, starts ziti-guard)
/etc/init.d/ziti-router procd init for router
/etc/init.d/ziti-guard procd service: continuous fail-open watchdog
/usr/libexec/rpcd/ziti rpcd backend for LuCI
/usr/libexec/ziti-boot-guard full-tunnel boot safeguard (called by ZET init + rpcd enroll)
/usr/libexec/ziti-boot-guard boot safeguard + watchdog + dns-sync (busybox sh)
/etc/ziti/boot-guard.state last guard verdict (surfaced in LuCI status)
/etc/ziti/autostart-failures consecutive boot-failure counter
/usr/share/rpcd/acl.d/ rpcd ACLs
/www/luci-static/resources/view/ziti/ LuCI views
```
Expand Down Expand Up @@ -107,3 +152,23 @@ depends on an external source); this note captures *why the travel-router path h

The full-tunnel runbook and its gotchas live in `docs/full-tunnel-travel-router.md` and
`docs/how-it-works-and-gotchas.md`.

### DNS integration: additive, never a single point of failure

Hard rule: general DNS must never break. DNS failures are the most visible outage a user can hit, so no design is
acceptable if normal name resolution can depend on a component that might fail (ZET, the tunnel, the exit).

- **dnsmasq keeps its own direct default resolver, untouched.** We only ADD per-domain forwarding of chosen domains
to ZET's resolver (`server=/<domain>/<resolver-ip>`). If ZET/the tunnel is down, only those domains fail; everything
else resolves via dnsmasq's default. There is no coupling to the watchdog/fall-open.
- **ZET's embedded resolver is at the `.2` of the DNS range (100.64.0.2 for the default 100.64.0.0/10), not `.1`.**
`.1` is the tun's own local IP, delivered locally to nothing; `.2` routes into `ziti0` and is what ZET adds to
`/etc/resolv.conf`. dnsmasq must forward there. `dns-sync` also adds `notinterface ziti0` so dnsmasq stops binding
ziti0 and squatting the resolver address.
- **Home-vantage without a fragile default.** ZET is started with `--dns-upstream <home resolver>` (e.g. a home
pi-hole on a home LAN IP). Non-Ziti names hitting ZET's resolver are forwarded there; because that IP is inside the
wildcard intercept, the query rides the tunnel and resolves at home. Ziti service names resolve to synthetic
`100.64.x` and tunnel directly. Public names not in the forwarded domains stay on dnsmasq's direct default.

Rejected alternative: pointing dnsmasq's ONLY upstream at ZET (to send all DNS home). That makes a ZET death a full
DNS outage -- exactly the single point of failure the hard rule forbids.
57 changes: 49 additions & 8 deletions docs/blog-series-idea.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The through-line: a device with NO OpenZiti software on it -- your phone, a work
run and behaves as if it is sitting on your home network, egressing from your home ISP, from anywhere in the world.
That is the hook. The overlay is invisible to the client; the travel router does all the work.

## Series arc (5 parts)
## Series arc (7 parts)

### Part 1 -- "Appear at home from anywhere: why a travel router, and why not a VPN"
- Hook: your home NAS, your internal dashboards, and the bank that flags foreign logins all behave as if you never
Expand All @@ -27,10 +27,12 @@ That is the hook. The overlay is invisible to the client; the travel router does
(changing the uplink with the intercept live means ZET cannot re-reach the controller -- it catches its own control
channel before the tunnel exists). The exit-node dead ends: the M1 mini whose service kept restarting, and Ziti
Desktop Edge for Windows which will host a service only after a manual disable/enable and then drops it on reboot.
And the one that reshaped the design: the exit terminator dying MID-SESSION -- the tunnel stayed up, the router kept
routing every client into it, and all traffic black-holed until a manual stop (the automatic fix is Part 4).
- Show: the exact black-hole sequence and the recovery (LAN-side SSH survives because it is not forwarded; reboot is
clean because boot-autostart was disabled). The "connected but no internet" symptom on an isolated network.
- Aha: with a global wildcard intercept there is no "just test it" -- stage the firewall and a controller pin first,
or you take yourself offline. Fail loud, fail closed, keep an out-of-band way in.
or you take yourself offline. Fail loud, fail open, keep an out-of-band way in.

### Part 3 -- "The architecture that worked: ZET, host.v1, and proving it with fabric events"
- Beat: the design that survived. ZET (the C tunneler, TUN + lwIP) as the gateway on the Slate -- NOT the ziti-router
Expand All @@ -43,9 +45,42 @@ That is the hook. The overlay is invisible to the client; the travel router does
serviceId=internet-exit-svc, and the `connect` event showing the exit's home source address.
- Aha: attribute-based policies mean you add a new traveling device by tagging an identity -- no policy edits ever.

### Part 4 -- "Making it usable: a LuCI app, a signed feed, and self-update"
- Beat: turning a pile of SSH commands into something a human operates. The luci-app-ziti tabs (Status, Identities,
Tunnel Mode with a full/split toggle that flips identities + firewall, Settings with a signed self-update). The
### Part 4 -- "Never black-hole the wifi: the resilience watchdog and fail-open"
- Hook: at a cowork desk the exit died mid-session and every client lost the internet -- the router happily kept
routing everyone into a tunnel that no longer went anywhere. Nobody wants to debug that from a coffee shop.
- Beat: the hard rule -- the wifi, and especially DNS, must NEVER stop; every failure path falls OPEN to plain direct
internet, never closed into a black-hole. Two safeguards enforce it. A boot-time preflight gate: before starting the
tunnel, refresh and pin every controller (ztAPI + ztAPIs[]) in /etc/hosts and prove one is reachable over the current
uplink -- if not, do not install the wildcard routes at all, so a dead controller or captive portal cannot black-hole
the wifi. A continuous watchdog running as its OWN procd service (so its own teardown cannot kill it): whenever the
wildcard /1 intercept is actually live on ziti0, it probes egress THROUGH the tunnel every few seconds and, after a
few consecutive failures, tears the tunnel down, restores direct internet, and stays open.
- Show: the log of the mid-session death and the automatic fall-open ~50s later; the LuCI "Last boot check" line; the
watchdog config (probe targets, interval, failure count). The kill test: stop the exit, watch direct internet return
on its own with nobody touching the box.
- Aha: a full-tunnel router is only safe if it knows how to give up. Trigger on the actual /1 route, not a config flag,
and it protects you no matter how full-tunnel got turned on.

### Part 5 -- "DNS that resolves as-at-home, without ever breaking DNS"
- Hook: your home name server answers as if you were on the couch -- a home hostname resolves to its home LAN IP --
from a foreign wifi, and if the tunnel drops, normal browsing DNS never even hiccups.
- Beat: the constraint that shaped everything -- DNS failures are the most visible outage there is, so general
resolution must never depend on the tunnel. dnsmasq keeps its own direct default resolver, untouched. We only ADD
per-domain forwarding of chosen domains (your overlay suffix, your home DHCP domain) to ZET's embedded resolver; ZET
answers Ziti service names with synthetic IPs and forwards everything else to a home resolver -- a pi-hole -- reached
OVER the tunnel, so home names resolve from the home vantage. Tunnel down? Only those domains fail; all other DNS is
unaffected. No single point of failure, no coupling to the tunnel's up/down state.
- Show: three lookups from a client behind the router -- a home name resolving to its home LAN IP, a Ziti service name
resolving to a synthetic 100.64.x, and a public name resolving normally via the untouched default.
- Aha (and the best war story): ZET's resolver does not live where you expect. It sits at the .2 of the DNS range
(100.64.0.2, not .1 -- .1 is the tun's own local address and answers nothing) and is reached by routing INTO the tun.
dnsmasq was squatting that address until we told it to let go. Found the hard way, live.

### Part 6 -- "Making it usable: a LuCI app, a signed feed, and self-update"
- Beat: turning a pile of SSH commands into something a human operates. The luci-app-ziti tabs (Status with a
"Last boot check" line and a start-at-boot toggle, Identities, Tunnel Mode with a full/split toggle that flips
identities + firewall, and Settings for the signed self-update, the resilience-watchdog tuning, and the OpenZiti DNS
domains + upstream). The
packaging: an OpenWRT SDK build in Docker, a usign-signed opkg feed on GitHub Pages, and why the device verifies the
feed signature explicitly (GL.iNet ships opkg with check_signature off, and turning it on globally breaks GL's own
unsigned feed). The CI that rebuilds + republishes on push, and a scheduled job that tracks upstream ZET releases
Expand All @@ -54,7 +89,7 @@ That is the hook. The overlay is invisible to the client; the travel router does
check-upstream-versions workflow.
- Aha: the trust anchor is the feed signature, not the transport -- prove provenance, do not just trust the URL.

### Part 5 -- "Will it run on your router? Portability and the compatibility matrix"
### Part 7 -- "Will it run on your router? Portability and the compatibility matrix"
- Beat: what hardware works, the OpenWRT 23.05 line and the libopenssl ABI constraint, the GL.iNet QSDK arch-label
repack (aarch64_cortex-a53 -> aarch64_cortex-a53_neon-vfpv4), and the one-line signed-feed install. The NSS
hardware-offload caveat on QSDK devices.
Expand All @@ -66,8 +101,9 @@ That is the hook. The overlay is invisible to the client; the travel router does

Title: "I turned a $100 travel router into a zero-trust home gateway (and bricked it twice first)."
Shape: lead with the payoff (2-hop traceroute + home IP from a hotel), then the one-paragraph why-not-a-VPN, then the
three gotchas that will bite anyone, then the minimal working recipe, then "here is the LuCI app so you do not have to
type any of it." Links out to the deeper docs for the full CLI.
three gotchas that will bite anyone, then the minimal working recipe, then the two things that make it livable -- it
never black-holes your wifi (fails open) and it resolves your home names as-at-home without touching general DNS --
then "here is the LuCI app so you do not have to type any of it." Links out to the deeper docs for the full CLI.

## Strongest visual moments

Expand All @@ -80,6 +116,11 @@ type any of it." Links out to the deeper docs for the full CLI.
overlay proving itself in real time.
- The Tunnel Mode toggle flipping full <-> split, and the signed Updates panel showing "verified (signed by the
project)".
- The watchdog fall-open: kill the exit and screen-record the router log tearing the tunnel down and restoring direct
internet on its own ~50s later -- the "it fixes itself" moment.
- The three DNS lookups side by side from one client: a home hostname -> its home LAN IP, a Ziti service name -> a
synthetic 100.64.x, a public name -> resolved normally -- proof that home-vantage DNS and untouched general DNS
coexist.

## Production notes

Expand Down
Loading