WIP 8 vpn: add gateway support

Author: pymq

README.md

@@ -231,6 +231,74 @@ On desktop you can also pick the active exit node from the web UI or the system
 
 Traffic through a peer has no restrictions beyond the connection between the two of you — direct and relayed paths both work. You can reach the remote peer's LAN, but not the remote peer's `localhost`.
 
+## VPN gateway (full-tunnel exit node)
+
+In addition to the per-application SOCKS5 proxy, awl can route **all** of your IPv4 traffic through a peer at the IP layer — same model as Tailscale exit nodes or a classic full-tunnel WireGuard/OpenVPN. This is a separate feature from SOCKS5: a peer can allow being used for SOCKS5 without serving as a VPN gateway, and vice versa.
+
+### Status
+
+- **Linux:** supported (this is the platform we have tested on).
+- **Android:** client mode is supported via the awl Android app (uses `VpnService` for routing). Serving as an exit node from Android is **not** supported.
+- **macOS / Windows / other:** **not supported yet** — awl will refuse to start with VPN gateway enabled. Windows-side code exists but is unfinished; see `vpn/sockmark/sockmark_windows.go` and `vpn/routes/nat_windows.go` for the open work items.
+- IPv6 traffic is not tunnelled through the gateway in either direction (it is forwarded as a regular awl peer packet instead).
+- Both client and exit-node sides require **CAP_NET_ADMIN** on Linux. AWL needs that already to bring up the TUN, so there is no extra capability to grant.
+
+### Use a friend as your exit node (client side)
+
+1. Make sure you and the peer are friends and the peer has VPN gateway service enabled on their side (see "Serve as an exit node" below). The CLI exposes the candidates:
+   ```bash
+   awl cli gateway list
+   ```
+2. Enable gateway client mode:
+   ```bash
+   awl cli gateway client use --name="peer-name"
+   # or:  awl cli gateway client use --pid=<peer-id>
+   ```
+   Calling `client use` again with a different peer atomically switches to the new gateway.
+3. **Restart `awl`** for the change to take effect. The OS-level routes and the libp2p socket marking that prevent the gateway from looping back through itself are wired up only at startup.
+4. Verify:
+   ```bash
+   awl cli gateway status
+   curl ifconfig.me   # should show the exit node's public IP
+   ```
+
+To turn the gateway off:
+
+```bash
+awl cli gateway client stop
+# then restart awl to remove the OS-level routes
+```
+
+You can also flip these settings from the web UI / system-tray menu, or by editing `config_awl.json` directly while awl is stopped (`gateway.enabled`, `gateway.exitNodePeerID`).
+
+### Serve as an exit node
+
+To allow others to route their internet traffic through this device:
+
+1. Set `gateway.serveAsVPNGateway: true` in `config_awl.json` (it is **off by default** — see "Why off by default" below) and restart awl. Equivalent: `awl cli gateway server enable` (and `awl cli gateway server disable` to turn it back off).
+2. For each friend you want to permit, also tick *Allow as exit node* in their per-peer settings (or `awl cli peers allow_exit_node --name=… --allow=true`). The same flag governs SOCKS5; if you want different per-peer policies for SOCKS5 vs VPN gateway, file a request and we'll split it.
+3. Restart awl. On startup AWL will:
+   - flip `net.ipv4.ip_forward` on,
+   - install a dedicated `AWL-FORWARD` iptables chain that DROPs traffic to your LAN/CGNAT/link-local subnets and ACCEPTs the rest,
+   - add a `MASQUERADE` rule for the awl subnet,
+   - reverse all of the above on a clean shutdown.
+4. Connected friends can now pick you in their `gateway list` output once the next status exchange propagates the new state (≤ 5 minutes, or on next reconnect).
+
+### Why off by default
+
+Every mainstream VPN solution we surveyed — Tailscale, Headscale, ZeroTier, Nebula, WireGuard, OpenVPN, strongSwan — makes accepting other peers' internet traffic strictly opt-in, and AWL follows the same convention. The reasons are the same here:
+
+- **Trust:** any friend in `KnownPeers` whom you have allowed will appear as your IP in destination logs, takedown notices, and abuse reports. That is a load-bearing trust position; the user should sign up for it explicitly, not inherit it from a default.
+- **Host side-effects:** flipping `net.ipv4.ip_forward` and installing iptables rules is global system state, not something AWL can sandbox. We don't want a routine awl install to silently change those.
+- **Conntrack/CPU cost:** the exit node sees every connection from every gateway client through `nf_conntrack`. On low-end hardware the load is non-trivial.
+
+### Security and privacy notes
+
+- **Your IP is exposed.** Once you serve as an exit node, the public IPs of your friends' destinations see your IP, not theirs.
+- **Your LAN is not.** AWL drops forwarded traffic to RFC 1918 / RFC 6598 / RFC 3927 ranges (`10/8`, `172.16/12`, `192.168/16`, `100.64/10`, `169.254/16`) so a gateway client cannot reach the exit node's home network.
+- **DNS:** in client gateway mode AWL forces upstream DNS to a public resolver to prevent the LAN resolver from leaking queries past the tunnel.
+- **No mutual auth at the IP layer.** Every gateway client's traffic is masqueraded out from the same NIC; you cannot per-flow attribute traffic back to a specific peer at the kernel level. The exit node only filters by `WeAllowUsingAsExitNode` at the awl protocol entry, so revoke aggressively if a peer abuses it.
+
 ## Configuration
 
 Awl stores all its state in a single JSON file called `config_awl.json`. The file is created automatically on the first launch and is rewritten by the application every time you change something through the web UI or CLI. You can also edit it by hand while awl is stopped.

api/vpn_gateway.go

@@ -74,6 +74,10 @@ func (h *Handler) SetVPNGatewayServerEnabled(c echo.Context) error {
 		return c.JSON(http.StatusInternalServerError, ErrorMessage(err.Error()))
 	}
 
+	go func() {
+		h.authStatus.ExchangeStatusInfoWithAllKnownPeers(h.ctx)
+	}()
+
 	return c.NoContent(http.StatusOK)
 }
 

cmd/awl-tray/tray.go

@@ -28,13 +28,13 @@ var (
 	openBrowserMenu *systray.MenuItem
 	peersMenu       *systray.MenuItem
 	proxyMenu       *systray.MenuItem
-	gatewayMenu     *systray.MenuItem // nil when service.VPNGatewaySupported() != nil
+	gatewayMenu     *systray.MenuItem // nil when service.VPNGatewayClientSupported() != nil
 	startStopMenu   *systray.MenuItem
 	restartMenu     *systray.MenuItem
 	updateMenu      *systray.MenuItem
 
 	proxyRouting   *routingMenu
-	gatewayRouting *routingMenu // .root is nil when service.VPNGatewaySupported() != nil
+	gatewayRouting *routingMenu // .root is nil when service.VPNGatewayClientSupported() != nil
 )
 
 const updateMenuLabel = "Check for updates"
@@ -126,12 +126,11 @@ func initTray() {
 		}
 	}()
 
-	if service.VPNGatewaySupported() == nil {
+	if service.VPNGatewayClientSupported() == nil {
 		gatewayMenu = systray.AddMenuItem("VPN Gateway", "")
-		gatewayRouting = newRoutingMenu(gatewayMenu, routingMenuConfig{
+		cfg := routingMenuConfig{
 			noneLabel:        "None (disabled)",
 			emptyLabel:       `No VPN gateways available — set "Allow as exit node" on a peer`,
-			serveLabel:       "Serve as VPN gateway",
 			listPeers:        listGatewayPeers,
 			currentSelection: currentGatewaySelection,
 			selectPeer: func(peerIDStr string) error {
@@ -144,15 +143,22 @@ func initTray() {
 			disable: func() {
 				app.VPNGateway.DisableClient()
 			},
-			serveCurrent: func() bool {
+		}
+		// Only expose the "Serve as VPN gateway" toggle on platforms where
+		// server mode actually runs. Empty serveLabel makes routingMenu skip
+		// the toggle entirely.
+		if service.VPNGatewayServerSupported() == nil {
+			cfg.serveLabel = "Serve as VPN gateway"
+			cfg.serveCurrent = func() bool {
 				app.Conf.RLock()
 				defer app.Conf.RUnlock()
 				return app.Conf.VPNGateway.ServerEnabled
-			},
-			serveSet: func(b bool) error {
+			}
+			cfg.serveSet = func(b bool) error {
 				return app.VPNGateway.SetServerEnabled(b)
-			},
-		})
+			}
+		}
+		gatewayRouting = newRoutingMenu(gatewayMenu, cfg)
 
 		go func() {
 			// On windows systray does not trigger clicked event on menus with submenus

service/vpn_gateway.go

@@ -71,28 +71,43 @@ func NewVPNGateway(conf *config.Config, tunnel *Tunnel, device *vpn.Device, p2p
 	}
 }
 
-// VPNGatewaySupported reports whether VPN gateway mode can run on this OS/build.
-// Returns nil when supported, or a descriptive error explaining why not. The
-// check is OS-only and does not depend on any VPNGateway state, so it is safe
-// to call before NewVPNGateway (e.g. from awl-tray during menu construction).
-//
-// Currently only Linux is fully supported. macOS lacks NAT/route glue (the
-// vpn/routes/*_other.go implementations return errors), and Windows has
-// partially-written code (see TODOs in vpn/sockmark/sockmark_windows.go
-// and vpn/routes/nat_windows.go) but is not safe to enable until NAT and
-// DNS-leak protection are finished.
-func VPNGatewaySupported() error {
+// VPNGatewayClientSupported reports whether client-side VPN gateway mode can
+// run on this OS/build. Linux has the full implementation; Android can run as
+// a client but only via the Android host's VpnService.Builder, so runtime API
+// changes only mutate the persisted config and take effect on next startup
+// (see EnableClient / DisableClient).
+func VPNGatewayClientSupported() error {
 	switch runtime.GOOS {
-	case "linux":
+	case "linux", "android":
 		return nil
 	case "windows":
-		return fmt.Errorf("VPN gateway mode is not yet supported on Windows; " +
+		return fmt.Errorf("VPN gateway client mode is not yet supported on Windows; " +
 			"see vpn/sockmark/sockmark_windows.go for outstanding work")
 	case "darwin":
-		return fmt.Errorf("VPN gateway mode is not yet supported on macOS")
+		return fmt.Errorf("VPN gateway client mode is not yet supported on macOS")
 	default:
-		return fmt.Errorf("VPN gateway mode is not supported on %s", runtime.GOOS)
+		return fmt.Errorf("VPN gateway client mode is not supported on %s", runtime.GOOS)
+	}
+}
+
+// VPNGatewayServerSupported reports whether server-side VPN gateway mode can
+// run on this OS/build. Currently only Linux: Android exit-node support
+// requires root or special system config, macOS lacks NAT/route glue, and the
+// Windows path (vpn/routes/nat_windows.go) is not yet safe to enable.
+func VPNGatewayServerSupported() error {
+	if runtime.GOOS == "linux" {
+		return nil
 	}
+	return fmt.Errorf("VPN gateway server mode is not supported on %s", runtime.GOOS)
+}
+
+// VPNGatewaySupported reports whether the full VPN gateway feature set (both
+// client and server) can run on this OS/build. Equivalent to
+// VPNGatewayServerSupported — server is the strictly stronger requirement.
+// Kept as a convenience for callers (awl-tray menu construction) that gate the
+// whole feature UI on full support.
+func VPNGatewaySupported() error {
+	return VPNGatewayServerSupported()
 }
 
 // ListAvailableVPNGateways returns peers that are currently a valid VPN
@@ -129,8 +144,18 @@ func (g *VPNGateway) ListAvailableVPNGateways() []entity.AvailableVPNGateway {
 
 // EnableClient turns on VPN gateway client mode using the given peer as the
 // gateway, applying OS-level routes immediately. Atomic: rolls back the
-// tunnel binding (and the persisted config) on apply failure.
+// tunnel binding on apply failure.
+//
+// On android the OS-level apply (routes.SetupGatewayRoutes / sockmark) is a
+// no-op — routing is owned by the host's VpnService.Builder, which is only
+// rebuilt at daemon (re)start. Runtime calls still flip the in-memory tunnel
+// binding and persist config, but a restart is required for the change to
+// actually affect OS traffic; the UI is responsible for prompting that
+// restart.
 func (g *VPNGateway) EnableClient(gatewayPeerID peer.ID) error {
+	if err := VPNGatewayClientSupported(); err != nil {
+		return err
+	}
 	if g.tunnel == nil {
 		return fmt.Errorf("VPN interface is disabled, cannot enable gateway")
 	}
@@ -169,6 +194,9 @@ func (g *VPNGateway) DisableClient() {
 // the config directly.
 func (g *VPNGateway) SetServerEnabled(enabled bool) error {
 	if enabled {
+		if err := VPNGatewayServerSupported(); err != nil {
+			return err
+		}
 		if err := g.applyServer(); err != nil {
 			return err
 		}
@@ -201,20 +229,19 @@ func (g *VPNGateway) SetupAtStartup() error {
 	gw := g.conf.VPNGateway
 	g.conf.RUnlock()
 
-	if err := VPNGatewaySupported(); err != nil {
-		if gw.ClientEnabled || gw.ServerEnabled {
-			g.logger.Errorf("VPN gateway not enabled at startup: %v", err)
-		}
-		return nil
-	}
-
 	if gw.ServerEnabled {
-		if err := g.SetServerEnabled(true); err != nil {
+		if err := VPNGatewayServerSupported(); err != nil {
+			g.logger.Errorf("VPN gateway server not enabled at startup: %v", err)
+		} else if err := g.SetServerEnabled(true); err != nil {
 			return fmt.Errorf("couldn't enable VPN gateway server at startup: %v", err)
 		}
 	}
 
 	if gw.ClientEnabled && gw.GatewayPeerID != "" {
+		if err := VPNGatewayClientSupported(); err != nil {
+			g.logger.Errorf("VPN gateway client not enabled at startup: %v", err)
+			return nil
+		}
 		gatewayPeerID, err := peer.Decode(gw.GatewayPeerID)
 		if err != nil {
 			g.logger.Errorf("failed to decode gateway peer id, disabling VPN gateway client and continue startup: %v", err)
@@ -278,7 +305,7 @@ func (g *VPNGateway) applyServer() error {
 	if g.serverNATState != nil {
 		return nil
 	}
-	if err := VPNGatewaySupported(); err != nil {
+	if err := VPNGatewayServerSupported(); err != nil {
 		return err
 	}
 	if g.device == nil {
@@ -332,7 +359,7 @@ func (g *VPNGateway) applyClient() error {
 	g.mu.Lock()
 	defer g.mu.Unlock()
 
-	if err := VPNGatewaySupported(); err != nil {
+	if err := VPNGatewayClientSupported(); err != nil {
 		return err
 	}
 	if g.tunnel == nil {