From 6bab3b8e739ba2e8dfff1022784ca17d6a2b9ca0 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Mon, 13 Apr 2026 19:50:31 -0700 Subject: [PATCH 01/16] Add Yubikey examples to docs --- README.md | 6 ++- docs/HSM-PKCS11.md | 131 +++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 130 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 7e5ad9ff52..8820d71b50 100644 --- a/README.md +++ b/README.md @@ -335,9 +335,11 @@ See [METRICS](docs/METRICS.md) for details. ### HSM/PKCS#11 support Ghostunnel has support for loading private keys from PKCS#11 modules, which -should work with any hardware security module that exposes a PKCS#11 interface. +should work with any hardware security module that exposes a PKCS#11 interface, +including YubiKeys (via the YKCS11 module). -See [HSM-PKCS11](docs/HSM-PKCS11.md) for details. +See [HSM-PKCS11](docs/HSM-PKCS11.md) for details, including a step-by-step +guide for using Ghostunnel with a YubiKey. ### Windows/macOS Keychain Support diff --git a/docs/HSM-PKCS11.md b/docs/HSM-PKCS11.md index e98e23aa26..d8f74b94c9 100644 --- a/docs/HSM-PKCS11.md +++ b/docs/HSM-PKCS11.md @@ -54,18 +54,139 @@ to the private key in the PKCS#11 module, with the leaf certificate being the first certificate in the chain. Ghostunnel currently cannot read the certificate chain directly from the module. -### Certificate hotswapping +## Using a YubiKey + +[YubiKey][yubikey] 4 and 5 series support the [PIV (FIPS 201)][piv] standard, +which exposes a PKCS#11 interface via the [YKCS11][ykcs11] module, so you +can use a YubiKey to hold Ghostunnel's private key in hardware. + +[yubikey]: https://www.yubico.com +[piv]: https://developers.yubico.com/PIV/ +[ykcs11]: https://developers.yubico.com/yubico-piv-tool/YKCS11/ + +### Prerequisites + +You'll need `yubico-piv-tool`, which ships the CLI and the `libykcs11` +PKCS#11 module: + +```bash +# macOS +brew install yubico-piv-tool + +# Debian/Ubuntu +apt install yubico-piv-tool ykcs11 +``` + +The module lives in different places depending on your platform: + +| Platform | Typical path | +|-----------------------|-------------------------------------------------| +| macOS (Apple Silicon) | `/opt/homebrew/lib/libykcs11.dylib` | +| macOS (Intel) | `/usr/local/lib/libykcs11.dylib` | +| Linux (x86_64) | `/usr/lib/x86_64-linux-gnu/libykcs11.so` or `/usr/local/lib/libykcs11.so` | + +### PIV slots + +YubiKey PIV has several key slots. For TLS with Ghostunnel, you'll +usually want slot **9a** (Authentication): + +| Slot | Purpose | Typical use | +|------|----------------------|--------------------------| +| 9a | Authentication | TLS client/server certs | +| 9c | Digital Signature | Code/document signing | +| 9d | Key Management | Encryption | +| 9e | Card Authentication | Physical access | + +### Generating a key and certificate + +Generate a key pair on the YubiKey itself (the private key never leaves +the device): + +```bash +# Generate an RSA 2048 key in slot 9a +yubico-piv-tool -s 9a -a generate -A RSA2048 -o public-key.pem + +# Create a certificate signing request (CSR) +yubico-piv-tool -s 9a -a verify-pin -a request-certificate \ + -S '/CN=my-server/' -i public-key.pem -o csr.pem +``` + +Sign the CSR with your CA, then import the signed certificate back: + +```bash +yubico-piv-tool -s 9a -a import-certificate -i server-cert.pem +``` + +### Exporting the certificate for Ghostunnel + +Ghostunnel reads the certificate chain from disk, not from the PKCS#11 +module, so you'll need to export it: + +```bash +yubico-piv-tool -s 9a -a read-certificate -o server-cert.pem +``` + +If your CA has an intermediate, concatenate them into a chain (leaf first): + +```bash +cat server-cert.pem intermediate.pem > chain.pem +``` + +### Launching Ghostunnel with a YubiKey + +```bash +ghostunnel server \ + --cert chain.pem \ + --pkcs11-module /opt/homebrew/lib/libykcs11.dylib \ + --pkcs11-token-label "YubiKey PIV #12345678" \ + --pkcs11-pin 123456 \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert ca-cert.pem \ + --allow-cn client +``` + +The default PIV PIN is `123456`. Change it before doing anything real. To +keep the PIN off the command line, use the `PKCS11_PIN` environment variable +instead of `--pkcs11-pin`. + +To find the correct token label for your YubiKey: + +```bash +pkcs11-tool --module /opt/homebrew/lib/libykcs11.dylib -L +``` + +### Debugging + +If things aren't working, set `YKCS11_DBG` (values 1–9) for verbose output +from the YKCS11 module: + +```bash +YKCS11_DBG=1 ghostunnel server ... +``` + +`pkcs11-tool` is also handy for poking around on the YubiKey: + +```bash +# List available slots/tokens +pkcs11-tool --module /path/to/libykcs11.dylib -L + +# List objects (keys, certificates) on the token +pkcs11-tool --module /path/to/libykcs11.dylib -O +``` + +## Certificate hotswapping When using PKCS#11, certificate hotswapping (via `SIGHUP`/`SIGUSR1` or -`--timed-reload`) reloads only the certificate from disk. The private key in -the HSM is assumed to remain the same. This means the updated or reissued -certificate must still match the private key that was loaded from the HSM. +`--timed-reload`) reloads only the certificate from disk. The private key +in the HSM stays put, so the new certificate still needs to match the key +that was loaded from the HSM. Note that Landlock sandboxing is automatically disabled when PKCS#11 is used, as PKCS#11 modules are opaque shared libraries that may need access to arbitrary files and sockets. -### Inspecting PKCS#11 state +## Inspecting PKCS#11 state If you need to inspect the state of a PKCS11 module/token, we recommend the [`pkcs11-tool`][pkcs11-tool] utility from OpenSC. For example, it can be used From aeea9daef44e9725c56e7c55a8aca3e1e3a0aa07 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:23:01 -0700 Subject: [PATCH 02/16] Add more keychain examples to docs --- docs/KEYCHAIN.md | 179 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 167 insertions(+), 12 deletions(-) diff --git a/docs/KEYCHAIN.md b/docs/KEYCHAIN.md index 5d5bd2cd16..97e1aa0c38 100644 --- a/docs/KEYCHAIN.md +++ b/docs/KEYCHAIN.md @@ -4,27 +4,142 @@ description: Load certificates and private keys from the macOS Keychain or Windo weight: 50 --- -If you have identities stored in the macOS Keychain or Windows Certificate -Store, Ghostunnel can load certificates directly from them. This is useful -when you want private keys backed by the Secure Enclave on Touch ID MacBooks, -or when managing certificates through the OS is preferable to managing files -on disk. +Ghostunnel can load certificates and private keys directly from the macOS +Keychain or Windows Certificate Store. This lets you use Secure Enclave-backed +keys on Touch ID MacBooks, hardware-backed keys via CNG on Windows, or simply +manage certificates through the OS instead of as files on disk. + +### Prerequisites: creating a PKCS#12 bundle + +Both macOS and Windows import certificates from [PKCS#12][openssl-pkcs12] +(`.p12` / `.pfx`) files. If you have a PEM certificate and key, bundle them +first: + +```bash +openssl pkcs12 -export \ + -in server-cert.pem \ + -inkey server-key.pem \ + -out server.p12 \ + -passout pass: +``` + +If you also need to include intermediate CA certificates in the bundle, add +`-certfile intermediate-ca.pem`. + +[openssl-pkcs12]: https://docs.openssl.org/master/man1/openssl-pkcs12/ + +### macOS: importing into the Keychain + +**Using the CLI** (recommended for automation): + +```bash +security import server.p12 \ + -k ~/Library/Keychains/login.keychain-db \ + -f pkcs12 \ + -P \ + -A +``` + +The `-A` flag allows all applications to access the imported key without a +confirmation prompt. Omit it if you prefer per-application access control. + +You can also **double-click the `.p12` file** in Finder or use the +[Keychain Access][apple-keychain-access] app to import through the GUI. + +**Verify** the import succeeded: + +```bash +security find-identity -v +``` + +This lists all identities (certificate + private key pairs) in your keychain +search list. Look for your certificate's Common Name in the output. + +See also Apple's [Keychain Services documentation][apple-keychain-services] +and [TN3137: On Mac keychain APIs and implementations][apple-tn3137]. + +[apple-keychain-access]: https://support.apple.com/guide/keychain-access/add-certificates-to-a-keychain-kyca2431/mac +[apple-keychain-services]: https://developer.apple.com/documentation/security/keychain-services +[apple-tn3137]: https://developer.apple.com/documentation/technotes/tn3137-on-mac-keychains + +### macOS: Secure Enclave and hardware tokens + +On Touch ID MacBooks, private keys can live in the Secure Enclave. Pass +`--keychain-require-token` so Ghostunnel only loads keys backed by a hardware +token (e.g. the Secure Enclave or a smart card): + +```bash +ghostunnel server \ + --keychain-identity \ + --keychain-require-token \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert cacert.pem \ + --allow-ou=client +``` + +This flag is only available on macOS and has no effect on Windows. + +See Apple's [Protecting keys with the Secure Enclave][apple-secure-enclave] +and the [Secure Enclave security overview][apple-se-overview] for more on +hardware-backed keys. + +[apple-secure-enclave]: https://developer.apple.com/documentation/security/protecting-keys-with-the-secure-enclave +[apple-se-overview]: https://support.apple.com/guide/security/sec59b0b31ff/web + +### Windows: importing into the Certificate Store + +**Using certutil** (recommended for automation): + +```bash +certutil -f -p -user -importpfx MY server.p12 +``` + +This imports the certificate and private key into the current user's "MY" +(Personal) store. The `-user` flag targets the current user context; replace +it with `-enterprise` to import into the Local Machine store instead. + +**Using PowerShell**: + +```powershell +Import-PfxCertificate -FilePath server.p12 ` + -CertStoreLocation Cert:\CurrentUser\My ` + -Password (ConvertTo-SecureString -String "" -AsPlainText -Force) +``` + +**Verify** the import: + +```powershell +Get-ChildItem Cert:\CurrentUser\My | Format-Table Subject, Thumbprint, NotAfter +``` + +**Which stores does Ghostunnel search?** When `--keychain-identity` is used +on Windows, Ghostunnel searches three stores in order: + +1. **MY** (Current User), the personal certificate store +2. **CURRENT_SERVICE**, the current service account's certificates +3. **LOCAL_MACHINE**, machine-wide certificates (may require elevation) + +See Microsoft's [certutil reference][ms-certutil], +[System Store Locations][ms-store-locations], and the +[Import-PfxCertificate][ms-import-pfx] cmdlet docs for more. + +[ms-certutil]: https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/certutil +[ms-store-locations]: https://learn.microsoft.com/en-us/windows/win32/seccrypto/system-store-locations +[ms-import-pfx]: https://learn.microsoft.com/en-us/powershell/module/pki/import-pfxcertificate ### Selecting a certificate Certificates from the keychain can be selected using one or both of the following flags: -* `--keychain-identity` — match by the certificate's Common Name (CN) or +* `--keychain-identity`: match by the certificate's Common Name (CN) or serial number. Ghostunnel checks both fields and uses the first match. -* `--keychain-issuer` — match by the issuer's Common Name (CN). +* `--keychain-issuer`: match by the issuer's Common Name (CN). When both flags are specified, Ghostunnel selects certificates where both -attributes match (logical AND). - -On macOS, `--keychain-require-token` additionally requires the loaded -certificate to come from a physical hardware token (e.g. the Secure Enclave). -This flag is not available on Windows. +attributes match (logical AND). If multiple certificates match, the one with +the latest expiration date (NotAfter) is used. ### macOS example @@ -68,3 +183,43 @@ Keychain certificates support reloading via `SIGHUP`/`SIGUSR1` or certificate matching the same identity/issuer criteria. If the certificate has been updated in the keychain (e.g. renewed), the new certificate will be used for subsequent connections. + +### Removing certificates + +**macOS**: remove an identity (certificate + private key) by Common Name: + +```bash +security delete-identity -c +``` + +Or use the Keychain Access app to find and delete the certificate in the GUI. + +**Windows**: remove via PowerShell: + +```powershell +Get-ChildItem Cert:\CurrentUser\My | + Where-Object { $_.Subject -match "CN=" } | + Remove-Item -DeleteKey +``` + +The `-DeleteKey` flag also removes the private key. You can alternatively +use `certutil -delstore MY `. + +### Troubleshooting + +**macOS: certificate not found** +- Check the keychain search list: `security list-keychains` +- Unlock the keychain if locked: `security unlock-keychain` +- List available identities: `security find-identity -v` +- Make sure the CN or serial matches what you passed to `--keychain-identity` + +**Windows: certificate not found** +- List certs in the store: `Get-ChildItem Cert:\CurrentUser\My` +- If using the Local Machine store, make sure Ghostunnel runs with sufficient permissions +- Make sure the CN or serial matches what you passed to `--keychain-identity` + +**Access denied / permission errors** +- **macOS**: the keychain may prompt for access. Use `-A` during import to allow all apps, or grant access to Ghostunnel specifically in Keychain Access. +- **Windows**: the account running Ghostunnel needs read access to the private key. See [Manage private key permissions][ms-private-key-perms]. + +[ms-private-key-perms]: https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/technical-reference/manage-ssl-certificates-ad-fs-wap#manage-private-key-permissions From c5d840f8e4b72dd2c49b532fa3ce91d291528f2e Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:31:35 -0700 Subject: [PATCH 03/16] Add more links to references in docs, fix dead links --- docs/ACCESS-FLAGS.md | 13 ++++++++----- docs/ACME.md | 25 +++++++++++++++---------- docs/HSM-PKCS11.md | 6 ++++-- docs/KEYCHAIN.md | 4 +--- docs/SECURITY.md | 13 ++++++++----- docs/SOCKET-ACTIVATION.md | 9 +++++++++ docs/SPIFFE-WORKLOAD-API.md | 6 +++++- docs/WATCHDOG.md | 11 +++++++---- 8 files changed, 57 insertions(+), 30 deletions(-) diff --git a/docs/ACCESS-FLAGS.md b/docs/ACCESS-FLAGS.md index 3b78fc0de6..958e84fff5 100644 --- a/docs/ACCESS-FLAGS.md +++ b/docs/ACCESS-FLAGS.md @@ -135,11 +135,14 @@ but the backend doesn't require mutual authentication. ## Open Policy Agent -Ghostunnel has support for Open Policy Agent (OPA), both in server and client -mode. The policy bundle must be present on disk for Ghostunnel to use it and the -use of OPA is mutually exclusive with any other `allow` (or `verify`) flags. -Policy bundles can be reloaded at runtime much like certificates, with the -`--timed-reload` flag or via `SIGHUP`. +Ghostunnel has support for [Open Policy Agent][opa] (OPA), both in server and +client mode. The policy must be provided as an [OPA bundle][opa-bundles] on +disk and the use of OPA is mutually exclusive with any other `allow` (or +`verify`) flags. Policy bundles can be reloaded at runtime much like +certificates, with the `--timed-reload` flag or via `SIGHUP`. + +[opa]: https://www.openpolicyagent.org/ +[opa-bundles]: https://www.openpolicyagent.org/docs/latest/management-bundles/ To use it in server mode, specify the `--allow-policy` and `--allow-query` flags. diff --git a/docs/ACME.md b/docs/ACME.md index 940ad4d3bb..a4852bd6fd 100644 --- a/docs/ACME.md +++ b/docs/ACME.md @@ -5,9 +5,9 @@ weight: 30 --- In server mode, Ghostunnel can automatically obtain and renew a public TLS -certificate via the ACME protocol. This is powered by -[certmagic](https://github.com/caddyserver/certmagic), which handles -certificate storage, renewal, and OCSP stapling. +certificate via the [ACME][acme-rfc] protocol. This is powered by +[certmagic][certmagic], which handles certificate storage, renewal, and OCSP +stapling. ### Basic usage @@ -26,10 +26,10 @@ ghostunnel server \ --allow-cn client ``` -Ghostunnel defaults to using Let's Encrypt as the ACME CA. You can specify a -different ACME CA URL using `--auto-acme-ca`. To test against a non-production -CA (e.g. Let's Encrypt's staging environment), use `--auto-acme-testca` — when -set, the `--auto-acme-ca` flag is ignored. +Ghostunnel defaults to using [Let's Encrypt][letsencrypt] as the ACME CA. You +can specify a different ACME CA URL using `--auto-acme-ca`. To test against a +non-production CA (e.g. Let's Encrypt's staging environment), use +`--auto-acme-testca`. When set, the `--auto-acme-ca` flag is ignored. ### Requirements @@ -38,14 +38,14 @@ a public interface on tcp/443, or have tcp/443 forwarded to it (e.g. via a systemd socket or iptables). Public DNS records must exist for the FQDN that resolve to the public listening interface IP. -Ghostunnel uses the TLS-ALPN-01 challenge type (HTTP-01 is disabled), so port -443 must be reachable. +Ghostunnel uses the [TLS-ALPN-01][tls-alpn-01] challenge type (HTTP-01 is +disabled), so port 443 must be reachable. ### Certificate storage and renewal Certificates are stored locally by certmagic in its default storage directory (typically `~/.local/share/certmagic` or the equivalent on your OS). Certmagic -automatically renews certificates before they expire — no manual intervention +automatically renews certificates before they expire, so no manual intervention or `--timed-reload` is needed for ACME certificates. If a valid certificate already exists locally, Ghostunnel loads it from cache @@ -56,3 +56,8 @@ on startup without contacting the CA. On startup, Ghostunnel attempts to obtain the initial certificate up to 5 times with exponential backoff (starting at 5 seconds, capped at 2 minutes). If all attempts fail, Ghostunnel exits with an error. + +[acme-rfc]: https://datatracker.ietf.org/doc/html/rfc8555 +[letsencrypt]: https://letsencrypt.org/ +[tls-alpn-01]: https://datatracker.ietf.org/doc/html/rfc8737 +[certmagic]: https://pkg.go.dev/github.com/caddyserver/certmagic diff --git a/docs/HSM-PKCS11.md b/docs/HSM-PKCS11.md index d8f74b94c9..cc0c5f590a 100644 --- a/docs/HSM-PKCS11.md +++ b/docs/HSM-PKCS11.md @@ -4,8 +4,9 @@ description: Load private keys from hardware security modules via the PKCS#11 in weight: 40 --- -Ghostunnel has support for loading private keys from PKCS#11 modules, which -should work with any hardware security module that exposes a PKCS#11 interface. +Ghostunnel has support for loading private keys from [PKCS#11][pkcs11-spec] +modules, which should work with any hardware security module that exposes a +PKCS#11 interface. An easy way to test the PKCS#11 interface for development purposes is with [SoftHSM][softhsm]. Note that CGO is required in order for PKCS#11 support to work. @@ -203,4 +204,5 @@ pkcs11-tool --module $MODULE -O -y cert pkcs11-tool --module $MODULE --label $LABEL --read-object -y cert ``` +[pkcs11-spec]: https://docs.oasis-open.org/pkcs11/pkcs11-spec/v3.1/pkcs11-spec-v3.1.html [pkcs11-tool]: https://github.com/OpenSC/OpenSC/wiki/SmartCardHSM#using-pkcs11-tool diff --git a/docs/KEYCHAIN.md b/docs/KEYCHAIN.md index 97e1aa0c38..8af88a5825 100644 --- a/docs/KEYCHAIN.md +++ b/docs/KEYCHAIN.md @@ -220,6 +220,4 @@ use `certutil -delstore MY `. **Access denied / permission errors** - **macOS**: the keychain may prompt for access. Use `-A` during import to allow all apps, or grant access to Ghostunnel specifically in Keychain Access. -- **Windows**: the account running Ghostunnel needs read access to the private key. See [Manage private key permissions][ms-private-key-perms]. - -[ms-private-key-perms]: https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/technical-reference/manage-ssl-certificates-ad-fs-wap#manage-private-key-permissions +- **Windows**: the account running Ghostunnel needs read access to the private key. You can manage private key permissions through the Certificates MMC snap-in (right-click a certificate, then "All Tasks > Manage Private Keys"). diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 99f4799487..0a2b18c29f 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -33,8 +33,8 @@ All suites use authenticated encryption (AEAD). CBC-mode ciphers are not enabled. ECDSA suites are listed before RSA to prefer ECDSA when both certificate types are available. -In TLS 1.3, cipher suite selection is handled by Go's `crypto/tls` and cannot -be configured by the application. The TLS 1.3 suites listed above are always +In TLS 1.3, cipher suite selection is handled by Go's [`crypto/tls`][crypto-tls] +and cannot be configured by the application. The TLS 1.3 suites listed above are always available when TLS 1.3 is negotiated. The configurable cipher suite list only affects TLS 1.2 connections. @@ -89,9 +89,9 @@ localhost risks unauthorized access to the proxied service. ## Landlock sandboxing -On Linux, Ghostunnel uses [Landlock](https://landlock.io) to restrict its own -process privileges after startup. Landlock is a kernel-level access control -mechanism that limits which files and network ports a process can access. +On Linux, Ghostunnel uses [Landlock][landlock] to restrict its own process +privileges after startup. Landlock is a kernel-level access control mechanism +that limits which files and network ports a process can access. ### How it works @@ -117,3 +117,6 @@ Landlock can be disabled with `--disable-landlock` if it causes issues with your deployment. This is not recommended. Landlock is also automatically disabled when PKCS#11 is in use, since PKCS#11 modules are opaque shared libraries that may require access to arbitrary files and sockets. + +[crypto-tls]: https://pkg.go.dev/crypto/tls +[landlock]: https://docs.kernel.org/userspace-api/landlock.html diff --git a/docs/SOCKET-ACTIVATION.md b/docs/SOCKET-ACTIVATION.md index d26933fe2a..859613af36 100644 --- a/docs/SOCKET-ACTIVATION.md +++ b/docs/SOCKET-ACTIVATION.md @@ -14,6 +14,9 @@ Note that socket activation is not available on Windows. ### launchd +See Apple's [Creating Launch Daemons and Agents][launchd-guide] for background +on launchd plists. + A launchd plist to launch Ghostunnel in server mode on :8081, listening for status connections on :8082, and forwarding connections to :8083 could look like this: @@ -78,6 +81,9 @@ pass them to Ghostunnel which is not currently supported. ### systemd +See the [`systemd.socket`][systemd-socket] man page for the full socket unit +reference. + A systemd unit for a `ghostunnel.socket` for listening on `*:8443` could look like this: @@ -117,3 +123,6 @@ used to distinguish the listening and status sockets. Ghostunnel also supports systemd notify and watchdog functionality. See [WATCHDOG]({{< ref "WATCHDOG.md" >}}) for details on configuring `Type=notify-reload` services. + +[launchd-guide]: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html +[systemd-socket]: https://www.freedesktop.org/software/systemd/man/latest/systemd.socket.html diff --git a/docs/SPIFFE-WORKLOAD-API.md b/docs/SPIFFE-WORKLOAD-API.md index 3ff07b60b5..892c2c8bb8 100644 --- a/docs/SPIFFE-WORKLOAD-API.md +++ b/docs/SPIFFE-WORKLOAD-API.md @@ -76,4 +76,8 @@ a manual reload or restart. See the [end-to-end demo](https://github.com/ghostunnel/ghostunnel/tree/master/docs/spiffe-workload-api-demo) for an example using Ghostunnel with SPIFFE Workload API support backed by -[SPIRE](https://spiffe.io/spire/). +[SPIRE](https://spiffe.io/spire/). If you are new to SPIFFE/SPIRE, the +[SPIRE getting started guide][spire-getting-started] walks through a basic +setup on Linux. + +[spire-getting-started]: https://spiffe.io/docs/latest/try/getting-started-linux-macos-x/ diff --git a/docs/WATCHDOG.md b/docs/WATCHDOG.md index 3a93a961ff..be15e98905 100644 --- a/docs/WATCHDOG.md +++ b/docs/WATCHDOG.md @@ -4,13 +4,13 @@ description: Integrate with the systemd watchdog timer for automatic restart on weight: 85 --- -Ghostunnel supports systemd's notify and watchdog functionality on Linux. This -allows systemd to know when Ghostunnel is ready and to automatically restart it -if it becomes unresponsive. +Ghostunnel supports systemd's [notify][sd-notify] and watchdog functionality on +Linux. This allows systemd to know when Ghostunnel is ready and to automatically +restart it if it becomes unresponsive. ### How it works -When running as a `Type=notify-reload` service: +When running as a [`Type=notify-reload`][systemd-service] service: * **Notify**: Ghostunnel signals readiness to systemd after it has successfully loaded certificates and started listening. Systemd will not consider the @@ -58,3 +58,6 @@ WantedBy=default.target native mechanisms. * For socket activation with systemd, see [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}). + +[sd-notify]: https://www.freedesktop.org/software/systemd/man/latest/sd_notify.html +[systemd-service]: https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html From 129daec4e1904e50222507eb0a382866221f270d Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:33:40 -0700 Subject: [PATCH 04/16] Better formatting in Markdown --- docs/ACME.md | 8 ++++---- docs/KEYCHAIN.md | 33 +++++++++++++++++++++------------ docs/METRICS.md | 14 +++++++------- docs/SOCKET-ACTIVATION.md | 4 ++-- docs/SPIFFE-WORKLOAD-API.md | 11 ++++++----- docs/WATCHDOG.md | 6 +++--- 6 files changed, 43 insertions(+), 33 deletions(-) diff --git a/docs/ACME.md b/docs/ACME.md index a4852bd6fd..ed38132164 100644 --- a/docs/ACME.md +++ b/docs/ACME.md @@ -9,7 +9,7 @@ certificate via the [ACME][acme-rfc] protocol. This is powered by [certmagic][certmagic], which handles certificate storage, renewal, and OCSP stapling. -### Basic usage +## Basic usage To enable ACME, use the `--auto-acme-cert` flag with the FQDN to obtain a certificate for. You must also specify an email address with @@ -31,7 +31,7 @@ can specify a different ACME CA URL using `--auto-acme-ca`. To test against a non-production CA (e.g. Let's Encrypt's staging environment), use `--auto-acme-testca`. When set, the `--auto-acme-ca` flag is ignored. -### Requirements +## Requirements ACME is only supported in server mode. Ghostunnel must either be listening on a public interface on tcp/443, or have tcp/443 forwarded to it (e.g. via a @@ -41,7 +41,7 @@ resolve to the public listening interface IP. Ghostunnel uses the [TLS-ALPN-01][tls-alpn-01] challenge type (HTTP-01 is disabled), so port 443 must be reachable. -### Certificate storage and renewal +## Certificate storage and renewal Certificates are stored locally by certmagic in its default storage directory (typically `~/.local/share/certmagic` or the equivalent on your OS). Certmagic @@ -51,7 +51,7 @@ or `--timed-reload` is needed for ACME certificates. If a valid certificate already exists locally, Ghostunnel loads it from cache on startup without contacting the CA. -### Startup retry behavior +## Startup retry behavior On startup, Ghostunnel attempts to obtain the initial certificate up to 5 times with exponential backoff (starting at 5 seconds, capped at 2 minutes). diff --git a/docs/KEYCHAIN.md b/docs/KEYCHAIN.md index 8af88a5825..19a3e160bc 100644 --- a/docs/KEYCHAIN.md +++ b/docs/KEYCHAIN.md @@ -9,7 +9,7 @@ Keychain or Windows Certificate Store. This lets you use Secure Enclave-backed keys on Touch ID MacBooks, hardware-backed keys via CNG on Windows, or simply manage certificates through the OS instead of as files on disk. -### Prerequisites: creating a PKCS#12 bundle +## Prerequisites: creating a PKCS#12 bundle Both macOS and Windows import certificates from [PKCS#12][openssl-pkcs12] (`.p12` / `.pfx`) files. If you have a PEM certificate and key, bundle them @@ -28,7 +28,9 @@ If you also need to include intermediate CA certificates in the bundle, add [openssl-pkcs12]: https://docs.openssl.org/master/man1/openssl-pkcs12/ -### macOS: importing into the Keychain +## macOS + +### Importing into the Keychain **Using the CLI** (recommended for automation): @@ -62,7 +64,7 @@ and [TN3137: On Mac keychain APIs and implementations][apple-tn3137]. [apple-keychain-services]: https://developer.apple.com/documentation/security/keychain-services [apple-tn3137]: https://developer.apple.com/documentation/technotes/tn3137-on-mac-keychains -### macOS: Secure Enclave and hardware tokens +### Secure Enclave and hardware tokens On Touch ID MacBooks, private keys can live in the Secure Enclave. Pass `--keychain-require-token` so Ghostunnel only loads keys backed by a hardware @@ -87,7 +89,9 @@ hardware-backed keys. [apple-secure-enclave]: https://developer.apple.com/documentation/security/protecting-keys-with-the-secure-enclave [apple-se-overview]: https://support.apple.com/guide/security/sec59b0b31ff/web -### Windows: importing into the Certificate Store +## Windows + +### Importing into the Certificate Store **Using certutil** (recommended for automation): @@ -128,7 +132,7 @@ See Microsoft's [certutil reference][ms-certutil], [ms-store-locations]: https://learn.microsoft.com/en-us/windows/win32/seccrypto/system-store-locations [ms-import-pfx]: https://learn.microsoft.com/en-us/powershell/module/pki/import-pfxcertificate -### Selecting a certificate +## Selecting a certificate Certificates from the keychain can be selected using one or both of the following flags: @@ -141,7 +145,9 @@ When both flags are specified, Ghostunnel selects certificates where both attributes match (logical AND). If multiple certificates match, the one with the latest expiration date (NotAfter) is used. -### macOS example +## Usage examples + +### macOS Load an identity from the login keychain by subject name: @@ -163,7 +169,7 @@ ghostunnel client \ --cacert cacert.pem ``` -### Windows example +### Windows On Windows, `--keychain-identity` and `--keychain-issuer` work the same way but search the Windows Certificate Store (the "MY" store for the current user): @@ -176,7 +182,7 @@ ghostunnel client \ --cacert cacert.pem ``` -### Certificate reloading +## Certificate reloading Keychain certificates support reloading via `SIGHUP`/`SIGUSR1` or `--timed-reload`. On reload, Ghostunnel re-queries the keychain for a @@ -184,7 +190,7 @@ certificate matching the same identity/issuer criteria. If the certificate has been updated in the keychain (e.g. renewed), the new certificate will be used for subsequent connections. -### Removing certificates +## Removing certificates **macOS**: remove an identity (certificate + private key) by Common Name: @@ -205,7 +211,7 @@ Get-ChildItem Cert:\CurrentUser\My | The `-DeleteKey` flag also removes the private key. You can alternatively use `certutil -delstore MY `. -### Troubleshooting +## Troubleshooting **macOS: certificate not found** - Check the keychain search list: `security list-keychains` @@ -219,5 +225,8 @@ use `certutil -delstore MY `. - Make sure the CN or serial matches what you passed to `--keychain-identity` **Access denied / permission errors** -- **macOS**: the keychain may prompt for access. Use `-A` during import to allow all apps, or grant access to Ghostunnel specifically in Keychain Access. -- **Windows**: the account running Ghostunnel needs read access to the private key. You can manage private key permissions through the Certificates MMC snap-in (right-click a certificate, then "All Tasks > Manage Private Keys"). +- **macOS**: the keychain may prompt for access. Use `-A` during import to allow all + apps, or grant access to Ghostunnel specifically in Keychain Access. +- **Windows**: the account running Ghostunnel needs read access to the private key. + You can manage private key permissions through the Certificates MMC snap-in + (right-click a certificate, then "All Tasks > Manage Private Keys"). diff --git a/docs/METRICS.md b/docs/METRICS.md index 1753b3f70f..287d9bdabc 100644 --- a/docs/METRICS.md +++ b/docs/METRICS.md @@ -63,14 +63,14 @@ information on profiling via pprof, see the [`runtime/pprof`][pprof] and [http-pprof]: https://pkg.go.dev/net/http/pprof [pprof-bug]: https://github.com/golang/go/issues/20939 -### Shutdown endpoint +## Shutdown endpoint If `--enable-shutdown` is set, a `/_shutdown` endpoint is available on the status port. Sending an HTTP POST request to this endpoint will trigger a graceful shutdown of the Ghostunnel process. Any other HTTP method returns 405 Method Not Allowed. -### Backend healthchecks +## Backend healthchecks The `/_status` endpoint includes a backend healthcheck. By default, Ghostunnel performs a TCP connection check against the `--target` address. You can override @@ -85,7 +85,7 @@ The `/_status` JSON response includes: If the backend check fails, the `/_status` endpoint returns HTTP 503. -### Metric names +## Metric names Ghostunnel exports the following base metrics: @@ -104,7 +104,7 @@ The `--metrics-prefix` flag (default: `ghostunnel`) is prepended to all metric names. How the prefix and metric names are formatted depends on the output format (see below). -### JSON format (`/_metrics/json`) +## JSON format (`/_metrics/json`) JSON output uses dot-separated names. Counters and gauges are emitted as a single value. Timers are expanded into count, min/max/mean, and percentile @@ -125,7 +125,7 @@ sub-metrics: Each metric is returned as a JSON object with `timestamp`, `metric`, `value`, and `hostname` fields. -### Prometheus format (`/_metrics/prometheus`) +## Prometheus format (`/_metrics/prometheus`) Prometheus output replaces dots, dashes, and other special characters with underscores to comply with Prometheus naming conventions. All metrics are @@ -149,7 +149,7 @@ statistical gauges, and a summary histogram: | `ghostunnel_conn_handshake_timer_bucket{le="..."}` | Histogram buckets (0.50, 0.95, 0.99, 0.999) | | `ghostunnel_conn_handshake_timer_count` | Histogram observation count | -### Metrics export +## Metrics export Metrics are always available via the status port endpoints (`/_metrics/json`, `/_metrics/prometheus`). Additionally, metrics can be pushed to external systems: @@ -159,7 +159,7 @@ Metrics are always available via the status port endpoints (`/_metrics/json`, * `--metrics-url=URL` — push via HTTP POST (JSON format) at the interval set by `--metrics-interval` (default: 30s) -### Exposing status port with HTTP instead of HTTPS +## Exposing status port with HTTP instead of HTTPS By default, Ghostunnel uses HTTPS for the status port. You can force it to use HTTP by prefixing the status address with "http://". diff --git a/docs/SOCKET-ACTIVATION.md b/docs/SOCKET-ACTIVATION.md index 859613af36..cb66a3c50d 100644 --- a/docs/SOCKET-ACTIVATION.md +++ b/docs/SOCKET-ACTIVATION.md @@ -12,7 +12,7 @@ your systemd/launchd configuration. Note that socket activation is not available on Windows. -### launchd +## launchd See Apple's [Creating Launch Daemons and Agents][launchd-guide] for background on launchd plists. @@ -79,7 +79,7 @@ defined for each socket. If for example the family were to be left out, launchd would open two sockets (IPv4 and IPv6) for the given key (like `Listener`) and pass them to Ghostunnel which is not currently supported. -### systemd +## systemd See the [`systemd.socket`][systemd-socket] man page for the full socket unit reference. diff --git a/docs/SPIFFE-WORKLOAD-API.md b/docs/SPIFFE-WORKLOAD-API.md index 892c2c8bb8..3740047c1e 100644 --- a/docs/SPIFFE-WORKLOAD-API.md +++ b/docs/SPIFFE-WORKLOAD-API.md @@ -38,10 +38,11 @@ ghostunnel server \ --allow-uri spiffe://domain.test/frontend ``` -### Authorization +## Authorization -The identity of the peer, i.e. the [SPIFFE ID](https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE-ID.md), is embedded as a URI SAN on the -X509-SVID. Accordingly, the existing `--verify-uri` and `--allow-uri` +The identity of the peer, i.e. the +[SPIFFE ID](https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE-ID.md), +is embedded as a URI SAN on the X509-SVID. Accordingly, the existing `--verify-uri` and `--allow-uri` flags can be used to authorize the peer: As a server: @@ -64,7 +65,7 @@ ghostunnel client \ --verify-uri spiffe://domain.test/backend ``` -### Trust bundle updates +## Trust bundle updates When using the Workload API, Ghostunnel automatically watches for updates to both the X.509 identity (certificate and key) and the trusted root CA @@ -72,7 +73,7 @@ bundle. When the SPIFFE provider (e.g. SPIRE) rotates certificates or updates the trust bundle, Ghostunnel picks up the changes without requiring a manual reload or restart. -### Demo +## Demo See the [end-to-end demo](https://github.com/ghostunnel/ghostunnel/tree/master/docs/spiffe-workload-api-demo) for an example using Ghostunnel with SPIFFE Workload API support backed by diff --git a/docs/WATCHDOG.md b/docs/WATCHDOG.md index be15e98905..6bd0ddfe91 100644 --- a/docs/WATCHDOG.md +++ b/docs/WATCHDOG.md @@ -8,7 +8,7 @@ Ghostunnel supports systemd's [notify][sd-notify] and watchdog functionality on Linux. This allows systemd to know when Ghostunnel is ready and to automatically restart it if it becomes unresponsive. -### How it works +## How it works When running as a [`Type=notify-reload`][systemd-service] service: @@ -23,7 +23,7 @@ When running as a [`Type=notify-reload`][systemd-service] service: `SIGHUP` to the process, which triggers a certificate reload (same as sending `SIGHUP` manually). -### Example unit file +## Example unit file ```ini [Unit] @@ -45,7 +45,7 @@ Restart=always WantedBy=default.target ``` -### Notes +## Notes * `Type=notify-reload` requires systemd v253 or later. If you are on an older version, use `Type=notify` instead (reload via `systemctl reload` will not From 3637ad622aeae4069c96fbea697bb8048972a02b Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:42:32 -0700 Subject: [PATCH 05/16] Add more links between docs for more info --- docs/FLAGS.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/FLAGS.md b/docs/FLAGS.md index 217e842dbb..50591e5137 100644 --- a/docs/FLAGS.md +++ b/docs/FLAGS.md @@ -69,6 +69,9 @@ See [Metrics]({{< ref "METRICS.md" >}}). ### Status / Logging +See [Metrics & Profiling]({{< ref "METRICS.md" >}}) for details on the status port, +metrics endpoints, and profiling. + | Flag | Description | Availability | |------|-------------|--------------| | `--status ADDR` | Enable `/_status` and `/_metrics` on given HOST:PORT (or `unix:SOCKET`). | All platforms | @@ -80,6 +83,9 @@ See [Metrics]({{< ref "METRICS.md" >}}). ### Landlock +See [Security & TLS Configuration]({{< ref "SECURITY.md" >}}) for details on +Landlock sandboxing. + | Flag | Description | Availability | |------|-------------|--------------| | `--disable-landlock` | Disable the best-effort Landlock sandboxing. Landlock is automatically disabled when PKCS#11 is used. | Linux only | @@ -90,6 +96,9 @@ Flags specific to `ghostunnel server`. ### Required +See [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}) for `systemd:NAME` and +`launchd:NAME` addresses. + | Flag | Description | |------|-------------| | `--listen ADDR` | Address and port to listen on (`HOST:PORT`, `unix:PATH`, `systemd:NAME`, or `launchd:NAME`). | @@ -101,7 +110,7 @@ Flags specific to `ghostunnel server`. |------|-------------| | `--target-status URL` | Address to target for status checking downstream healthchecks. Defaults to TCP healthcheck if not passed. | | `--proxy-protocol` | Enable PROXY protocol v2 to signal connection info to backend. | -| `--unsafe-target` | Do not limit target to localhost, `127.0.0.1`, `[::1]`, or UNIX sockets. | +| `--unsafe-target` | Do not limit target to localhost, `127.0.0.1`, `[::1]`, or UNIX sockets. See [Security]({{< ref "SECURITY.md" >}}). | ### Access Control @@ -130,6 +139,8 @@ See [ACME Support]({{< ref "ACME.md" >}}). ### OPA Policy (Server) +See [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}) for OPA/Rego policy details. + | Flag | Description | |------|-------------| | `--allow-policy BUNDLE` | Location of an OPA policy bundle. | @@ -141,6 +152,9 @@ Flags specific to `ghostunnel client`. ### Required +See [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}) for `systemd:NAME` and +`launchd:NAME` addresses. + | Flag | Description | |------|-------------| | `--listen ADDR` | Address and port to listen on (`HOST:PORT`, `unix:PATH`, `systemd:NAME`, or `launchd:NAME`). | @@ -150,7 +164,7 @@ Flags specific to `ghostunnel client`. | Flag | Description | |------|-------------| -| `--unsafe-listen` | Do not limit listen to localhost, `127.0.0.1`, `[::1]`, or UNIX sockets. | +| `--unsafe-listen` | Do not limit listen to localhost, `127.0.0.1`, `[::1]`, or UNIX sockets. See [Security]({{< ref "SECURITY.md" >}}). | | `--override-server-name NAME` | Override the server name used for hostname verification. | | `--proxy URL` | Connect to target over given proxy (HTTP CONNECT or SOCKS5). Must be a proxy URL. | | `--disable-authentication` | Disable client authentication, no certificate will be provided to the server. | @@ -168,6 +182,8 @@ See [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}). ### OPA Policy (Client) +See [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}) for OPA/Rego policy details. + | Flag | Description | |------|-------------| | `--verify-policy BUNDLE` | Location of an OPA policy bundle. | From 2ac3dcf103e7f83a5ad8c47db2c4e98e3db4de58 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:48:53 -0700 Subject: [PATCH 06/16] Add certificate formats doc --- docs/CERTIFICATES.md | 172 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 docs/CERTIFICATES.md diff --git a/docs/CERTIFICATES.md b/docs/CERTIFICATES.md new file mode 100644 index 0000000000..7bf9b9b59a --- /dev/null +++ b/docs/CERTIFICATES.md @@ -0,0 +1,172 @@ +--- +title: Certificate Formats +description: Supported certificate and key formats, how to prepare them, and how Ghostunnel selects the right loader. +weight: 12 +--- + +Ghostunnel supports several certificate and private key formats. The format +is auto-detected from the file extension or by inspecting the first few +bytes, so you don't need to specify it explicitly. + +## Formats at a glance + +| Format | Extensions | Flag | Notes | +|--------|-----------|------|-------| +| PEM (separate files) | `.pem`, `.crt` + `.pem` | `--cert` + `--key` | Most common; leaf cert must be first in chain | +| PEM (combined) | `.pem` | `--keystore` | Single file with cert chain and private key | +| PKCS#12 | `.p12`, `.pfx` | `--keystore` | Binary bundle; optional `--storepass` for password | +| JCEKS | `.jceks`, `.jks` | `--keystore` | Java keystore; requires `--storepass` | +| DER | `.der` | `--keystore` | Raw X.509 or PKCS#7; less common | + +These options are mutually exclusive with each other and with `--use-workload-api`, +`--keychain-identity`, and PKCS#11 flags. + +## PEM files (separate cert and key) + +Pass the certificate chain and private key as two separate PEM files: + +```bash +ghostunnel server \ + --cert server-chain.pem \ + --key server-key.pem \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert cacert.pem \ + --allow-cn client +``` + +The certificate file must contain the **leaf certificate first**, followed by +any intermediate CA certificates: + +``` +-----BEGIN CERTIFICATE----- +(leaf / end-entity certificate) +-----END CERTIFICATE----- +-----BEGIN CERTIFICATE----- +(intermediate CA certificate) +-----END CERTIFICATE----- +``` + +The key file must contain a single PEM-encoded private key (RSA, ECDSA, +or Ed25519). + +## PEM keystore (combined file) + +A single PEM file containing both the certificate chain and private key can +be passed with `--keystore`. The private key can appear anywhere in the file, +but the leaf certificate must still come before any intermediates: + +```bash +ghostunnel server \ + --keystore server-combined.pem \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert cacert.pem \ + --allow-cn client +``` + +To create a combined PEM file: + +```bash +cat server-cert.pem intermediate.pem server-key.pem > server-combined.pem +``` + +## PKCS#12 + +PKCS#12 (`.p12` / `.pfx`) bundles the certificate chain and private key into a +single password-protected binary file. This is also the format used when +importing into the macOS Keychain or Windows Certificate Store (see +[Keychain Support]({{< ref "KEYCHAIN.md" >}})). + +```bash +ghostunnel server \ + --keystore server.p12 \ + --storepass \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert cacert.pem \ + --allow-cn client +``` + +To create a PKCS#12 file from PEM files: + +```bash +openssl pkcs12 -export \ + -in server-cert.pem \ + -inkey server-key.pem \ + -certfile intermediate.pem \ + -out server.p12 \ + -passout pass: +``` + +See the [openssl-pkcs12][openssl-pkcs12] man page for all options. + +[openssl-pkcs12]: https://docs.openssl.org/master/man1/openssl-pkcs12/ + +## JCEKS + +Ghostunnel can read Java keystores in JCEKS or JKS format. This is mainly +useful when migrating from a Java-based TLS terminator: + +```bash +ghostunnel server \ + --keystore server.jceks \ + --storepass \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cacert cacert.pem \ + --allow-cn client +``` + +## CA bundle + +The `--cacert` flag accepts a PEM file containing one or more trusted CA +certificates. If omitted, Ghostunnel uses the system trust store. + +To build a CA bundle from individual certificates: + +```bash +cat root-ca.pem intermediate-ca.pem > cacert.pem +``` + +## Format auto-detection + +Ghostunnel detects the format in this order: + +1. **File extension**: `.pem`/`.crt` → PEM, `.p12`/`.pfx` → PKCS#12, + `.jceks`/`.jks` → JCEKS, `.der` → DER. +2. **Magic bytes**: if the extension is ambiguous, the first bytes of the file + are inspected (e.g. `-----BEGIN` → PEM, ASN.1 sequence → PKCS#12 or DER). + +In practice, just use the right file extension and Ghostunnel will do the +right thing. + +## Common operations + +### Inspect a PEM certificate + +```bash +openssl x509 -in server-cert.pem -noout -text +``` + +### Inspect a PKCS#12 file + +```bash +openssl pkcs12 -in server.p12 -info -nokeys +``` + +### Convert PKCS#12 to PEM + +```bash +# Extract certificate chain +openssl pkcs12 -in server.p12 -clcerts -nokeys -out server-cert.pem + +# Extract private key +openssl pkcs12 -in server.p12 -nocerts -nodes -out server-key.pem +``` + +### Verify a certificate chain + +```bash +openssl verify -CAfile cacert.pem server-cert.pem +``` From 56197e5c380bec7397ae9dbb6e028e87fe5fe2de Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:49:02 -0700 Subject: [PATCH 07/16] Index and better wording for docs --- docs/BETTER_DOCS.md | 49 +++++++++++++++++++++++++++++++++++++ docs/FLAGS.md | 3 +++ docs/HSM-PKCS11.md | 5 ++-- docs/METRICS.md | 10 ++++---- docs/SECURITY.md | 4 +-- docs/SPIFFE-WORKLOAD-API.md | 5 ++-- docs/_index.md | 16 ++++++++++++ 7 files changed, 80 insertions(+), 12 deletions(-) create mode 100644 docs/BETTER_DOCS.md diff --git a/docs/BETTER_DOCS.md b/docs/BETTER_DOCS.md new file mode 100644 index 0000000000..c3da9f0332 --- /dev/null +++ b/docs/BETTER_DOCS.md @@ -0,0 +1,49 @@ +# Documentation Improvements + +Ideas for making the Ghostunnel documentation more useful and complete. + +## Structure / Navigation + +- `_index.md` could include a short "which doc do I need?" guide, e.g. "want mTLS + between services? start here", "need automatic certs? see ACME", "managing certs + through the OS? see Keychain or HSM". Right now there's no guided entry point. +- The FLAGS.md quick reference could link each flag row to the relevant section in + the detailed doc (e.g. `--proxy-protocol` links to a PROXY protocol section, + `--landlock` links to SECURITY.md). + +## Missing Topics + +- **PROXY protocol v2**: `--proxy-protocol` is mentioned in FLAGS.md but there's no + explanation of what it does, when you'd use it, or how the backend should consume + the header. +- **`--proxy` flag (HTTP CONNECT / SOCKS5)**: useful for corporate environments, + worth at least a short section covering client-mode proxy support. +- **Certificate formats**: the differences between `--keystore` (PKCS#12 or combined + PEM), `--cert`/`--key` (separate PEM), and how cert chains should be ordered + (leaf first). This is a common source of confusion. +- **Quick start / tutorial**: every doc jumps straight into flags and reference. A + single "get ghostunnel running in 5 minutes" page with a self-signed CA would + help newcomers. +- **Graceful shutdown behavior**: `--shutdown-timeout`, what happens to in-flight + connections, how SIGTERM/SIGINT are handled, how `/_shutdown` interacts with this. + +## Existing Content Gaps + +- `SECURITY.md` lists cipher suites but doesn't explain why these were chosen or how + to verify what's negotiated (e.g. `openssl s_client -connect`). +- `ACME.md` doesn't mention how to revoke or force-renew a certificate, or where + exactly certmagic stores things on each OS. +- `METRICS.md` doesn't show a Grafana/Prometheus scrape config example, which would + make the Prometheus endpoint more immediately useful. +- `SOCKET-ACTIVATION.md` shows full unit files but doesn't explain how to actually + install and enable them (`systemctl enable`, `launchctl load`). +- The OPA section in `ACCESS-FLAGS.md` doesn't explain how to build a bundle from a + `.rego` file (`opa build`), just says "policy bundle must be present on disk." + +## Consistency + +- Some docs use `###` headings exclusively (KEYCHAIN, ACME, WATCHDOG), others mix + `##` and `###` (ACCESS-FLAGS, METRICS, SECURITY). Standardizing would help. +- The Secure Enclave section in KEYCHAIN.md only shows a `server` example; the + existing macOS/Windows examples only show `client`. Having both modes in each + section (or a note that the flags work identically in both) would reduce guesswork. diff --git a/docs/FLAGS.md b/docs/FLAGS.md index 50591e5137..00eeb1e00e 100644 --- a/docs/FLAGS.md +++ b/docs/FLAGS.md @@ -13,6 +13,9 @@ These flags are available in both `server` and `client` modes. ### Certificate / Key +See [Certificate Formats]({{< ref "CERTIFICATES.md" >}}) for details on +supported file formats and chain ordering. + | Flag | Description | |------|-------------| | `--keystore PATH` | Path to keystore (combined PEM with cert/key, or PKCS12 keystore). | diff --git a/docs/HSM-PKCS11.md b/docs/HSM-PKCS11.md index cc0c5f590a..d64d8c7fc3 100644 --- a/docs/HSM-PKCS11.md +++ b/docs/HSM-PKCS11.md @@ -52,8 +52,9 @@ want to show the PIN on the command line. Note that `--cert` needs to point to the certificate chain that corresponds to the private key in the PKCS#11 module, with the leaf certificate being the -first certificate in the chain. Ghostunnel currently cannot read the -certificate chain directly from the module. +first certificate in the chain (see +[Certificate Formats]({{< ref "CERTIFICATES.md" >}})). Ghostunnel currently +cannot read the certificate chain directly from the module. ## Using a YubiKey diff --git a/docs/METRICS.md b/docs/METRICS.md index 287d9bdabc..18496cfad4 100644 --- a/docs/METRICS.md +++ b/docs/METRICS.md @@ -79,9 +79,9 @@ instead. Ghostunnel expects an HTTP 200 response. The `/_status` JSON response includes: -* `backend_ok` — boolean indicating if the backend check passed -* `backend_status` — string of `ok` or `critical` -* `backend_error` — string of error message if the check failed +* `backend_ok`: boolean indicating if the backend check passed +* `backend_status`: string of `ok` or `critical` +* `backend_error`: string of error message if the check failed If the backend check fails, the `/_status` endpoint returns HTTP 503. @@ -154,9 +154,9 @@ statistical gauges, and a summary histogram: Metrics are always available via the status port endpoints (`/_metrics/json`, `/_metrics/prometheus`). Additionally, metrics can be pushed to external systems: -* `--metrics-graphite=ADDR` — push to a Graphite instance via raw TCP +* `--metrics-graphite=ADDR`: push to a Graphite instance via raw TCP (dot-separated names, same as JSON format) -* `--metrics-url=URL` — push via HTTP POST (JSON format) at the interval set by +* `--metrics-url=URL`: push via HTTP POST (JSON format) at the interval set by `--metrics-interval` (default: 30s) ## Exposing status port with HTTP instead of HTTPS diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 0a2b18c29f..0d54fe1208 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -42,8 +42,8 @@ affects TLS 1.2 connections. In server mode, key exchange prefers the following elliptic curves: -1. **X25519** — fast, constant-time, widely supported -2. **P-256 (secp256r1)** — hardware-accelerated on most platforms +1. **X25519**: fast, constant-time, widely supported +2. **P-256 (secp256r1)**: hardware-accelerated on most platforms ### Client authentication diff --git a/docs/SPIFFE-WORKLOAD-API.md b/docs/SPIFFE-WORKLOAD-API.md index 3740047c1e..9eaa01bc5d 100644 --- a/docs/SPIFFE-WORKLOAD-API.md +++ b/docs/SPIFFE-WORKLOAD-API.md @@ -77,8 +77,7 @@ a manual reload or restart. See the [end-to-end demo](https://github.com/ghostunnel/ghostunnel/tree/master/docs/spiffe-workload-api-demo) for an example using Ghostunnel with SPIFFE Workload API support backed by -[SPIRE](https://spiffe.io/spire/). If you are new to SPIFFE/SPIRE, the -[SPIRE getting started guide][spire-getting-started] walks through a basic -setup on Linux. +[SPIRE](https://spiffe.io/spire/). The [SPIRE getting started guide][spire-getting-started] covers setting up +SPIRE from scratch on Linux/macOS. [spire-getting-started]: https://spiffe.io/docs/latest/try/getting-started-linux-macos-x/ diff --git a/docs/_index.md b/docs/_index.md index 1a88170e1d..61c662d1c3 100644 --- a/docs/_index.md +++ b/docs/_index.md @@ -6,3 +6,19 @@ weight: 10 Documentation for Ghostunnel, covering access control, certificate management, metrics, and platform-specific features. + +## Where to start + +| I want to... | Start here | +|--------------|------------| +| See all available flags | [Command-Line Flags]({{< ref "FLAGS.md" >}}) | +| Understand cert/key file formats (PEM, PKCS#12, JCEKS) | [Certificate Formats]({{< ref "CERTIFICATES.md" >}}) | +| Set up mTLS between services | [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}) | +| Get automatic certs from Let's Encrypt | [ACME Support]({{< ref "ACME.md" >}}) | +| Use certificates from the macOS Keychain or Windows Certificate Store | [Keychain Support]({{< ref "KEYCHAIN.md" >}}) | +| Store private keys in a hardware security module | [HSM/PKCS#11 Support]({{< ref "HSM-PKCS11.md" >}}) | +| Use SPIFFE/SPIRE for workload identity | [SPIFFE Workload API]({{< ref "SPIFFE-WORKLOAD-API.md" >}}) | +| Understand the TLS settings and security model | [Security & TLS Configuration]({{< ref "SECURITY.md" >}}) | +| Monitor connections or scrape Prometheus metrics | [Metrics & Profiling]({{< ref "METRICS.md" >}}) | +| Run Ghostunnel via systemd or launchd | [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}) | +| Set up systemd watchdog integration | [Systemd Watchdog]({{< ref "WATCHDOG.md" >}}) | From cb7fc9805a67d965effde044afff158bb59e0b0e Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:56:47 -0700 Subject: [PATCH 08/16] Remove file that was added by mistake, get rid of table in index (seems redundant) --- docs/BETTER_DOCS.md | 49 --------------------------------------------- docs/_index.md | 16 --------------- 2 files changed, 65 deletions(-) delete mode 100644 docs/BETTER_DOCS.md diff --git a/docs/BETTER_DOCS.md b/docs/BETTER_DOCS.md deleted file mode 100644 index c3da9f0332..0000000000 --- a/docs/BETTER_DOCS.md +++ /dev/null @@ -1,49 +0,0 @@ -# Documentation Improvements - -Ideas for making the Ghostunnel documentation more useful and complete. - -## Structure / Navigation - -- `_index.md` could include a short "which doc do I need?" guide, e.g. "want mTLS - between services? start here", "need automatic certs? see ACME", "managing certs - through the OS? see Keychain or HSM". Right now there's no guided entry point. -- The FLAGS.md quick reference could link each flag row to the relevant section in - the detailed doc (e.g. `--proxy-protocol` links to a PROXY protocol section, - `--landlock` links to SECURITY.md). - -## Missing Topics - -- **PROXY protocol v2**: `--proxy-protocol` is mentioned in FLAGS.md but there's no - explanation of what it does, when you'd use it, or how the backend should consume - the header. -- **`--proxy` flag (HTTP CONNECT / SOCKS5)**: useful for corporate environments, - worth at least a short section covering client-mode proxy support. -- **Certificate formats**: the differences between `--keystore` (PKCS#12 or combined - PEM), `--cert`/`--key` (separate PEM), and how cert chains should be ordered - (leaf first). This is a common source of confusion. -- **Quick start / tutorial**: every doc jumps straight into flags and reference. A - single "get ghostunnel running in 5 minutes" page with a self-signed CA would - help newcomers. -- **Graceful shutdown behavior**: `--shutdown-timeout`, what happens to in-flight - connections, how SIGTERM/SIGINT are handled, how `/_shutdown` interacts with this. - -## Existing Content Gaps - -- `SECURITY.md` lists cipher suites but doesn't explain why these were chosen or how - to verify what's negotiated (e.g. `openssl s_client -connect`). -- `ACME.md` doesn't mention how to revoke or force-renew a certificate, or where - exactly certmagic stores things on each OS. -- `METRICS.md` doesn't show a Grafana/Prometheus scrape config example, which would - make the Prometheus endpoint more immediately useful. -- `SOCKET-ACTIVATION.md` shows full unit files but doesn't explain how to actually - install and enable them (`systemctl enable`, `launchctl load`). -- The OPA section in `ACCESS-FLAGS.md` doesn't explain how to build a bundle from a - `.rego` file (`opa build`), just says "policy bundle must be present on disk." - -## Consistency - -- Some docs use `###` headings exclusively (KEYCHAIN, ACME, WATCHDOG), others mix - `##` and `###` (ACCESS-FLAGS, METRICS, SECURITY). Standardizing would help. -- The Secure Enclave section in KEYCHAIN.md only shows a `server` example; the - existing macOS/Windows examples only show `client`. Having both modes in each - section (or a note that the flags work identically in both) would reduce guesswork. diff --git a/docs/_index.md b/docs/_index.md index 61c662d1c3..1a88170e1d 100644 --- a/docs/_index.md +++ b/docs/_index.md @@ -6,19 +6,3 @@ weight: 10 Documentation for Ghostunnel, covering access control, certificate management, metrics, and platform-specific features. - -## Where to start - -| I want to... | Start here | -|--------------|------------| -| See all available flags | [Command-Line Flags]({{< ref "FLAGS.md" >}}) | -| Understand cert/key file formats (PEM, PKCS#12, JCEKS) | [Certificate Formats]({{< ref "CERTIFICATES.md" >}}) | -| Set up mTLS between services | [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}) | -| Get automatic certs from Let's Encrypt | [ACME Support]({{< ref "ACME.md" >}}) | -| Use certificates from the macOS Keychain or Windows Certificate Store | [Keychain Support]({{< ref "KEYCHAIN.md" >}}) | -| Store private keys in a hardware security module | [HSM/PKCS#11 Support]({{< ref "HSM-PKCS11.md" >}}) | -| Use SPIFFE/SPIRE for workload identity | [SPIFFE Workload API]({{< ref "SPIFFE-WORKLOAD-API.md" >}}) | -| Understand the TLS settings and security model | [Security & TLS Configuration]({{< ref "SECURITY.md" >}}) | -| Monitor connections or scrape Prometheus metrics | [Metrics & Profiling]({{< ref "METRICS.md" >}}) | -| Run Ghostunnel via systemd or launchd | [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}) | -| Set up systemd watchdog integration | [Systemd Watchdog]({{< ref "WATCHDOG.md" >}}) | From 83d54d709cd3cfaf85d07b3ef94ebfb3b6f24fb0 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 13:57:27 -0700 Subject: [PATCH 09/16] Fix content gaps, make existing docs more comprehensive --- docs/ACCESS-FLAGS.md | 9 +++++++++ docs/ACME.md | 28 ++++++++++++++++++++++++---- docs/METRICS.md | 21 +++++++++++++++++++++ docs/SECURITY.md | 9 +++++++++ docs/SOCKET-ACTIVATION.md | 30 ++++++++++++++++++++++++++++++ 5 files changed, 93 insertions(+), 4 deletions(-) diff --git a/docs/ACCESS-FLAGS.md b/docs/ACCESS-FLAGS.md index 958e84fff5..26fc7e573b 100644 --- a/docs/ACCESS-FLAGS.md +++ b/docs/ACCESS-FLAGS.md @@ -144,6 +144,15 @@ certificates, with the `--timed-reload` flag or via `SIGHUP`. [opa]: https://www.openpolicyagent.org/ [opa-bundles]: https://www.openpolicyagent.org/docs/latest/management-bundles/ +To build a bundle from a `.rego` file, use the `opa build` command: + +```bash +opa build -b policy.rego -o bundle.tar.gz +``` + +See the [OPA bundle documentation][opa-bundles] for details on bundle +structure and manifest options. + To use it in server mode, specify the `--allow-policy` and `--allow-query` flags. Example: diff --git a/docs/ACME.md b/docs/ACME.md index ed38132164..93156e96e0 100644 --- a/docs/ACME.md +++ b/docs/ACME.md @@ -43,14 +43,34 @@ disabled), so port 443 must be reachable. ## Certificate storage and renewal -Certificates are stored locally by certmagic in its default storage directory -(typically `~/.local/share/certmagic` or the equivalent on your OS). Certmagic -automatically renews certificates before they expire, so no manual intervention -or `--timed-reload` is needed for ACME certificates. +Certmagic stores certificates and account keys on disk. The default location +depends on your OS: + +| OS | Default path | +|----|-------------| +| Linux / macOS | `~/.local/share/certmagic` (or `$XDG_DATA_HOME/certmagic`) | +| Windows | `%APPDATA%\certmagic` | + +Certmagic automatically renews certificates before they expire, so no manual +intervention or `--timed-reload` is needed for ACME certificates. If a valid certificate already exists locally, Ghostunnel loads it from cache on startup without contacting the CA. +## Revoking or force-renewing + +Certmagic handles renewal automatically, but if you need to force a renewal +(e.g. after a key compromise), delete the certificate and key files from the +certmagic storage directory and restart Ghostunnel. It will obtain a fresh +certificate on startup. + +To revoke a certificate with Let's Encrypt directly, use the +[certbot revoke][certbot-revoke] command or the ACME revocation endpoint +described in [RFC 8555 Section 7.6][acme-revoke]. + +[certbot-revoke]: https://eff-certbot.readthedocs.io/en/latest/using.html#revoking-certificates +[acme-revoke]: https://datatracker.ietf.org/doc/html/rfc8555#section-7.6 + ## Startup retry behavior On startup, Ghostunnel attempts to obtain the initial certificate up to 5 diff --git a/docs/METRICS.md b/docs/METRICS.md index 18496cfad4..5fade24e52 100644 --- a/docs/METRICS.md +++ b/docs/METRICS.md @@ -149,6 +149,27 @@ statistical gauges, and a summary histogram: | `ghostunnel_conn_handshake_timer_bucket{le="..."}` | Histogram buckets (0.50, 0.95, 0.99, 0.999) | | `ghostunnel_conn_handshake_timer_count` | Histogram observation count | +### Prometheus scrape config + +To scrape Ghostunnel metrics with Prometheus, add a job to your +`prometheus.yml`: + +```yaml +scrape_configs: + - job_name: ghostunnel + scheme: https + tls_config: + ca_file: /path/to/cacert.pem + cert_file: /path/to/client-cert.pem + key_file: /path/to/client-key.pem + metrics_path: /_metrics/prometheus + static_configs: + - targets: ['localhost:6060'] +``` + +If the status port uses HTTP (see below), set `scheme: http` and drop the +`tls_config` block. + ## Metrics export Metrics are always available via the status port endpoints (`/_metrics/json`, diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 0d54fe1208..da12ba9bbb 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -33,6 +33,15 @@ All suites use authenticated encryption (AEAD). CBC-mode ciphers are not enabled. ECDSA suites are listed before RSA to prefer ECDSA when both certificate types are available. +To check which cipher suite and protocol version were negotiated for a +connection: + +```bash +openssl s_client -connect localhost:8443 \ + -cert client-cert.pem -key client-key.pem -CAfile cacert.pem \ + /dev/null | grep -E 'Protocol|Cipher' +``` + In TLS 1.3, cipher suite selection is handled by Go's [`crypto/tls`][crypto-tls] and cannot be configured by the application. The TLS 1.3 suites listed above are always available when TLS 1.3 is negotiated. The configurable cipher suite list only diff --git a/docs/SOCKET-ACTIVATION.md b/docs/SOCKET-ACTIVATION.md index cb66a3c50d..c64b57c698 100644 --- a/docs/SOCKET-ACTIVATION.md +++ b/docs/SOCKET-ACTIVATION.md @@ -79,6 +79,22 @@ defined for each socket. If for example the family were to be left out, launchd would open two sockets (IPv4 and IPv6) for the given key (like `Listener`) and pass them to Ghostunnel which is not currently supported. +To install and enable: + +```bash +# Copy the plist into place +sudo cp com.square.ghostunnel.plist /Library/LaunchDaemons/ + +# Load and start +sudo launchctl load /Library/LaunchDaemons/com.square.ghostunnel.plist + +# Stop and unload +sudo launchctl unload /Library/LaunchDaemons/com.square.ghostunnel.plist +``` + +Use `~/Library/LaunchAgents/` instead of `/Library/LaunchDaemons/` if running +as a user agent rather than a system daemon. + ## systemd See the [`systemd.socket`][systemd-socket] man page for the full socket unit @@ -121,6 +137,20 @@ Note that the `FileDescriptorName` in `ghostunnel.socket` matches the name passe `--listen`. If multiple sockets are needed, e.g. for a status port, the name can be used to distinguish the listening and status sockets. +To install and enable: + +```bash +# Copy unit files into place +sudo cp ghostunnel.socket ghostunnel.service /etc/systemd/system/ + +# Reload, enable, and start the socket +sudo systemctl daemon-reload +sudo systemctl enable --now ghostunnel.socket +``` + +systemd will start `ghostunnel.service` on demand when a connection arrives +on the socket. + Ghostunnel also supports systemd notify and watchdog functionality. See [WATCHDOG]({{< ref "WATCHDOG.md" >}}) for details on configuring `Type=notify-reload` services. From 4691fbc36e1dcb741d5634db4c027bddec3dc1e2 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 14:02:10 -0700 Subject: [PATCH 10/16] Fix some mistakes in the docs --- docs/ACCESS-FLAGS.md | 2 +- docs/ACME.md | 2 +- docs/CERTIFICATES.md | 5 ++++- docs/KEYCHAIN.md | 4 ++-- 4 files changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/ACCESS-FLAGS.md b/docs/ACCESS-FLAGS.md index 26fc7e573b..5432ee580c 100644 --- a/docs/ACCESS-FLAGS.md +++ b/docs/ACCESS-FLAGS.md @@ -147,7 +147,7 @@ certificates, with the `--timed-reload` flag or via `SIGHUP`. To build a bundle from a `.rego` file, use the `opa build` command: ```bash -opa build -b policy.rego -o bundle.tar.gz +opa build policy.rego -o bundle.tar.gz ``` See the [OPA bundle documentation][opa-bundles] for details on bundle diff --git a/docs/ACME.md b/docs/ACME.md index 93156e96e0..07bc1f9deb 100644 --- a/docs/ACME.md +++ b/docs/ACME.md @@ -49,7 +49,7 @@ depends on your OS: | OS | Default path | |----|-------------| | Linux / macOS | `~/.local/share/certmagic` (or `$XDG_DATA_HOME/certmagic`) | -| Windows | `%APPDATA%\certmagic` | +| Windows | `%USERPROFILE%\.local\share\certmagic` | Certmagic automatically renews certificates before they expire, so no manual intervention or `--timed-reload` is needed for ACME certificates. diff --git a/docs/CERTIFICATES.md b/docs/CERTIFICATES.md index 7bf9b9b59a..d025acc67a 100644 --- a/docs/CERTIFICATES.md +++ b/docs/CERTIFICATES.md @@ -158,9 +158,12 @@ openssl pkcs12 -in server.p12 -info -nokeys ### Convert PKCS#12 to PEM ```bash -# Extract certificate chain +# Extract the leaf certificate openssl pkcs12 -in server.p12 -clcerts -nokeys -out server-cert.pem +# Extract CA/intermediate certificates +openssl pkcs12 -in server.p12 -cacerts -nokeys -out ca-chain.pem + # Extract private key openssl pkcs12 -in server.p12 -nocerts -nodes -out server-key.pem ``` diff --git a/docs/KEYCHAIN.md b/docs/KEYCHAIN.md index 19a3e160bc..7f36078902 100644 --- a/docs/KEYCHAIN.md +++ b/docs/KEYCHAIN.md @@ -100,8 +100,8 @@ certutil -f -p -user -importpfx MY server.p12 ``` This imports the certificate and private key into the current user's "MY" -(Personal) store. The `-user` flag targets the current user context; replace -it with `-enterprise` to import into the Local Machine store instead. +(Personal) store. The `-user` flag targets the current user context. To import +into the Local Machine store instead, omit `-user` (and run as administrator). **Using PowerShell**: From 424f198deedce8f1504678bf02d5308f62fa0f1c Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 14:29:48 -0700 Subject: [PATCH 11/16] Add quick start guide and docker images docs --- docs/DOCKER.md | 60 ++++++++++++++++++ docs/QUICKSTART.md | 129 ++++++++++++++++++++++++++++++++++++++ docs/tunnel-diagram.dot | 15 +++++ website/content/_index.md | 52 ++------------- 4 files changed, 208 insertions(+), 48 deletions(-) create mode 100644 docs/DOCKER.md create mode 100644 docs/QUICKSTART.md create mode 100644 docs/tunnel-diagram.dot diff --git a/docs/DOCKER.md b/docs/DOCKER.md new file mode 100644 index 0000000000..9d57cd3c0e --- /dev/null +++ b/docs/DOCKER.md @@ -0,0 +1,60 @@ +--- +title: Docker Images +description: Available Docker image variants and tags for running Ghostunnel in containers. +weight: 90 +--- + +Docker images are published to [Docker Hub][hub] on each release. Three +variants are available: + +| Variant | Tag | Base | +|---------|-----|------| +| Alpine | `ghostunnel/ghostunnel:latest`, `ghostunnel/ghostunnel:v1.x.x` | Alpine Linux | +| Debian | `ghostunnel/ghostunnel:latest-debian`, `ghostunnel/ghostunnel:v1.x.x-debian` | Debian slim | +| Distroless | `ghostunnel/ghostunnel:latest-distroless`, `ghostunnel/ghostunnel:v1.x.x-distroless` | Google Distroless | + +The `latest` tags always point to the most recent release. + +## Pulling an image + +```bash +# Distroless (smallest, no shell) +docker pull ghostunnel/ghostunnel:latest-distroless + +# Alpine (includes shell, good for debugging) +docker pull ghostunnel/ghostunnel:latest + +# Debian (includes shell and package manager) +docker pull ghostunnel/ghostunnel:latest-debian +``` + +## Running in Docker + +Mount your certificate files into the container and pass flags as normal: + +```bash +docker run --rm \ + -v /path/to/certs:/certs:ro \ + -p 8443:8443 \ + ghostunnel/ghostunnel:latest-distroless \ + server \ + --listen 0.0.0.0:8443 \ + --target host.docker.internal:8080 \ + --cert /certs/server-cert.pem \ + --key /certs/server-key.pem \ + --cacert /certs/cacert.pem \ + --allow-cn client +``` + +Note the use of `0.0.0.0` for `--listen` (to bind all interfaces inside the +container) and `host.docker.internal` for `--target` (to reach services on +the Docker host). You may need `--unsafe-target` since `host.docker.internal` +is not localhost. + +## Building images from source + +```bash +go tool mage docker:build +``` + +[hub]: https://hub.docker.com/r/ghostunnel/ghostunnel diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md new file mode 100644 index 0000000000..9abdba921a --- /dev/null +++ b/docs/QUICKSTART.md @@ -0,0 +1,129 @@ +--- +title: Quick Start +description: Get Ghostunnel running with mTLS in 5 minutes using a self-signed CA. +weight: 5 +--- + +This guide walks through setting up a Ghostunnel server and client with mutual +TLS, using a self-signed CA for testing. + +## Install Ghostunnel + +```bash +# Homebrew +brew install ghostunnel + +# Or pull a Docker image (see Docker docs for all variants) +docker pull ghostunnel/ghostunnel:latest-distroless +``` + +Pre-built binaries are also available on the +[GitHub releases](https://github.com/ghostunnel/ghostunnel/releases) page. +See [Docker Images]({{< ref "DOCKER.md" >}}) for all available image variants. + +To build from source (requires [Go](https://go.dev/doc/install)): + +```bash +go tool mage go:build +``` + +## Generate test certificates + +You need a CA, a server certificate, and a client certificate. The rest of +this guide uses the paths from `test-keys/`, so adjust if you use a different +method. + +**From the Ghostunnel repo** (requires Go): + +```bash +go tool mage test:keys +``` + +This creates a `test-keys/` directory with everything you need: CA cert, +server cert+key, client cert+key, and PKCS#12 keystores. + +**Using [mkcert](https://github.com/FiloSottile/mkcert)**: + +```bash +mkcert -install +mkcert -cert-file test-keys/server-cert.pem -key-file test-keys/server-key.pem localhost 127.0.0.1 +mkcert -client -cert-file test-keys/client-cert.pem -key-file test-keys/client-key.pem localhost +``` + +Note: mkcert sets SANs, not CNs, so use `--allow-dns localhost` instead of +`--allow-cn client` when authorizing clients. The CA cert is at +`$(mkcert -CAROOT)/rootCA.pem`, copy it to `test-keys/cacert.pem` to match +the paths below. + +**Using OpenSSL** manually: see the [openssl-req](https://docs.openssl.org/master/man1/openssl-req/) +and [openssl-x509](https://docs.openssl.org/master/man1/openssl-x509/) docs +for creating CAs and signing certificates. + +## Start a backend service + +Ghostunnel is protocol-agnostic and works with any TCP-based protocol, not +just HTTP. For this demo we'll use a simple HTTP server as the backend: + +```bash +python3 -m http.server 8080 & +``` + +## Run Ghostunnel server + +In a new terminal, start a server that listens for TLS on port 8443 and +forwards plaintext to the backend on port 8080. Only clients with CN=client +are allowed: + +```bash +ghostunnel server \ + --listen localhost:8443 \ + --target localhost:8080 \ + --cert test-keys/server-cert.pem \ + --key test-keys/server-key.pem \ + --cacert test-keys/cacert.pem \ + --allow-cn client +``` + +## Run Ghostunnel client + +In another terminal, start a client that listens for plaintext on port 8081 +and connects to the server over TLS: + +```bash +ghostunnel client \ + --listen localhost:8081 \ + --target localhost:8443 \ + --cert test-keys/client-cert.pem \ + --key test-keys/client-key.pem \ + --cacert test-keys/cacert.pem +``` + +## Test the tunnel + +In a third terminal, send a request through the tunnel: + +```bash +curl http://localhost:8081 +``` + +You should see the directory listing from the Python HTTP server. The +connection between client and server is encrypted with mTLS, even though +curl speaks plain HTTP. + +
+ +![Tunnel diagram](/tunnel-diagram.svg) + +The Ghostunnel client accepted a plaintext connection from curl, wrapped it +in TLS with the client certificate, and forwarded it to the Ghostunnel +server. The server verified the client cert (CN=client), unwrapped TLS, and +forwarded the plaintext request to the backend. + +## Next steps + +- [Command-Line Flags]({{< ref "FLAGS.md" >}}): full flag reference +- [Certificate Formats]({{< ref "CERTIFICATES.md" >}}): PEM, PKCS#12, JCEKS, and chain ordering +- [Access Control Flags]({{< ref "ACCESS-FLAGS.md" >}}): control who can connect (CN, OU, DNS/URI SAN, OPA) +- [ACME Support]({{< ref "ACME.md" >}}): automatic certificates from Let's Encrypt +- [Metrics & Profiling]({{< ref "METRICS.md" >}}): status port, Prometheus metrics, pprof +- [Socket Activation]({{< ref "SOCKET-ACTIVATION.md" >}}) and [Systemd Watchdog]({{< ref "WATCHDOG.md" >}}): run Ghostunnel as a service diff --git a/docs/tunnel-diagram.dot b/docs/tunnel-diagram.dot new file mode 100644 index 0000000000..3d1883afe5 --- /dev/null +++ b/docs/tunnel-diagram.dot @@ -0,0 +1,15 @@ +digraph tunnel { + rankdir=LR; + bgcolor="transparent"; + node [fontname="Helvetica" fontsize=12 style=filled]; + edge [fontname="Helvetica" fontsize=10]; + + curl [label="curl\n:8081" shape=box fillcolor="#e8e8e8" color="#999999"]; + client [label="ghostunnel\nclient" shape=box fillcolor="#4a90d9" fontcolor=white color="#3a7bc8"]; + server [label="ghostunnel\nserver" shape=box fillcolor="#4a90d9" fontcolor=white color="#3a7bc8"]; + backend [label="backend\n:8080" shape=box fillcolor="#e8e8e8" color="#999999"]; + + curl -> client [label=" plaintext " color="#999999" fontcolor="#666666"]; + client -> server [label=" mTLS " color="#d94a4a" fontcolor="#d94a4a" penwidth=2]; + server -> backend [label=" plaintext " color="#999999" fontcolor="#666666"]; +} diff --git a/website/content/_index.md b/website/content/_index.md index fd6e51fd66..67106ab8b1 100644 --- a/website/content/_index.md +++ b/website/content/_index.md @@ -34,55 +34,11 @@ socket. Ghostunnel also supports UNIX domain sockets, PROXY protocol v2, systemd/launchd socket activation, and more. See the [documentation](docs/) for details. -## Install +## Getting Started -Pre-built binaries for Linux, macOS, and Windows are available under [Releases](/releases). - -Via Homebrew: - -```bash -brew install ghostunnel -``` - -Via Docker (see [Docker Hub](https://hub.docker.com/r/ghostunnel/ghostunnel) for all -available tags): - -```bash -docker pull ghostunnel/ghostunnel:latest-distroless # Distroless (recommended) -docker pull ghostunnel/ghostunnel:latest-alpine # Alpine -docker pull ghostunnel/ghostunnel:latest-debian # Debian -``` - -Compile from source (replace `VERSION` with a [release tag](https://github.com/ghostunnel/ghostunnel/releases)): - -```bash -go install github.com/ghostunnel/ghostunnel@VERSION -``` - -## Usage - -Start a Ghostunnel in server mode to proxy TLS connections to a backend: - -```bash -ghostunnel server \ - --listen :8443 \ - --target localhost:8080 \ - --keystore server-keystore.p12 \ - --cacert cacert.pem \ - --allow-cn client -``` - -Start a Ghostunnel in client mode to wrap connections in TLS: - -```bash -ghostunnel client \ - --listen localhost:8080 \ - --target example.com:8443 \ - --keystore client-combined.pem \ - --cacert cacert.pem -``` - -See [Docs](/docs) for more in-depth usage information. +See the [Quick Start](/docs/quickstart/) guide for installation, generating +test certificates, and running your first tunnel. The full documentation is +available under [Docs](/docs/). ## Supported Platforms From 550cba615b031ae53c3e32573f2e7a7ee87c08f5 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 14:31:59 -0700 Subject: [PATCH 12/16] Fix typos/wording --- docs/ACCESS-FLAGS.md | 4 ++-- docs/DOCKER.md | 2 +- docs/HSM-PKCS11.md | 8 ++++---- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/ACCESS-FLAGS.md b/docs/ACCESS-FLAGS.md index 5432ee580c..743172cff2 100644 --- a/docs/ACCESS-FLAGS.md +++ b/docs/ACCESS-FLAGS.md @@ -238,8 +238,8 @@ for more about the policy language. different process. * Older versions of Ghostunnel allowed specifying a Rego file rather than a bundle as an argument to the `--allow-policy` and `--verify-policy` flags. This - still works, but the policy will be treated as a V0 policy for compatibility - versions. It's recommended to specify a bundle so you can set the language + still works, but the policy will be treated as a V0 policy for backward + compatibility. It's recommended to specify a bundle so you can set the language version directly in the bundle manifest. * By standard OPA convention, we consider a policy to be "allowed" if the query is exactly one result with exactly one element that has the value `true`. diff --git a/docs/DOCKER.md b/docs/DOCKER.md index 9d57cd3c0e..e1d37f6f56 100644 --- a/docs/DOCKER.md +++ b/docs/DOCKER.md @@ -1,7 +1,7 @@ --- title: Docker Images description: Available Docker image variants and tags for running Ghostunnel in containers. -weight: 90 +weight: 85 --- Docker images are published to [Docker Hub][hub] on each release. Three diff --git a/docs/HSM-PKCS11.md b/docs/HSM-PKCS11.md index d64d8c7fc3..a5711dff50 100644 --- a/docs/HSM-PKCS11.md +++ b/docs/HSM-PKCS11.md @@ -30,7 +30,7 @@ softhsm2-util --id 01 \ --pin 1234 ``` -To launch Ghostunnel with the SoftHSM-backed PKCS11 key: +To launch Ghostunnel with the SoftHSM-backed PKCS#11 key: ```bash ghostunnel server \ @@ -45,8 +45,8 @@ ghostunnel server \ ``` The `--pkcs11-module`, `--pkcs11-token-label` and `--pkcs11-pin` flags can be -used to select the private key to be used from the PKCS11 module. It's also possible -to use environment variables to set PKCS11 options instead of flags (via +used to select the private key to be used from the PKCS#11 module. It's also possible +to use environment variables to set PKCS#11 options instead of flags (via `PKCS11_MODULE`, `PKCS11_TOKEN_LABEL` and `PKCS11_PIN`), useful if you don't want to show the PIN on the command line. @@ -190,7 +190,7 @@ arbitrary files and sockets. ## Inspecting PKCS#11 state -If you need to inspect the state of a PKCS11 module/token, we recommend the +If you need to inspect the state of a PKCS#11 module/token, we recommend the [`pkcs11-tool`][pkcs11-tool] utility from OpenSC. For example, it can be used to list slots or read certificate(s) from a module: From 538eccb96d95c84d8bab50507059f8f81d689182 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 14:52:08 -0700 Subject: [PATCH 13/16] Add forgotten tunnel diagram image file --- website/static/tunnel-diagram.svg | 61 +++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 website/static/tunnel-diagram.svg diff --git a/website/static/tunnel-diagram.svg b/website/static/tunnel-diagram.svg new file mode 100644 index 0000000000..0e261b08e2 --- /dev/null +++ b/website/static/tunnel-diagram.svg @@ -0,0 +1,61 @@ + + + + + + +tunnel + + +curl + +curl +:8081 + + + +client + +ghostunnel +client + + + +curl->client + + +  plaintext   + + + +server + +ghostunnel +server + + + +client->server + + +  mTLS   + + + +backend + +backend +:8080 + + + +server->backend + + +  plaintext   + + + From 7e91d35867629e7de8bbbd270ad9bf4fcf926ac0 Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 14:59:08 -0700 Subject: [PATCH 14/16] Add docs for shutdown behavior --- docs/GRACEFUL-SHUTDOWN.md | 99 +++++++++++++++++++++++++++++++++++++++ docs/METRICS.md | 5 +- 2 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 docs/GRACEFUL-SHUTDOWN.md diff --git a/docs/GRACEFUL-SHUTDOWN.md b/docs/GRACEFUL-SHUTDOWN.md new file mode 100644 index 0000000000..70a6f23612 --- /dev/null +++ b/docs/GRACEFUL-SHUTDOWN.md @@ -0,0 +1,99 @@ +--- +title: Graceful Shutdown +description: How Ghostunnel handles shutdown signals, drains in-flight connections, and force-exits after a timeout. +weight: 87 +--- + +Ghostunnel supports graceful shutdown: when a shutdown is triggered, it stops +accepting new connections and waits for existing connections to drain before +exiting. If connections do not drain within the configured timeout, the process +force-exits. + +## Shutdown triggers + +There are three ways to initiate a graceful shutdown: + +### Signals (Unix) + +On Unix (Linux, macOS), sending `SIGTERM` or `SIGINT` to the Ghostunnel +process triggers a graceful shutdown: + +```bash +# Graceful shutdown via signal +kill -TERM +kill -INT # also sent by Ctrl+C +``` + +> **Note:** `SIGHUP` and `SIGUSR1` do *not* shut down the process. They +> trigger a reload of certificates and OPA policies instead. + +### Signals (Windows) + +On Windows, only the `Interrupt` signal (Ctrl+C) triggers shutdown. There are +no reload signals on Windows, but `--timed-reload` can be used to periodically +reload certificates and OPA policies on a fixed interval. + +### HTTP endpoint (`/_shutdown`) + +If `--enable-shutdown` is set (requires `--status`), you can trigger a +shutdown via HTTP POST: + +```bash +curl -X POST --cacert test-keys/cacert.pem https://localhost:6060/_shutdown +``` + +Any HTTP method other than POST returns 405 Method Not Allowed. + +## Shutdown sequence + +When a shutdown is triggered, the following happens in order: + +1. **Status transitions to "stopping"**: the `/_status` endpoint reflects + that the process is shutting down. +2. **Status HTTP server begins shutting down**: best-effort graceful shutdown + of the internal status listener. +3. **Listener closes**: Ghostunnel stops accepting new connections. +4. **In-flight connections continue**: existing connections are not + interrupted. Data continues to flow until both sides close normally. +5. **Force-exit timer starts**: a timer begins counting down from the + `--shutdown-timeout` value (default: 5 minutes). +6. **Process exits** when either: + - All in-flight connections have drained (exit code 0), or + - The shutdown timeout fires (exit code 1). + +## Flags + +| Flag | Default | Description | +|------|---------|-------------| +| `--shutdown-timeout` | `5m` | Maximum time to wait for in-flight connections to drain. If connections are still open after this duration, the process force-exits with code 1. | +| `--enable-shutdown` | `false` | Enable the `/_shutdown` HTTP endpoint on the status port. Requires `--status`. | +| `--status` | *(none)* | HOST:PORT (or `unix:SOCKET`) for the status listener. Required for `/_shutdown`. | + +See [Command-Line Flags]({{< ref "FLAGS.md" >}}) for the full flag reference. + +## Choosing a shutdown timeout + +The default timeout of 5 minutes is deliberately generous. Consider your +workload when tuning this value: + +- **Short-lived requests** (e.g. REST APIs): a lower timeout like `30s` or + `1m` is usually sufficient. +- **Long-lived connections** (e.g. streaming, WebSocket-like traffic): you may + need to increase the timeout or accept that some connections will be + force-closed. +- **Zero-downtime deployments**: coordinate the shutdown timeout with your + orchestrator's termination grace period (e.g. Kubernetes + `terminationGracePeriodSeconds`) to avoid the orchestrator killing the + process before Ghostunnel's own timeout fires. + +Other flags like `--connect-timeout` and `--conn-max-lifetime` also influence +connection behavior and may be relevant when tuning shutdown. See +[Command-Line Flags]({{< ref "FLAGS.md" >}}) for the full list. + +## Integration with systemd + +When running as a systemd service with `Type=notify-reload`, Ghostunnel +notifies systemd of its state transitions (ready, reloading, stopping). The +graceful shutdown sequence integrates naturally with systemd's service +lifecycle. See [Systemd Watchdog]({{< ref "WATCHDOG.md" >}}) for unit file +examples and configuration details. diff --git a/docs/METRICS.md b/docs/METRICS.md index 5fade24e52..2df67c97cf 100644 --- a/docs/METRICS.md +++ b/docs/METRICS.md @@ -68,7 +68,10 @@ information on profiling via pprof, see the [`runtime/pprof`][pprof] and If `--enable-shutdown` is set, a `/_shutdown` endpoint is available on the status port. Sending an HTTP POST request to this endpoint will trigger a graceful shutdown of the Ghostunnel process. Any other HTTP method returns 405 -Method Not Allowed. +Method Not Allowed. For details on what happens after shutdown is triggered, +including signal handling, connection draining, and the `--shutdown-timeout` +flag, see +[Graceful Shutdown]({{< ref "GRACEFUL-SHUTDOWN.md" >}}). ## Backend healthchecks From 34ed2d5428bef86ae2cf4119cd7fd064603bb82b Mon Sep 17 00:00:00 2001 From: Cedric Staub Date: Fri, 17 Apr 2026 15:11:25 -0700 Subject: [PATCH 15/16] Better wording around PKI, cert generation tools --- README.md | 9 ++++++--- docs/QUICKSTART.md | 8 ++++++++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8820d71b50..c2bb42f8f6 100644 --- a/README.md +++ b/README.md @@ -55,10 +55,13 @@ Getting Started =============== To get started and play around with Ghostunnel you will need X.509 client -and server certificates. If you don't already maintain a PKI, a good way to get -started is to use a package like [cloudflare/cfssl](https://github.com/cloudflare/cfssl). +and server certificates. If you already maintain a PKI, you can use your +existing certificates. Otherwise, you can use tools like +[mkcert](https://github.com/FiloSottile/mkcert) or +[cloudflare/cfssl](https://github.com/cloudflare/cfssl) to build one. -For testing and development purposes, you can generate test certificates using: +For quick testing and development, you can also generate throwaway test +certificates using the built-in generator: # Generate test certificates and keys go tool mage test:keys diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index 9abdba921a..97dcc890e7 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -29,6 +29,10 @@ go tool mage go:build ## Generate test certificates +If you already maintain a PKI, you can skip this step and use your existing +certificates. The steps below are for generating test certificates for +testing and development purposes only. + You need a CA, a server certificate, and a client certificate. The rest of this guide uses the paths from `test-keys/`, so adjust if you use a different method. @@ -55,6 +59,10 @@ Note: mkcert sets SANs, not CNs, so use `--allow-dns localhost` instead of `$(mkcert -CAROOT)/rootCA.pem`, copy it to `test-keys/cacert.pem` to match the paths below. +**Using [cfssl](https://github.com/cloudflare/cfssl)**: cfssl is a full-featured +PKI toolkit that can generate CAs and sign certificates. See the +[cfssl documentation](https://github.com/cloudflare/cfssl#readme) for usage. + **Using OpenSSL** manually: see the [openssl-req](https://docs.openssl.org/master/man1/openssl-req/) and [openssl-x509](https://docs.openssl.org/master/man1/openssl-x509/) docs for creating CAs and signing certificates. From 64f52fa9aa642bb89457424413ac679a3ad16473 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sun, 19 Apr 2026 04:39:24 +0000 Subject: [PATCH 16/16] Fix Windows cert store search order: use fixed-order slice, update docs Agent-Logs-Url: https://github.com/ghostunnel/ghostunnel/sessions/5d5a209d-879a-40d0-a2cc-37c78c1f3857 Co-authored-by: csstaub <639883+csstaub@users.noreply.github.com> --- certstore/certstore_windows.go | 19 +++++++++++-------- docs/KEYCHAIN.md | 10 ++++++---- 2 files changed, 17 insertions(+), 12 deletions(-) diff --git a/certstore/certstore_windows.go b/certstore/certstore_windows.go index cd92d0d0cf..d239e4060e 100644 --- a/certstore/certstore_windows.go +++ b/certstore/certstore_windows.go @@ -96,15 +96,18 @@ func openStore(logger *log.Logger) (*winStore, error) { // Additional stores to use to look for certificates in Identities(). // The identity we want to load might be in the "current service" or "local - // machine" stores, so we need to open those to check. - extraStores := map[string]C.DWORD{ - "CURRENT_SERVICE": C.CERT_SYSTEM_STORE_CURRENT_SERVICE, - "LOCAL_MACHINE": C.CERT_SYSTEM_STORE_LOCAL_MACHINE, - } - for friendlyName, storeIdent := range extraStores { - store := C.CertOpenStore(CERT_STORE_PROV_SYSTEM_W, 0, 0, storeIdent, storeName) + // machine" stores, so we need to open those to check. The order here is + // fixed: CURRENT_SERVICE is searched before LOCAL_MACHINE. + for _, extra := range []struct { + name string + ident C.DWORD + }{ + {"CURRENT_SERVICE", C.CERT_SYSTEM_STORE_CURRENT_SERVICE}, + {"LOCAL_MACHINE", C.CERT_SYSTEM_STORE_LOCAL_MACHINE}, + } { + store := C.CertOpenStore(CERT_STORE_PROV_SYSTEM_W, 0, 0, extra.ident, storeName) if store == nil { - logger.Printf("certstore: failed to open key store '%s', skipping", friendlyName) + logger.Printf("certstore: failed to open key store '%s', skipping", extra.name) continue } diff --git a/docs/KEYCHAIN.md b/docs/KEYCHAIN.md index 7f36078902..fddb775bea 100644 --- a/docs/KEYCHAIN.md +++ b/docs/KEYCHAIN.md @@ -118,11 +118,13 @@ Get-ChildItem Cert:\CurrentUser\My | Format-Table Subject, Thumbprint, NotAfter ``` **Which stores does Ghostunnel search?** When `--keychain-identity` is used -on Windows, Ghostunnel searches three stores in order: +on Windows, Ghostunnel searches three stores in this order: 1. **MY** (Current User), the personal certificate store -2. **CURRENT_SERVICE**, the current service account's certificates -3. **LOCAL_MACHINE**, machine-wide certificates (may require elevation) +2. **CURRENT_SERVICE**, the current service account's certificates (if accessible) +3. **LOCAL_MACHINE**, machine-wide certificates (if accessible; may require elevation) + +Stores that fail to open are skipped rather than causing an error. See Microsoft's [certutil reference][ms-certutil], [System Store Locations][ms-store-locations], and the @@ -138,7 +140,7 @@ Certificates from the keychain can be selected using one or both of the following flags: * `--keychain-identity`: match by the certificate's Common Name (CN) or - serial number. Ghostunnel checks both fields and uses the first match. + serial number. Ghostunnel collects all certificates where either field matches. * `--keychain-issuer`: match by the issuer's Common Name (CN). When both flags are specified, Ghostunnel selects certificates where both