|
| 1 | +# Certificate issuance without a DNS credential (`dns-persist-01`) |
| 2 | + |
| 3 | +`dns-01` asks certbot to write a fresh `_acme-challenge` TXT record for every |
| 4 | +order, so whatever runs certbot holds a DNS API token with write access to the |
| 5 | +zone, forever. In dstack that token lives inside the gateway CVM. Attestation |
| 6 | +covers what the CVM is running, but a token is a token: anything that gets hold |
| 7 | +of it can rewrite the zone, including records that have nothing to do with |
| 8 | +certificates. |
| 9 | + |
| 10 | +`dns-persist-01` moves the proof out of the issuance loop. The zone owner |
| 11 | +publishes one record naming the CA and the ACME account allowed to issue: |
| 12 | + |
| 13 | +```dns |
| 14 | +_validation-persist.example.com. IN TXT "letsencrypt.org; accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/1234567890" |
| 15 | +``` |
| 16 | + |
| 17 | +The account key proves who is asking, the record proves the zone owner agreed, |
| 18 | +and neither changes between orders. certbot reads DNS and never writes it, so |
| 19 | +the CVM holds no DNS credential and the zone can be hosted anywhere — no |
| 20 | +Cloudflare account, no API token, no provider integration. |
| 21 | + |
| 22 | +> **Experimental.** `dns-persist-01` is specified in |
| 23 | +> [draft-ietf-acme-dns-persist-01][draft], which is still changing: an open |
| 24 | +> working-group issue may add a client-key-derived value to the record, and |
| 25 | +> Let's Encrypt has said it will not deploy the challenge to production until |
| 26 | +> that is resolved. It is live on Let's Encrypt **staging** and in |
| 27 | +> [Pebble][pebble]. Treat the record format as unstable, and expect to |
| 28 | +> republish when the draft settles. |
| 29 | +
|
| 30 | +[draft]: https://datatracker.ietf.org/doc/html/draft-ietf-acme-dns-persist-01 |
| 31 | +[pebble]: https://github.com/letsencrypt/pebble |
| 32 | + |
| 33 | +## What the record means |
| 34 | + |
| 35 | +| Part | Effect | |
| 36 | +| --- | --- | |
| 37 | +| `letsencrypt.org` | Issuer Domain Name. A CA ignores records naming a different issuer, so one label can hold records for several CAs. | |
| 38 | +| `accounturi=` | The ACME account authorized to issue. Compared byte for byte — no case folding, no URI normalization. | |
| 39 | +| `policy=wildcard` | Extends the record to `*.example.com`. Without it the CA authorizes `example.com` alone and refuses wildcard orders. | |
| 40 | +| `persistUntil=` | Optional UNIX timestamp after which the CA stops accepting the record. | |
| 41 | + |
| 42 | +A wildcard order authorizes from its base name — `*.example.com` is validated |
| 43 | +against `_validation-persist.example.com`, not |
| 44 | +`_validation-persist.*.example.com` — so one record covers a name and its |
| 45 | +wildcard. |
| 46 | + |
| 47 | +Two things about the syntax bite in practice, because a CA rejects the whole |
| 48 | +record rather than ignoring the offending part: **no trailing semicolon**, and |
| 49 | +**no whitespace inside a value**. certbot renders records that satisfy both; |
| 50 | +copy them verbatim rather than retyping. |
| 51 | + |
| 52 | +The scope stops at the names above. Let's Encrypt does not walk up the tree, so |
| 53 | +a record on `example.com` does not authorize `sub.example.com` — give each base |
| 54 | +name its own record. |
| 55 | + |
| 56 | +## Standalone certbot |
| 57 | + |
| 58 | +`certbot` never writes DNS in this mode, so setup is: create the account, read |
| 59 | +the records off it, publish them, then issue. |
| 60 | + |
| 61 | +```toml |
| 62 | +# certbot.toml |
| 63 | +workdir = "/var/lib/certbot" |
| 64 | +acme_url = "https://acme-staging-v02.api.letsencrypt.org/directory" |
| 65 | +challenge = "dns-persist-01" |
| 66 | +issuer_domain_name = "letsencrypt.org" |
| 67 | +# auto_set_caa promises certbot keeps CAA in sync, which it cannot do without |
| 68 | +# write access. Leave it off and publish the CAA records below by hand. |
| 69 | +auto_set_caa = false |
| 70 | +domains = ["example.com", "*.example.com"] |
| 71 | +renew_interval = 3600 |
| 72 | +renew_days_before = 10 |
| 73 | +renew_timeout = 120 |
| 74 | +# Capped at half of renew_timeout, so this one waits 60s, not 300s. The whole |
| 75 | +# order -- the wait, the authorizations, finalize, and fetching the certificate |
| 76 | +# -- has to fit inside renew_timeout, and a wait that outlasts it takes the |
| 77 | +# timeout with it, losing the warning naming the record that was missing. |
| 78 | +# Raising this alone changes nothing; raise renew_timeout with it. |
| 79 | +max_dns_wait = 300 |
| 80 | +``` |
| 81 | + |
| 82 | +`cf_api_token` is unused and can be left out; certbot warns if one is set. |
| 83 | + |
| 84 | +```console |
| 85 | +$ certbot init -c certbot.toml |
| 86 | +INFO certbot::bot: creating new ACME account |
| 87 | +INFO certbot::bot: created new ACME account: https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890 |
| 88 | + |
| 89 | +$ certbot dns-records -c certbot.toml |
| 90 | +_validation-persist.example.com. IN TXT "letsencrypt.org; accounturi=https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890; policy=wildcard" |
| 91 | +example.com. IN CAA 0 issue "letsencrypt.org;validationmethods=dns-persist-01;accounturi=https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890" |
| 92 | +example.com. IN CAA 0 issuewild "letsencrypt.org;validationmethods=dns-persist-01;accounturi=https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890" |
| 93 | +``` |
| 94 | + |
| 95 | +Publish all three, wait for them to propagate, then issue: |
| 96 | + |
| 97 | +```console |
| 98 | +$ certbot renew --once -c certbot.toml |
| 99 | +INFO certbot::acme_client: requesting new certificates for example.com, *.example.com |
| 100 | +INFO certbot::bot: created new certificate |
| 101 | +``` |
| 102 | + |
| 103 | +Renewals need nothing further. The record stays, and `certbot renew` reuses it |
| 104 | +for every order. |
| 105 | + |
| 106 | +The CAA records are optional but recommended — they stop any other account, at |
| 107 | +Let's Encrypt or elsewhere, from being issued for your name. Note the |
| 108 | +`validationmethods=dns-persist-01` in them: a CAA record left pinned to |
| 109 | +`dns-01` refuses every `dns-persist-01` order, so switching methods means |
| 110 | +updating CAA and the validation record together. |
| 111 | + |
| 112 | +### When issuance fails |
| 113 | + |
| 114 | +certbot checks its own resolver before starting an order, and says exactly what |
| 115 | +it expected to find: |
| 116 | + |
| 117 | +``` |
| 118 | +WARN certbot::acme_client: no TXT record at _validation-persist.example.com matches the expected value: letsencrypt.org; accounturi=https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890; policy=wildcard |
| 119 | +Error: order is invalid: API error: Checking DNS-PERSIST-01 challenge TXT record with issuer-domain-name "letsencrypt.org": accounturi mismatch: expected "https://acme-staging-v02.api.letsencrypt.org/acme/acct/1234567890", got "https://acme-staging-v02.api.letsencrypt.org/acme/acct/9876543210" (urn:ietf:params:acme:error:unauthorized) |
| 120 | +``` |
| 121 | + |
| 122 | +The check is advisory and never blocks an order: certbot's resolver is not the |
| 123 | +CA's, and its expectation can be stricter than what the CA would accept. A |
| 124 | +warning with a successful issuance underneath it is a resolver difference, not |
| 125 | +a problem. A record that genuinely does not match costs the whole DNS budget |
| 126 | +before the order is sent, because the check waits it out first -- that is |
| 127 | +`max_dns_wait`, or half of `renew_timeout` when that is smaller, which for the |
| 128 | +configuration above is 60s. certbot logs the budget it settled on at `debug`. |
| 129 | + |
| 130 | +If the CA rejects the order, compare the published record against |
| 131 | +`certbot dns-records` character by character. The usual causes are a stale |
| 132 | +`accounturi` after the account was recreated, a missing `policy=wildcard` on a |
| 133 | +wildcard order, and an issuer domain name that does not match the CA. |
| 134 | + |
| 135 | +## dstack-gateway |
| 136 | + |
| 137 | +A ZT domain picks its method with the `challenge` field, which defaults to |
| 138 | +`dns-01` — existing domains are unaffected: |
| 139 | + |
| 140 | +```json |
| 141 | +{ "domain": "app.example.com", "port": 443, "challenge": "dns-persist-01" } |
| 142 | +``` |
| 143 | + |
| 144 | +Such a domain needs no `dns_cred_id`, and the gateway CVM never receives a DNS |
| 145 | +credential for it. `GetZtDomain` and `ListZtDomains` return the records to |
| 146 | +publish in `required_dns_records`, and the gateway logs them whenever it cannot |
| 147 | +write DNS itself: |
| 148 | + |
| 149 | +``` |
| 150 | +WARN cert[app.example.com]: publish this record by hand: _validation-persist.app.example.com. IN TXT "letsencrypt.org; accounturi=...; policy=wildcard" |
| 151 | +``` |
| 152 | + |
| 153 | +The gateway issues for `*.{domain}` only, so each domain needs one validation |
| 154 | +record with `policy=wildcard`. |
| 155 | + |
| 156 | +### Setup order |
| 157 | + |
| 158 | +The records name the ACME account, so the account comes first: |
| 159 | + |
| 160 | +1. `SetCertbotConfig` — `acme_url`, and `issuer_domain_name` if the CA is not |
| 161 | + Let's Encrypt. |
| 162 | +2. `RotateAcmeCredentials` — registers the account and returns its `account_uri`. |
| 163 | + It needs no ZT domain: registration proves nothing about a domain. Every call |
| 164 | + registers a *new* account, so call it once here; running it again later is a |
| 165 | + rotation, with the consequences described below. |
| 166 | +3. `AddZtDomain` with `challenge: "dns-persist-01"`. No `dns_cred_id`: the |
| 167 | + domain needs none, and the gateway CVM never receives one for it. |
| 168 | +4. Read the records — the dashboard shows them when the domain is added and |
| 169 | + again under its **Records** button, and `GetZtDomain`, `ListZtDomains` and |
| 170 | + the gateway log all carry them — then publish the `_validation-persist` TXT |
| 171 | + record, plus the CAA records if you want issuance pinned to this account. |
| 172 | +5. `RenewZtDomainCert`. Renewals from here need nothing further. |
| 173 | + |
| 174 | +Adding another dns-persist-01 domain later starts at step 3: the account already |
| 175 | +exists, so the new domain's records are available as soon as it is. A domain's |
| 176 | +challenge is chosen when it is added — in the dashboard's ZT-Domain form as well |
| 177 | +as over the API — and carried forward by every edit. `UpdateZtDomain` replaces |
| 178 | +the whole record, so `challenge` is optional there and omitting it means "leave |
| 179 | +it as it is": a caller that predates the field — a cached dashboard bundle, a |
| 180 | +script, an older SDK — cannot downgrade a `dns-persist-01` domain to `dns-01` by |
| 181 | +editing something else, which its hand-published CAA would then refuse. Send the |
| 182 | +field explicitly to switch a domain over. |
| 183 | + |
| 184 | +### Naming the CA |
| 185 | + |
| 186 | +For a non-production ACME server, set `issuer_domain_name` in the global certbot |
| 187 | +config — `Admin.SetCertbotConfig`, or the field of that name in the dashboard's |
| 188 | +Certbot Configuration — to whatever that server puts in `issuer-domain-names`, |
| 189 | +`pebble.letsencrypt.org` for Pebble. Empty means `letsencrypt.org`. |
| 190 | + |
| 191 | +It also names the CA in the CAA records certbot writes for `dns-01` domains, so |
| 192 | +one setting covers both challenges rather than pinning CAA to Let's Encrypt |
| 193 | +while orders go somewhere else. `acme_url` is one setting for the whole |
| 194 | +deployment; this follows it. |
| 195 | + |
| 196 | +The value is checked when it is set. It ends up verbatim inside an RFC 8659 |
| 197 | +`issue-value`, and publishing CAA deletes the records it replaces before writing |
| 198 | +the new ones, so a name carrying a space or a `;` would leave the zone holding a |
| 199 | +malformed `issue` property with nothing valid behind it — CAA that forbids every |
| 200 | +issuer. Anything that is not a DNS name is refused by `SetCertbotConfig`, and by |
| 201 | +`certbot` at startup. |
| 202 | + |
| 203 | +Publishing CAA installs a temporary `;` guard, deletes the records it replaces, |
| 204 | +writes the new ones and drops the guard. A run interrupted part-way leaves the |
| 205 | +guard behind, which denies every issuer until a later run finishes the job — so |
| 206 | +each run now sweeps stale guards before installing its own, and a rerun is |
| 207 | +enough to recover. |
| 208 | + |
| 209 | +### How long the check waits |
| 210 | + |
| 211 | +The self-check before each order waits for the record to become visible, but |
| 212 | +never longer than half of `renew_timeout`, the budget for the whole renewal. |
| 213 | +Past that the outer timeout fires first and the renewal ends as `certificate |
| 214 | +request timed out` with no mention of the record it could not find — the |
| 215 | +opposite of what the check is for. Both values defaulted to 300s in the gateway |
| 216 | +(and the CLI wrapped a 300s wait in a 120s budget), so this was the default |
| 217 | +behaviour rather than an edge case. |
| 218 | + |
| 219 | +Two operations behave differently on these domains: |
| 220 | + |
| 221 | +- **`SetCaa`** skips them. There is nothing to reconcile without write access; |
| 222 | + the records are logged instead, and the summary reports how many were left to |
| 223 | + the operator. |
| 224 | +- **`RotateAcmeCredentials`** is not self-service. Rotation moves the cluster |
| 225 | + to a new ACME account, and every `_validation-persist` record still names the |
| 226 | + old one, so orders for those domains fail until the operator republishes. The |
| 227 | + response returns the new records in `required_dns_records` and the gateway |
| 228 | + logs them; publish before the next renewal comes due. |
| 229 | + |
| 230 | + A rotation that registers the account but cannot re-pin every `dns-01` |
| 231 | + domain's CAA still succeeds — the account exists and the cluster is on it — |
| 232 | + and names those domains in `repin_failed_domains`. Finish with `SetCaa` |
| 233 | + rather than another rotation, which would register yet another account |
| 234 | + against a rate-limited quota. The dashboard reports both lists in one dialog |
| 235 | + after a rotation, with a **Run SetCaa** button for the second. |
| 236 | + |
| 237 | +## Related |
| 238 | + |
| 239 | +- [dstack-gateway](dstack-gateway.md) — gateway architecture and TLS termination |
| 240 | +- [deployment.md](deployment.md#4-zero-trust-https-optional) — the `dns-01` setup this replaces |
0 commit comments