|
| 1 | +# Release runbook: keeping the server and SDKs in sync |
| 2 | + |
| 3 | +Authorizer ships as one server repo plus five satellite repos that must stay |
| 4 | +in lockstep with it: three hand-maintained client SDKs (`authorizer-go`, |
| 5 | +`authorizer-py`, `authorizer-js`) and two generated-only proto packages |
| 6 | +(`authorizer-proto-go`, `authorizer-proto-python`) that the Go/Python SDKs |
| 7 | +depend on. Nearly every cross-repo bug found in this project's history has |
| 8 | +been the same shape: **the server's actual wire contract changed, and some |
| 9 | +downstream artifact — a vendored stub, a hand-written type, a doc page — |
| 10 | +didn't**. This runbook exists to make that class of bug structurally harder, |
| 11 | +not just something a reviewer has to remember to check for. |
| 12 | + |
| 13 | +## The dependency graph |
| 14 | + |
| 15 | +``` |
| 16 | +proto/*.proto (source of truth) |
| 17 | + │ |
| 18 | + ├─▶ gen/go, gen/openapi (server-side; make proto-gen / proto-check) |
| 19 | + │ |
| 20 | + ├─▶ buf.build/authorizerdev/authorizer (BSR module, pushed on every merge to main) |
| 21 | + │ │ |
| 22 | + │ ├─▶ authorizer-proto-go (own repo, regenerates from BSR on its own schedule) |
| 23 | + │ │ └─▶ authorizer-go depends on it as a real go.mod dependency |
| 24 | + │ │ |
| 25 | + │ └─▶ authorizer-proto-python (own repo, regenerates from BSR on its own schedule) |
| 26 | + │ └─▶ authorizer-py depends on it as a real PyPI dependency |
| 27 | + │ |
| 28 | + └─▶ internal/graph/schema.graphqls (GraphQL — independently maintained, NOT |
| 29 | + generated from proto; can drift from it silently, see "Known drift |
| 30 | + sources" below) |
| 31 | + └─▶ authorizer-js hand-types its GraphQL/REST surface directly |
| 32 | + (cannot use gen/ts's message types — see "Why authorizer-js |
| 33 | + stays hand-typed" below) |
| 34 | +``` |
| 35 | + |
| 36 | +Two facts fall out of this that drive the whole runbook: |
| 37 | + |
| 38 | +1. **`authorizer-go`/`authorizer-py` are no longer built by copying generated |
| 39 | + code.** They depend on `authorizer-proto-go`/`authorizer-proto` as real, |
| 40 | + versioned dependencies — the same shape as `prometheus/client_golang` |
| 41 | + depending on `prometheus/client_model`. There is no manual "regenerate, |
| 42 | + then copy into 3 repos" step left for those two. |
| 43 | +2. **`authorizer-js`'s GraphQL/REST surface and the GraphQL schema itself are |
| 44 | + both hand-maintained and can drift independently of proto.** This is the |
| 45 | + one place in the whole graph without a generated source of truth, so it |
| 46 | + needs the most manual vigilance. |
| 47 | + |
| 48 | +## Release order |
| 49 | + |
| 50 | +Always release in this order — each step's version needs to exist before the |
| 51 | +next step can depend on it correctly: |
| 52 | + |
| 53 | +1. **`authorizer` (server)** — merge to `main`, cut a release (`gh release |
| 54 | + create <tag> --prerelease` for an RC). This pushes the updated schema to |
| 55 | + BSR automatically (`.github/workflows/buf.yml`'s `push to main -> publish |
| 56 | + (buf push)` job — verify it actually ran and succeeded, don't assume). |
| 57 | +2. **Wait for the published Docker image to actually exist** before running |
| 58 | + any SDK CI that pulls it. `release.yaml` triggers on `release: created`, |
| 59 | + not on merge — the image build+push takes real time (observed: ~10-12 |
| 60 | + minutes end to end). Check `docker manifest inspect |
| 61 | + quay.io/authorizer/authorizer:<tag>` succeeds before re-running SDK CI |
| 62 | + against it — CI that runs too early gets a 404/timeout, not a useful |
| 63 | + signal that something's actually broken. |
| 64 | +3. **`authorizer-proto-go` / `authorizer-proto-python`** — if the proto |
| 65 | + change affects generated messages, trigger `regenerate.yml` |
| 66 | + (`workflow_dispatch`, don't wait for the weekly cron if you need it now), |
| 67 | + review the auto-opened PR, merge, tag a new version, and for |
| 68 | + `authorizer-proto-python` confirm the PyPI publish actually succeeded |
| 69 | + (`release.yml` is tag-triggered — the workflow that runs is the one |
| 70 | + **at that tag's commit**, not whatever's on `main` right now; if you |
| 71 | + retag after fixing the release workflow itself, you must move the tag to |
| 72 | + the new commit or the old workflow content still runs). |
| 73 | +4. **`authorizer-go` / `authorizer-py`** — bump their dependency on the |
| 74 | + proto package(s), update any code touching removed/changed fields, run |
| 75 | + the FULL live-integration suite against a locally built server image |
| 76 | + (not just unit tests — see "Verification bar" below), then release. |
| 77 | +5. **`authorizer-js`** — only if the GraphQL/REST surface actually changed |
| 78 | + (see "Known drift sources"); it never touches the proto packages. |
| 79 | +6. **Examples repo, docs repo** — update pinned SDK versions and any |
| 80 | + documented method signatures last, once the SDKs they reference are |
| 81 | + actually published and installable. |
| 82 | + |
| 83 | +Skipping the order (e.g. releasing an SDK before its proto dependency is |
| 84 | +published) produces exactly the failure this session hit: CI that resolves |
| 85 | +a version that doesn't exist yet, or worse, silently resolves an *older* |
| 86 | +cached version and passes for the wrong reason. |
| 87 | + |
| 88 | +## Pre-release checklist |
| 89 | + |
| 90 | +Run all of these before cutting **any** release, server or SDK: |
| 91 | + |
| 92 | +- [ ] **`make proto-check`** (server-side gen/go, gen/openapi) is clean. |
| 93 | +- [ ] **`make proto-check-clients`** (gen/ts) is clean. Note this does |
| 94 | + **not** cover `authorizer-proto-go`/`authorizer-proto-python` — those |
| 95 | + regenerate from BSR independently on their own repos, not from this |
| 96 | + repo's `gen/` at all anymore. |
| 97 | +- [ ] **GraphQL schema vs. proto schema**: if you touched a field/message |
| 98 | + that exists conceptually in both (pagination, request envelopes, |
| 99 | + anything with a GraphQL input type mirroring a proto message), grep |
| 100 | + the other one and confirm they still agree. There is **no automated |
| 101 | + check for this** — it's the one drift class this whole graph doesn't |
| 102 | + structurally prevent. When in doubt, diff the GraphQL input type |
| 103 | + against the proto message field-by-field by hand. |
| 104 | +- [ ] **Full test suite green** — `go build ./...`, `go vet ./...`, `make |
| 105 | + test` (SQLite), `make lint`. For storage-layer changes, at least one |
| 106 | + non-SQL backend too. |
| 107 | +- [ ] **`make smoke`** before a server release specifically (build tag |
| 108 | + `smoke`, boots the real binary, exercises GraphQL/REST/gRPC/MCP). |
| 109 | +- [ ] **Docs**: if a public method's signature, a request/response shape, or |
| 110 | + a config flag changed, grep `docs/sdks/*/functions.md` and |
| 111 | + `docs/sdks/*/admin.md` in the `authorizerdev/docs` repo for the old |
| 112 | + shape. These are hand-written and have gone stale before (46 of 81 Go |
| 113 | + admin methods were once undocumented — caught by an explicit audit, |
| 114 | + not any automated process). |
| 115 | + |
| 116 | +## Verification bar for SDK releases |
| 117 | + |
| 118 | +"The unit tests pass" is not sufficient — the bugs this project has actually |
| 119 | +shipped were wire-shape mismatches that only a real round trip catches. |
| 120 | +Before tagging an SDK release: |
| 121 | + |
| 122 | +1. Build a local server image from the exact commit you're releasing |
| 123 | + against: `docker build -t authorizer:localmain .` from the server repo. |
| 124 | +2. Run it with the flags the SDK's own test suite expects (check |
| 125 | + `test/authorizer_test.go` / `tests/integration/test_live.py` for the |
| 126 | + exact `client-id`/`admin-secret`/ports/`--disable-mfa` defaults — they're |
| 127 | + documented at the top of those files specifically so this doesn't need |
| 128 | + re-discovering every time). |
| 129 | +3. Run the **full** live-integration suite (not `-m "not live"`, not just |
| 130 | + unit tests) across every transport the SDK supports (graphql, rest, |
| 131 | + grpc) — a bug can be transport-specific (the `_users` admin query's stale |
| 132 | + `PaginatedRequest` type broke GraphQL only; REST/gRPC use proto messages |
| 133 | + directly and were unaffected, which is exactly the kind of thing a |
| 134 | + single-transport check would miss). |
| 135 | +4. For a **published-package** change (not just a git branch), do an actual |
| 136 | + clean install from the registry (`go get module@version` in a throwaway |
| 137 | + module, `pip install package==version` in a fresh venv) and confirm it |
| 138 | + resolves and imports — `go.sum`/PyPI propagation lag and packaging |
| 139 | + mistakes (a missing `/v2` module suffix, a broken sdist) don't show up |
| 140 | + from testing a local checkout. |
| 141 | + |
| 142 | +## Known drift sources (from this project's actual incident history) |
| 143 | + |
| 144 | +Keep this section updated — every entry here is a real bug that shipped, |
| 145 | +not a hypothetical: |
| 146 | + |
| 147 | +- **GraphQL `PaginatedRequest` double-wrapper vs. proto's flat |
| 148 | + `PaginationRequest`.** The two schemas were never generated from the same |
| 149 | + source for this type; they silently diverged. Fixed by standardizing on |
| 150 | + the proto shape everywhere and removing the wrapper — but nothing |
| 151 | + prevents a *new* divergence like this from being introduced again. Grep |
| 152 | + both schemas by hand for anything that looks like a shared concept. |
| 153 | +- **A field removed from the server (`is_multi_factor_auth_enabled` on |
| 154 | + `SignupRequest`, a security fix) kept getting sent by SDKs that hadn't |
| 155 | + been updated**, in some cases for weeks. The removal was correct and |
| 156 | + reviewed; the *propagation* to dependents wasn't tracked anywhere. |
| 157 | +- **`gen/go-client`/`gen/python` (before they were replaced by dedicated |
| 158 | + proto packages) went stale for the same reason** — the regen step was a |
| 159 | + manual "clone the server repo, run buf generate, copy the output in" |
| 160 | + dance that nobody was reminded to do. This is why those directories don't |
| 161 | + exist anymore; the dedicated proto-package repos regenerate themselves |
| 162 | + from BSR instead of depending on a human remembering. |
| 163 | +- **`authorizer-go`'s `go.mod` never got the `/v2` module-path suffix** Go |
| 164 | + requires once a module is tagged v2+. Every v2.x release before this was |
| 165 | + found was **completely unresolvable** via `go get` — not a subtle bug, a |
| 166 | + release that silently could never be consumed, for an unknown amount of |
| 167 | + time before anyone tried a clean install and noticed. This is exactly why |
| 168 | + step 4 of the verification bar above (a real clean install, not a local |
| 169 | + checkout test) exists. |
| 170 | +- **A CI-config secret dependency wasn't set until the first real release |
| 171 | + attempt.** A repo's `release.yml` referencing `secrets.PYPI_API_TOKEN` |
| 172 | + looks complete in code review; it doesn't fail until someone actually |
| 173 | + tries to cut a release and the token doesn't exist. Prefer PyPI's Trusted |
| 174 | + Publishing (OIDC via `pypa/gh-action-pypi-publish`) for new packages — |
| 175 | + there's no secret to forget to set in the first place. |
| 176 | + |
| 177 | +## Why `authorizer-js` stays hand-typed |
| 178 | + |
| 179 | +Documented here because the temptation to "just use the generated `gen/ts` |
| 180 | +types" comes up naturally once Go/Python are on generated packages, and the |
| 181 | +answer is a real architectural mismatch, not an oversight: |
| 182 | + |
| 183 | +- `gen/ts` (protobuf-es) generates **camelCase** message shapes |
| 184 | + (`phoneNumber`, `confirmPassword`) — the standard JS/TS convention for |
| 185 | + generated protobuf code. |
| 186 | +- This server's REST transport is **deliberately configured to snake_case** |
| 187 | + (`internal/gateway/mount.go`, `UseProtoNames: true`) specifically to keep |
| 188 | + REST payloads aligned with the GraphQL surface, which is also snake_case. |
| 189 | +- `authorizer-js` only speaks GraphQL and REST (browsers can't speak raw |
| 190 | + gRPC, so it never needed the generated message types for that reason |
| 191 | + either) — both of its real transports are snake_case, so generated |
| 192 | + camelCase types would require a rename/mapping layer to actually match |
| 193 | + the wire format, at which point the "just use the generated types" |
| 194 | + shortcut isn't actually a shortcut. |
| 195 | + |
| 196 | +If this server's REST config ever changes to emit camelCase by default, |
| 197 | +this tradeoff should be re-evaluated — but as long as REST intentionally |
| 198 | +mirrors GraphQL's snake_case, generated TS types would be actively wrong |
| 199 | +for both of `authorizer-js`'s transports, not just unnecessary. |
| 200 | + |
| 201 | +## SDK-level e2e coverage |
| 202 | + |
| 203 | +`e2e-playground/` (this repo's live-playground Playwright suite) exercises |
| 204 | +the full feature surface — OIDC, SAML, SCIM, WebAuthn, TOTP/SMS-OTP/WebOTP, |
| 205 | +10 social OAuth providers, MFA enforcement, OTP lockout — but does so via |
| 206 | +raw GraphQL/REST calls and browser automation, never through the actual |
| 207 | +published SDKs. `e2e-playground/sdk-tests/{go,python}/` exercises the same |
| 208 | +live stack through the real `authorizer-go`/`authorizer-py` packages |
| 209 | +specifically to catch the wire-shape-mismatch class of bug described above |
| 210 | +before it reaches a real consumer. When adding a new feature to the server, |
| 211 | +consider whether it needs coverage in both places — the Playwright suite |
| 212 | +proves the feature works; the SDK-driven suite proves the SDKs correctly |
| 213 | +expose it. |
0 commit comments