fix(openshift): apply route IP allowlist to production only - #226
fix(openshift): apply route IP allowlist to production only#226alex-struk wants to merge 5 commits into
Conversation
The base Route allowlist used 142.16.0.0/11, which a /11 mask expands to 142.0.0.0–142.31.255.255. That silently excluded BC Gov clients in 142.32.0.0/16–142.36.0.0/16 (e.g. a VPN client on 142.32.84.136 got HTTP 403 at the router despite being on a legitimate BC Gov network). Replace it with the documented BC Gov public 142.x allocation (ARIN org PBC-51-Z): 142.22.0.0/16–142.36.0.0/16. This covers VPN egress and the Silver/Gold/Gold-DR NAT pools (all within 142.34.0.0/16), so the previously enumerated 142.34.* NAT IPs are now redundant and dropped. Move the allowlist to a single source of truth in base/ so every environment (dev, test, prod, instance-template) inherits the correct value, and remove the now-redundant per-overlay route-allowlist-patch.yml files from prod and test. Docs: update KUSTOMIZE_INSTANCE_TEMPLATE.md to describe the corrected range. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks the previous approach: instead of restricting every environment via a
base allowlist, restrictions now apply to production only, matching the intent
that test/dev/PR instances stay open (reachable without VPN).
- base/{backend-services,frontend}/route.yml: remove the ip allowlist entirely
so base (inherited by all instance-template deployments) is open. Backend keeps
the /metrics deny-list.
- components/prod-resources: add route-allowlist-{backend,frontend}.yml applying
haproxy.router.openshift.io/ip_whitelist = 142.22.0.0/16-142.36.0.0/16 (BC Gov
ARIN PBC-51-Z). This component is auto-included by scripts/lib/generate-overlay.sh
only for *-prod namespaces, so it hooks into the real deploy path (Deploy Instance
workflow -> instance-template) rather than the dead overlays/prod.
- Use the ip_whitelist key (not ip_allowlist): ip_whitelist is verified to enforce
on Silver's OpenShift 4.18 router; ip_allowlist support is unconfirmed and using
an ignored key on prod would silently drop the restriction.
Verified via kustomize build: base open; fd34fb-prod renders ip_whitelist on both
prefixed routes; fd34fb-test renders no allowlist; prod memory/PVC patches intact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alex-struk
changed the title
…ction Adopt the ip_allowlist annotation name (renamed from ip_whitelist in PR #218 / AI-1341 for inclusive naming) for the production-only route restriction added by components/prod-resources, and align the base-route comments and docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
overlays/{prod,test,dev} are not part of any deploy path — the Deploy Instance
workflow renders from overlays/instance-template. They were stale/misleading
(their route patches never reached any cluster). Remove them and repoint the
README's OpenShift deploy snippet at the real mechanism (Deploy Instance /
instance-template).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Do we really want to open the IP ranges that wide?
The previous OpenShift egress IPs came from this resource: https://digital.gov.bc.ca/technology/cloud/private/internal-resources/topology/
Can we confirm that the wider cidr notation still only encompasses BC Gov addresses?
|
Good question @dbarkowsky — I dug into this and the short version is the new notation is actually narrower and more precise than the old one, and it's provably all BC Gov. Where the list comes from: the 15 blocks Why it's not wider than before: the old value was The three cluster NAT/egress pools from the topology page (Silver One honest caveat: this does admit BC Gov's whole 142.x allocation, not just our three NAT pools + VPN, so it's broader than the strict minimum. Everything in it is BC Gov, but it's whole-of-government rather than app-specific. I went this route because it's stable against NAT-pool changes and matches the openshift-wiki's documented approach; happy to tighten it back to VPN-range + the specific NAT /32s if you'd prefer the minimal surface. |
Restore the fixed-overlay files removed earlier in this PR and comment them out instead of deleting. Per review discussion with Kaegan, keep them as a legible reference for a possible future hybrid deploy model (fixed overlays for the stable test/prod envs alongside the instance-template script used for ephemeral branch instances). Nothing in the pipeline builds these files; they are inert. Decision tracked in AI-1687. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fair, I wasn't thinking of this /11 spread, just the exact IPs below. 
Nice to confirm that we own that whole 142 range. |
3 participants
Intent
Source-IP restrictions should apply to production only. Ephemeral PR,
test, anddevinstances should stay open (reachable without VPN). The original design tried to do this withoverlays/prod, but that overlay is never applied — so prod got the wrong (base) value and the restriction leaked to every environment.Root cause (why the old overlay never worked)
Prod and test are deployed by the Deploy Instance workflow (
.github/workflows/deploy-instance.yml), which renders fromoverlays/instance-templateviascripts/lib/generate-overlay.shand runskustomize build | oc apply.instance-templateinheritsbase; nothing appliesoverlays/{prod,test,dev}. So every deploy used the base allowlist (142.16.0.0/11), which also happened to exclude142.32.0.0/16–142.36.0.0/16(a coworker on142.32.84.136got HTTP 403).Change
Hook the restriction into the real deploy path using the existing prod-only component mechanism (the same one that raises prod memory limits / PVC sizes):
base/*/route.yml— remove the IP allowlist entirely. Base is now open, sotest/dev/PR instances need no VPN. Backend keeps the/metricsdeny-list.components/prod-resources/route-allowlist-{backend,frontend}.yml(new) — applyhaproxy.router.openshift.io/ip_whitelist = 142.22.0.0/16–142.36.0.0/16(BC Gov ARINPBC-51-Z; covers VPN egress + Silver/Gold/Gold-DR NAT pools in142.34.0.0/16).generate-overlay.shauto-includes this component only for*-prodnamespaces.overlays/{prod,test}/route-allowlist-patch.ymland their kustomization refs.Annotation key:
ip_allowlistUses
haproxy.router.openshift.io/ip_allowlist, the inclusive-naming rename adopted in PR #218 / AI-1341. The base routes carry no allowlist at all, so the enforcement annotation exists only incomponents/prod-resources(prod-only).Verification (
kustomize build)basefd34fb-prod(via generate-overlay)ip_whiteliston bothbcgov-di-{backend-services,frontend}✅fd34fb-test(via generate-overlay)fd34fb-prodmemory/PVC patchesmemory: 2Gi,storage: 5Gistill applied ✅Operational notes
oc annotate(ip_whitelist, equivalent range) and is currently restricted correctly. Once this reachesmain, a prod deploy sets the same key/value from the component.developdoesn't prune it, clear it once with:oc annotate route <name> haproxy.router.openshift.io/ip_whitelist- haproxy.router.openshift.io/ip_allowlist- -n fd34fb-test🤖 Generated with Claude Code