prom-namespace-auth-proxy

A Prometheus-API-compatible reverse proxy that enforces namespace-scoped access to the Thanos tenancy endpoint. It sits between Perses (or any Prometheus-compatible dashboard) and Thanos port 9092, ensuring users can only query metrics for namespaces they are authorized to access.

Overview

The Thanos tenancy endpoint requires callers to pass explicit namespace query-string parameters. This proxy eliminates that requirement — clients issue standard Prometheus API queries with PromQL namespace matchers, and the proxy resolves and injects the appropriate namespace parameters automatically while enforcing authorization based on the caller's OpenShift identity:

Receives a Prometheus API request with the user's Authorization: Bearer <token>.
Parses the PromQL query (or match[] selectors) to extract namespace matchers.
Calls the OpenShift Projects API (GET /apis/project.openshift.io/v1/projects) with the user's own token to determine which namespaces they can access (single API call).
Computes the intersection of queried namespaces and authorized namespaces.
Rewrites the query via prom-label-proxy (defense-in-depth) and forwards the scoped request to Thanos with the user's token.
Rejects unauthenticated, unscoped, or unauthorized requests before they reach Thanos.

Architecture

Perses (user session)
    │
    │  Authorization: Bearer <user-token>
    │  GET /api/v1/query?query=up{namespace="my-ns"}
    ▼
prom-namespace-auth-proxy  (port 8443 HTTPS)
    │
    ├─► OpenShift Projects API  →  authorized namespaces list
    │   (user's own token, single request, 30s TTL cache)
    │
    ├─  Parse PromQL namespace matchers
    ├─  Compute intersection(query namespaces, authorized namespaces)
    ├─  Enforce namespace scope (prom-label-proxy defense-in-depth)
    ├─  Inject namespace= params
    │
    └─► Thanos querier (port 9092 HTTPS)
            │  Authorization: Bearer <user-token>  ← user's token forwarded unchanged

Request lifecycle: token extraction → path routing → PromQL parsing → namespace resolution → intersection → enforcement → forward

Supported API endpoints:

Query: /api/v1/query, /api/v1/query_range, /api/v1/query_exemplars (GET and POST)
Series/labels: /api/v1/series, /api/v1/labels, /api/v1/label/<name>/values (GET only)
All other paths return HTTP 404.

Two-port design:

:8443 — HTTPS proxy endpoint (TLS via OpenShift Service CA Operator)
:8080 — HTTP health/readiness probes at /livez and /readyz (no TLS, no auth required)

Rejection rules:

Missing Authorization header → HTTP 401
No positive namespace label matcher → HTTP 400
Empty intersection (user not authorized for any queried namespace) → HTTP 403

Authorization Examples

The examples below assume a user Alice who has view access to exactly two namespaces: team-frontend and team-backend. She has no access to team-payments or any other namespace.

Queries that succeed (HTTP 200)

Single authorized namespace:

up{namespace="team-frontend"}

Alice has access to team-frontend — the query is forwarded to Thanos.

Another authorized namespace:

container_memory_usage_bytes{namespace="team-backend"}

Alice has access to team-backend — allowed.

Union of both authorized namespaces (regex):

up{namespace=~"team-frontend|team-backend"}

Alice has access to both — all namespaces in the regex are authorized, so the query is forwarded.

Authorized namespace with additional label filters:

http_requests_total{namespace="team-frontend", handler="/api/v1/users"}

Extra label matchers are fine — the proxy only inspects the namespace label.

Queries that are rejected (HTTP 400)

No namespace matcher at all:

up

→ 400 Bad Request — every query must include at least one positive namespace label matcher.

Namespace label present but not a positive matcher:

up{namespace!="team-frontend"}

→ 400 Bad Request — negation matchers (!=, !~) do not count as a positive scope.

Queries that are rejected (HTTP 403)

Single unauthorized namespace:

up{namespace="team-payments"}

→ 403 Forbidden — Alice does not have access to team-payments.

Regex union where one namespace is unauthorized:

up{namespace=~"team-frontend|team-payments"}

→ 403 Forbidden — team-payments is not authorized. The proxy does not silently drop the unauthorized namespace; it rejects the entire query so the user is aware of the scope mismatch.

Regex union where all namespaces are unauthorized:

up{namespace=~"team-payments|kube-system"}

→ 403 Forbidden — Alice has access to neither namespace.

Both authorized namespaces plus one unauthorized:

up{namespace=~"team-frontend|team-backend|team-payments"}

→ 403 Forbidden — even though two of the three are authorized, team-payments is not. Every namespace in the query must be authorized.

Queries that are rejected (HTTP 401)

Missing Authorization header:

curl https://proxy-host/api/v1/query?query=up{namespace="team-frontend"}

→ 401 Unauthorized — no bearer token provided.

Summary

Query	HTTP	Reason
`up{namespace="team-frontend"}`	200	Authorized namespace
`up{namespace="team-backend"}`	200	Authorized namespace
`up{namespace=~"team-frontend\|team-backend"}`	200	Both namespaces authorized
`up`	400	No namespace matcher
`up{namespace!="team-frontend"}`	400	Negation is not a positive matcher
`up{namespace="team-payments"}`	403	Unauthorized namespace
`up{namespace=~"team-frontend\|team-payments"}`	403	One namespace unauthorized
`up{namespace=~"team-frontend\|team-backend\|team-payments"}`	403	One namespace unauthorized
(no Authorization header)	401	Missing token

Design principle — explicit matchers: For exact (namespace="X") and literal alternation (namespace=~"X|Y") matchers, the proxy enforces an all-or-nothing policy. If any namespace in the query is not in the user's authorized set, the entire request is rejected. This prevents accidental data leaks from partial query rewrites and makes authorization failures visible to the caller.

Complex regex patterns (namespace=~"team-.*") are handled differently: the proxy resolves the pattern against the user's authorized namespace list and forwards only the matching authorized namespaces. If no authorized namespace matches the pattern, the request is rejected with HTTP 403. This allows wildcard patterns to work without requiring the user to enumerate every namespace explicitly.

Configuration Reference

All configuration is via CLI flags passed as container args in the Deployment:

Flag	Default	Description
`-upstream`	(required)	Thanos tenancy endpoint, e.g. `https://thanos-querier.openshift-monitoring.svc.cluster.local:9092`
`-secure-listen-address`	`:8443`	HTTPS proxy listen address
`-health-listen-address`	`:8080`	HTTP health probe listen address
`-tls-cert-file`	`/etc/serving-certs/tls.crt`	Path to serving TLS certificate (generated by Service CA Operator)
`-tls-private-key-file`	`/etc/serving-certs/tls.key`	Path to serving TLS private key
`-upstream-ca-file`	`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem`	CA bundle for upstream TLS verification (injected by Service CA Operator)
`-namespace-cache-ttl`	`30s`	TTL for in-memory namespace auth cache
`-api-server-url`	`https://kubernetes.default.svc`	Kubernetes/OpenShift API server URL for Projects API calls
`-kube-ca-file`	`/etc/kube-ca/ca.crt`	Path to the Kubernetes API server CA bundle (from the `kube-root-ca.crt` ConfigMap)
`-metrics-listen-address`	(disabled)	Address for internal `/metrics` endpoint, e.g. `:9090`. Empty string disables it.
`-log-level`	`info`	Structured log level: `debug`, `info`, `warn`, `error`

Building

# Build binary
make build

# Run tests
make test

# Run linter
make lint

# Build container image
make image IMAGE=quay.io/your-org/prom-namespace-auth-proxy:latest

# Push container image
make push IMAGE=quay.io/your-org/prom-namespace-auth-proxy:latest

# Build and push in one step
make image-push IMAGE=quay.io/your-org/prom-namespace-auth-proxy:latest

# Run code generation and verify generated files are committed (CI gate)
make generate
make verify

# Remove build artifacts
make clean

Deployment

Production (base overlay)

oc apply -k deploy/base/ -n <namespace>

The manifests are namespace-agnostic — supply the target namespace at apply time. The deployment expects:

An OpenShift cluster with the Service CA Operator running (standard in OpenShift 4.x).
The prom-namespace-auth-proxy-tls Secret will be auto-created by the Service CA Operator based on the annotation on the Service.

Before deploying, update the image reference in deploy/base/deployment.yaml to your registry.

Live Cluster Testing

See docs/quickstart.md for a step-by-step guide to deploying and testing the proxy on a live cluster.

Troubleshooting

HTTP 400 — Missing namespace matcher

The PromQL query does not contain a positive namespace label matcher. Ensure your Perses datasource is configured with a variable or static namespace filter such as {namespace="my-ns"}.

Check logs:

oc logs -l app=prom-namespace-auth-proxy -n <namespace> | grep "AuthorizationDecision" | grep "forbidden\|bad_data"

HTTP 403 — Empty namespace intersection

The user's token does not grant access to any of the namespaces referenced in the query. Verify the user has view access to the target namespace:

oc auth can-i get pods -n <namespace> --as=<username>

Pod fails to start — TLS errors

The proxy reads TLS certs from /etc/serving-certs/tls.crt and /etc/serving-certs/tls.key. These are auto-generated by the OpenShift Service CA Operator when the Service is annotated. If the pod restarts immediately, check:

oc describe secret prom-namespace-auth-proxy-tls -n <namespace>
oc describe pod -l app=prom-namespace-auth-proxy -n <namespace>

CA rotation

Go's x509.CertPool does not support live certificate refresh. After a CA rotation event, the proxy pod must be restarted to pick up the updated CA bundle:

oc rollout restart deployment/prom-namespace-auth-proxy -n <namespace>

Metrics

When -metrics-listen-address is set (e.g. :9090), the proxy exposes a /metrics endpoint with the following application metrics in addition to the standard Go runtime and process collectors:

Metric	Type	Labels	Description
`proxy_http_requests_total`	Counter	`handler`, `code`, `method`	Total HTTP requests handled by the proxy
`proxy_http_request_duration_seconds`	Histogram	`handler`, `code`, `method`	Request duration in seconds

Security Notes

Queries without namespace matchers are rejected — this prevents users from accidentally (or intentionally) querying cluster-wide metrics.
The user's own bearer token is forwarded to both Thanos and the OpenShift Projects API — no service account token is used for authorization decisions.
No ClusterRole or ClusterRoleBinding is needed — the proxy has zero cluster permissions. All authorization uses the end-user's own token.
In-memory cache TTL — namespace authorization results are cached for at most 30 seconds (configurable via -namespace-cache-ttl). Tokens are stored only as SHA-256 hashes.
TLS 1.2 minimum — enforced for both the proxy listener and upstream connections.

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
cmd/proxy		cmd/proxy
deploy		deploy
docs		docs
hack		hack
internal		internal
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
go.mod		go.mod
go.sum		go.sum

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

prom-namespace-auth-proxy

Overview

Architecture

Authorization Examples

Queries that succeed (HTTP 200)

Queries that are rejected (HTTP 400)

Queries that are rejected (HTTP 403)

Queries that are rejected (HTTP 401)

Summary

Configuration Reference

Building

Deployment

Production (base overlay)

Live Cluster Testing

Troubleshooting

HTTP 400 — Missing namespace matcher

HTTP 403 — Empty namespace intersection

Pod fails to start — TLS errors

CA rotation

Metrics

Security Notes

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

prom-namespace-auth-proxy

Overview

Architecture

Authorization Examples

Queries that succeed (HTTP 200)

Queries that are rejected (HTTP 400)

Queries that are rejected (HTTP 403)

Queries that are rejected (HTTP 401)

Summary

Configuration Reference

Building

Deployment

Production (base overlay)

Live Cluster Testing

Troubleshooting

HTTP 400 — Missing namespace matcher

HTTP 403 — Empty namespace intersection

Pod fails to start — TLS errors

CA rotation

Metrics

Security Notes

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages