Skip to content

SECURITY.md

Latest commit

 

History

History
754 lines (505 loc) · 49.7 KB

SECURITY.md

File metadata and controls

754 lines (505 loc) · 49.7 KB
Raw

Security

SDEP is an API-first application designed for machine-to-machine (M2M) integrations.

The following security considerations apply.

Table of contents

This document applies to the application scope only (CI/CD-aspects are outside the scope of this repo).

Identification

Confidential machine clients must be identified upfront. This is assumed to be handled through established operational processes and is therefore outside the scope of this document.

Authentication and authorization

For machine authentication, SDEP supports OAuth 2.0 with the Client Credentials Grant (grant_type=client_credentials).

  • OAuth 2.0 with the Client Credentials Grant is the standard framework for trusted machine-to-machine (M2M) communication.
    • It uses one underlying Client Credentials Flow, designed for machine-to-machine communication without an end-user context.
    • It supports two types of Client Authentication, to authenticate against a same "token endpoint".
  • On successful authentication, clients acquire a Bearer Access Token.
    • The access token is used to invoke the actual (authorized) endpoints.
    • The access token is short-lived, and should be refreshed frequently (programmatically).

Client authentication types are:

  • Client ID & Secret
    • The client sends a static symmetric shared secret to an Authorization Server (client_secret_post / client_secret_basic).
  • Client-Signed JWT
    • The client generates a short-lived JSON Web Token (JWT) and signs it using its own private key (private_key_jwt).
    • An Authorization Server validates this request using the client's registered public key, offering a higher level of security since no shared secrets are transmitted over the wire.

Client-signed JWT is regarded as the most secure:

  • The OAuth 2.0 Security Best Current Practice (RFC 9700, section 2.5) recommends asymmetric client authentication (private-key JWT per RFC 7523) over shared secrets.
  • Client-signed JWTs avoid distributing long-lived shared secrets to API clients.
  • The client proves possession of its private key by signing a short-lived JWT for each token request.
    • SDEP and Keycloak only need the corresponding public key to verify it.
  • Key rotation becomes explicit.
    • Private key material stays outside SDEP configuration.
  • Interactive testing in the Swagger UI is supported.
    • But, it requires a programmatic call to the /token endpoint first, in order to acquire a bearer token.

The Authorization Server for the reference implementation in this repository is Keycloak .

Specification

Implementation

SDEP-NL supports both authentication methods, via the same /token endpoint:

  • Both methods are supported in the SDEP-NL Test and Pre-Production environments (TST, PRE)
  • Only client-signed JWT is supported in the SDEP-NL Production environment (PRD)

Authentication

Authentication (obtaining a bearer token) proves who the client is, and can take place at the same /token endpoint, via:

  • Client-signed JWT (client_signed_jwt)
  • Client secret (client_id and client_secret)

The Swagger UI Authorize button follows the application configuration CLIENT_SECRET_AUTH_ENABLED.

  • When CLIENT_SECRET_AUTH_ENABLED is true:
    • The Swagger UI uses the OAuth 2.0 Client Credentials flow with client-secret authentication (client id and secret).
  • When CLIENT_SECRET_AUTH_ENABLED is false:
    • The Swagger UI accepts a Bearer token that the operator obtained out of band with a client-signed JWT, because Swagger cannot sign a client JWT itself.

A Note on 2FA

Software, scripts, and servers cannot approve push notifications or type one-time passcodes. M2M authentication therefore uses non-interactive, cryptographic credentials, instead of human-style two-factor authentication (2FA).

Authorization

Authorization determines what the client is allowed to do.

  • Based on the client's identity and pre-configured permissions, the server issues an access token containing specific scopes (roles).
  • The client then presents this as a Bearer token during API calls to access protected resources.

Supported scopes (roles) are:

Scope (role) Purpose
sdep_ca Competent Authority access
sdep_str STR Platform access
sdep_sta Statistics authority access
sdep_lsa Listing screening authority access
sdep_lma Listing monitoring authority access
sdep_ama Activity monitoring authority access
sdep_read Read operations
sdep_write Write operations

JWT Claims

JWT Claims used by the application:

Claim Maps to
client_id Private client ID of the Platform or Competent Authority, linked to its public platformId or competentAuthorityId but separate from it
client_name Platform or Competent Authority display name
realm_access.roles Role-based authorization

Smaller platforms

Smaller platforms can delegate SDEP API-invocation to a third party.

In that case, the platform arranges data submission with that third party. The third party becomes registered in SDEP.

Audit log

The audit log, implemented in audit.py, logs "who did what, where, when, from where, and with what result".

Scope:

  • (Yet) for technical management only (troubleshooting security, performance, ...)
  • Enough context to reconstruct important actions
  • No sensitive (personal) data

Implementation approach as follows.

Middleware-Based Audit Capture

A Starlette BaseHTTPMiddleware intercepts each request/response cycle and creates an audit record for every relevant interaction.

Non-Blocking Audit Writes

Audit records are persisted asynchronously using asyncio.create_task(), so audit logging does not block or delay the application response path.

Primary Output: audit_log Database Table

Audit records are written to the audit_log table in the application database.

  • Append-only: records are inserted only; existing audit records are not updated or deleted.
  • Error-resilient: audit write failures are logged, but they do not interrupt or fail the original request.
  • Application-managed retention: database retention is handled by the application and may be shorter than external log retention.

Secondary Output: Structured JSON to Stdout

Each audit record is also emitted as a single-line structured JSON object to stdout.

Complementary Access Paths

Together, the database table and stdout output provide complementary access paths:

  • Database audit log: convenient for application-level querying, investigation, and short-to-medium-term retention.
  • Stdout: useful for real-time operational visibility, for example when viewing container logs.
  • Stdout → external log management: stdout can be collected by the runtime environment and forwarded to external log tooling, such as an Elastic/Kibana-based stack, for centralized search and longer retention.

Deployment and Log Shipping Are Out of Scope

  • This document defines how the application produces audit records and where it emits them.
  • (Kubernetes) deployment details and external log management configuration are outside the scope of this repo.

For more details, see section Audit log (details).

OWASP

Measures taken based on:

ID Subject Explanation Measure
A01:2025 Broken Access Control Unauthorized access to data or functions Endpoints secured by OAuth 2.0 with JWT
A02:2025 Security misconfiguration Bad configurations, insecure defaults, environment mistakes Externalized config (config.py)
A03:2025 Software supply chain failures Vulnerabilities in dependencies and external libraries Container Image Scans (part of CI/CD)
A04:2025 Cryptographic failures Failures in encryption, key management TLS terminated at the gateway; RS256 for JWT in the IAM (e.g. Keycloak), both CI/CD
A05:2025 Injection SQL, XSS, command, path injection See XSS, CSP, SQL, Path (Injection) below
A06:2025 Insecure design Security not considered at design/architecture phase Security by design (SDEP documentation)
A07:2025 Authentication Failures Weak login, session management or credential handling Endpoints secured by OAuth 2.0 with JWT
A08:2025 Software or data integrity failures Failures in ensuring data or code integrity Pydantic validation (application), source code control (CI/CD)
A09:2025 Logging and alerting failures Insufficient logging, monitoring or alerting Audit log
A10:2025 Mishandling of exceptional conditions Improper handling of errors, edge cases, unexpected input Exception handling (exception_handlers.py)
API1:2023 Broken Object Level Authorization Access to objects by manipulating IDs Object-level scoping via JWT client_id in the service/CRUD layer
API2:2023 Broken Authentication Flawed identity verification OAuth 2.0 with JWT (RS256), JWKS key rotation, token proxy timeout
API3:2023 Broken Object Property Level Authorization Exposing or modifying properties without authorization Pydantic schemas (explicit fields, no mass assignment, frozen=True on auth models)
API4:2023 Unrestricted Resource Consumption Missing limits allowing DoS or cost exploitation Upload, batch and pagination limits; rate limiting at deployment level
API5:2023 Broken Function Level Authorization Access to admin or restricted functions Role-based endpoint protection via RequireRoles
API6:2023 Unrestricted Access to Sensitive Business Flows Automated abuse of business operations at scale Not applicable: M2M only, client registration is process-controlled
API7:2023 Server-Side Request Forgery Fetching remote resources from user-supplied URLs Not applicable: no user-supplied URL fetching, the IAM URL is server-configured
API8:2023 Security Misconfiguration Inappropriate hardening across the stack Security headers, TLS, non-root container, no stack traces in responses
API9:2023 Improper Inventory Management No visibility of API assets and versions Versioned API mounts (/api/{domain}/v1); auto-generated OpenAPI docs
API10:2023 Unsafe Consumption of APIs External APIs integrated without security controls IAM (e.g. Keycloak) integration with TLS, timeout (10 s), JWKS caching

XSS, CSP, SQL, path (injection)

XSS, CSP

A Cross-Site Scripting attack (XSS) has three phases:

  1. Input - the attacker injects malicious content (e.g. <script>)
  2. Storage / Reflection - the application returns that content to a user
  3. Output / Execution - the browser executes the script

SDEP mitigates these phases as follows.

Input is validated and rejected, to avoid injection.

  • Server-side validation by FastAPI and Pydantic
  • Rejects incorrect data types before the handler runs

Output is escaped, to avoid returning executable content:

  • FastAPI automatically JSON-serializes all responses
  • Special characters (<, >, ", ') are escaped in JSON output

Route-specific Content Security Policy headers (CSP) are added, to avoid foreign script execution in browser:

  • Implemented in the headers.py middleware (tells the browser which sources are allowed to execute, and blocks everything else)
  • Applies a route-specific CSP (strict on all API and root paths, relaxed only on Swagger UI docs pages)
  • Motivation: Swagger UI requires 'unsafe-inline' for its inline scripts and style="" attributes (nonces/hashes cannot cover the inline style attributes here)
  • https://thecodebuzz.com/content-security-policy-csp-swagger-ui-openapi/

SQL

A SQL injection attack manipulates database queries by inserting malicious SQL fragments into user input (e.g. ' OR 1=1 --), potentially reading, modifying, or deleting data.

SQL injection is mitigated by design through the technology stack:

  • All database access uses SQLAlchemy ORM with its query builder API (select(), insert(), update(), delete())
  • All user-supplied values are passed as bound parameters - SQLAlchemy never interpolates values into SQL strings
  • There are no raw SQL strings anywhere in the codebase (including audit log retention, which also uses the SQLAlchemy query builder)
  • Pydantic validates and constrains all input before it reaches the database layer (type checks, max lengths, regex patterns)

Path

A path traversal attack manipulates file paths by inserting directory traversal sequences (e.g. ../../etc/passwd) into user input, potentially reading or overwriting files outside the intended directory.

Path traversal is mitigated by design through the application architecture:

  • No filesystem operations on user-supplied input - uploaded files are read into memory and stored as binary blobs (LargeBinary) in the database, not written to disk
  • The uploaded filename is stored as metadata in the database only; it is never used to construct filesystem paths
  • All functional IDs (used in URL path parameters and form fields) are validated against a strict alphanumeric pattern (^[A-Za-z0-9\-]+$ in common.py), which rejects path traversal characters (/, \, ., ..)
  • JWT claims used as identifiers (client_id) are validated against a similarly strict pattern (^[A-Za-z0-9._-]+$), which additionally permits . and _ for Keycloak client naming while still rejecting path separators (/, \)

CSRF

Cross-Site Request Forgery (CSRF) allows an attacker to trick a logged-in user into performing actions they didn't intend to do (via another website).

Cross-Site Request Forgery (CSRF) is not applicable for SDEP:

  • SDEP uses stateless JWT bearer tokens in the Authorization header, not cookies - a browser cannot automatically attach credentials to a forged request, so CSRF is not possible
  • Swagger UI authenticates via the same bearer token mechanism - no cookies are used, so CSRF-tokens and cookie attributes (SameSite=Strict; Secure; HttpOnly) do not apply

See also security.py (OAuth2ClientCredentials).

Swagger UI

The Swagger UI is intentionally served publicly by FastAPI without authentication, because the API itself is open source.

  • As such, exposing the API documentation is considered an accepted and safe design decision rather than a security risk
  • Potential risks related to public Swagger UI exposure and unauthorized access to API documentation are therefore not applicable in this context

Unauthorized usage of API endpoints is mitigated through the OAuth 2.0 Client Credentials flow using JWT bearer tokens.

File upload

File upload is implemented in areas.py (post_area).

File uploads are protected by:

  • Format: only .zip files are accepted (validated by filename extension and ZIP magic bytes PK\x03\x04); non-zip uploads return 422
  • Size: max 1 MiB (MAX_FILE_SIZE = 1_048_576); oversized uploads return 422
  • Malware scanning: uploads are scanned with ClamAV before being accepted; infected files return 400. A scan that could not run (ClamAV unreachable, timeout, ClamAV ERROR) returns 503: the file is still refused, but the client learns to retry later instead of that its file is suspect
  • Filename sanitization at upload time: the uploaded filename is sanitized before it is stored in the database, using the shared filename.py utility (sanitize_upload_filename). Malicious filenames are never persisted. Sanitization:
    • Path separators are stripped (extracts basename from Unix / and Windows \ paths)
    • Control characters (C0 range \x00-\x1f, CR, LF), double quotes, and backslashes are removed
    • Leading/trailing dots and whitespace are stripped; consecutive dots are collapsed
    • If the resulting base name (before .zip) is empty, the upload is rejected with 422
    • Filenames exceeding 64 characters after sanitization are rejected with 422

This works together with the download-time sanitization and RFC 5987 encoding, described in File Download (Content-Disposition). Defense-in-depth: malicious filenames are rejected at upload, re-sanitized at download, and safely encoded in the response header.

File download (Content-Disposition)

Area file downloads construct the Content-Disposition header using the shared filename.py utility. This complements the upload-time sanitization described in File Upload with two download-time measures:

Defense-in-Depth Re-Sanitization (sanitize_download_filename)

As a second line of defense, the download path re-sanitizes the stored filename before constructing the header.

  • Re-sanitization can strip all characters, e.g. a filename consisting entirely of control characters
  • Rather than emitting an empty Content-Disposition filename, the function then returns the literal string "download", so the client always receives a usable filename
  • In practice this cannot occur: upload-time sanitization (see File Upload) already rejects such filenames - the fallback is a defensive guard only

RFC 5987 Encoding (content_disposition_header)

The header is encoded per RFC 5987, which defines how to include non-ASCII characters in HTTP header field parameters. Both filename= and filename*= are emitted:

Content-Disposition: attachment; filename="ascii-safe.zip"; filename*=UTF-8''percent-encoded.zip
  • filename="..." is the ASCII-safe fallback for legacy clients (non-ASCII characters are replaced with _)
  • filename*=UTF-8''... is the RFC 5987 form for modern clients, using UTF-8 encoding with percent-encoded characters (via urllib.parse.quote)
  • When both are present, compliant clients prefer filename*= over filename= (per RFC 6266, section 4.3)

RFC 5987 solves three problems:

  • Non-ASCII filenames: characters like é, ü, or ñ are percent-encoded instead of being silently dropped or causing encoding errors
  • Header injection prevention: percent-encoding neutralizes characters that would otherwise break HTTP header syntax (CR, LF, ", ;)
  • Cross-browser compatibility: the dual filename= / filename*= pattern ensures all clients receive a usable filename, regardless of their RFC 5987 support

Malware scanning

The application uses ClamAV for malware scanning of uploaded files. Configuration is done via environment variables:

Environment variable Default Description
MALWARE_SCAN_ENABLED true Enables or disables upload malware scanning
MALWARE_SCAN_CLAMAV_HOST "" ClamAV daemon host
MALWARE_SCAN_CLAMAV_PORT 3310 ClamAV daemon port
MALWARE_SCAN_CLAMAV_TIMEOUT 10 ClamAV scan timeout in seconds

For local testing, Docker Compose starts ClamAV together with the rest of the stack:

make up

This uses docker-compose.yml through the top-level Makefile, and loads .env plus .env.extra when that optional override file exists.

The malware scan is tested automatically. To test malware detection manually, generate an EICAR test archive:

scripts/generate-eicar-zip.sh

Upload the generated file shown by the script through the CA area upload endpoint. The upload should be rejected with 400 because ClamAV detects the EICAR test signature.

Secrets

To avoid data leaks, secrets are externalized in config.py.

Security headers

To avoid misuse on various layers, HTTP-headers are hardened in main.py and headers.py:

Layer HTTP-header Avoids
API version API-Version: <major.minor.patch> on every response [1] Clients guessing which release serves them
Cache control (sensitive paths) Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0 on every API domain path Cached responses leaking authentication tokens or personal data
Content security Content-Security-Policy: default-src 'self'; script-src 'self'; ... (CSP) Cross-site scripting (XSS), code injection, and data exfiltration
Cross-origin Cross-Origin-Embedder-Policy: require-corp (COEP) [2] Cross-origin resource leaks via embedded content
Cross-origin Cross-Origin-Opener-Policy: same-origin (COOP) Browsing context from cross-origin openers
Cross-origin Cross-Origin-Resource-Policy: same-origin (CORP) Other origins loading SDEP responses
Encryption Strict-Transport-Security: max-age=31536000; includeSubDomains; preload (HSTS) Plain (unencrypted) HTTP-sniffing
Frame protection frame-ancestors 'none' Clickjacking
Frame protection X-Frame-Options: DENY Clickjacking
MIME protection X-Content-Type-Options: nosniff MIME-sniffing
Permissions Permissions-Policy: geolocation=(), microphone=(), camera=(), ... [3] Unauthorized access to device features (geolocation, microphone, ...)
Referrer policy Referrer-Policy: no-referrer Information leakage via Referer

[1] The release version; the contract major is in the path (NLgov REST API Design Rules /core/version-header, see API).

[2] Consider unsafe-none when encountering 504 issues in deployment.

[3] Full value: geolocation=(), microphone=(), camera=(), payment=(), usb=(), magnetometer=(), gyroscope=().

Although CI/CD-related aspects are outside the scope of this repo, test results for SDEP-NL are as follows.

  • SDEP-NL scores A+ on securityheaders.com
  • This validates that response headers provide adequate browser-side protection

CORS policy: no cross-origin access. SDEP grants no origin access to the API. There is no CORS middleware and no Access-Control-Allow-Origin header, so a browser denies every cross-origin request by default, the most restrictive outcome. This is the deliberate policy, not an omission (NLgov REST API Design Rules /core/transport/cors, see API):

  • SDEP is a backend API consumed by server-side clients using machine-to-machine (M2M) OAuth 2.0 tokens
    • Server-to-server calls do not go through a browser
    • So CORS is never triggered
  • Swagger UI is served from the same origin as the API
    • So its requests are same-origin and CORS does not apply
  • Should a browser client ever need access (e.g. a monitoring dashboard on another origin), add Starlette's CORSMiddleware with that one origin allowlisted; never *

Middleware ordering

Starlette processes middleware LIFO (last added = outermost = runs first). In main.py:

  1. SecurityHeadersMiddleware (outermost) - added last, runs first
  2. AuditLogMiddleware (inner) - added first, runs inside security headers

Security headers, DNS, TLS

Although CI/CD-related aspects are outside the scope of this repo, additional test results for SDEP-NL are as follows.

  • SDEP-NL scores 100% on internet.nl
  • This validates that transport-level security is correctly applied (poor basic configuration would increase the attack surface)

Rate limiting (throttling)

Rate limiting helps protect against brute-force attacks and abuse. The risk is highest on unauthenticated endpoints such as /token, where an attacker could attempt credential stuffing at network speed.

Rate limiting is typically applied per client IP address, and is enforced at the deployment or infrastructure layer, e.g.:

  • Kubernetes Gateway API
  • HAProxy load balancer
  • IAM (e.g. Keycloak) authorization server

These deployment-specific concerns are outside the scope of this repository.

Dependency version pinning

Dependencies are declared with flexible lower bounds (>=) in pyproject.toml and locked to exact versions in uv.lock.

Why >= instead of == in pyproject.toml

  • pyproject.toml declares the minimum acceptable version of each dependency (the intent)
  • uv.lock records the exact resolved version of every package, including transitive dependencies (the pin)
  • The Dockerfile installs with uv sync --frozen, which enforces the lock file exactly - no version can drift at build time
  • Using == in pyproject.toml would duplicate what the lock file already does, while making legitimate upgrades harder and not covering transitive dependencies
  • Security remediations follow the same rule: the >= floor is raised to the version that fixes the CVE (for transitive packages via [tool.uv] constraint-dependencies), while uv.lock keeps pinning the exact shipped version

Docker Base Images

  • The Python base image is pinned to a minor version (python:3.14-slim) via ARG PYTHON_IMAGE
  • The uv installer is pinned to a specific release (ghcr.io/astral-sh/uv:0.12.21)
  • PostgreSQL, Keycloak and ClamAV versions are externalized via environment variables in docker-compose.yml

Keeping Dependencies Up to Date

  • Running uv lock --upgrade regenerates the lock file with the latest compatible versions
  • The lock file should be committed and reviewed as part of the normal change process

Non-root containers

The Docker container runs as a non-root user (app), following the principle of least privilege. This limits the impact of a container escape or application compromise.

Container image scans

To minimize exposure to Common Vulnerabilities and Exposures (CVEs), the reference implementation includes container image scanning as part of CI/CD:

  • Continuously monitor and remediate Critical and High severity CVEs.
  • Implement remediation according to a "comply (fix) or explain" policy.

The reference implementation provides two CVE checks:

  • make test-cve builds the image, scans it with Trivy, and compares the findings with the CVE allowlist in docs/CVE_EXPLAINS.md. It fails when:
    • The scan reports a CVE the allowlist does not justify
    • The allowlist still lists a CVE the scan no longer reports
    • A listed package or severity differs from what the scan reports
    • The same CVE is listed twice
  • make test-cve-offline scans nothing. It feeds prepared reports to the comparison script, to confirm it still catches each of those cases, and checks that the allowlist identifiers have a plausible year. It needs no image, so it is the fast check and runs in make all, while the image scan runs in make ci-gate.

The image scan reads a vulnerability database that upstream rebuilds every few hours.

  • Trivy caches that database locally, and keeps it until its recorded next-update time (up to a day later)
  • So a cached local scan and a fresh pipeline scan can report different CVEs and severities for the same image
  • make test-cve therefore drops the cached database before every scan and downloads the current one, matching the pipeline
  • Set TRIVY_SKIP_DB_REFRESH=1 to reuse the cached database when working offline or iterating quickly - the result may then no longer match the pipeline

Note: docs/CVE_EXPLAINS.md is intentionally not committed. Each EU member state implementing an SDEP is responsible for maintaining its own CVE allowlist and remediation process within its CI/CD pipeline.

Running the scanner and validating its report should be regarded as blocking CI/CD checks (make ci-gate).

Each EU member state implementing an SDEP is responsible for monitoring and remediating CVEs within its own CI/CD.

Authentication and authorization (details)

SDEP interacts with IAM (e.g. Keycloak) in two distinct ways: token issuance (active HTTP call) and token validation (local signature verification using cached public keys).

Token Issuance (Proxy to Keycloak)

When an external client needs a JWT, it calls SDEP's /api/auth/v1/token endpoint.

SDEP acts as a proxy:

  1. It takes either the client's client_id + client_secret (from HTTP Basic Auth or form body) or its client_id + client_signed_jwt
  2. It maps it to Keycloak's OAuth private_key_jwt request fields
  3. It forwards the token request to Keycloak's token endpoint at /realms/sdep/protocol/openid-connect/token
  4. It returns the resulting JWT

This is a synchronous request-response - every token request hits Keycloak directly.

See auth.py.

Token Validation (Client-Signed JWT)

On every subsequent API call, the client sends the JWT as a Bearer token. SDEP verifies the token signature locally - without calling Keycloak on every request - using the JSON Web Key Set (JWKS) protocol:

  1. Keycloak signs JWTs with its private RSA key and publishes the corresponding public keys at the JWKS endpoint: /realms/sdep/protocol/openid-connect/certs
  2. PyJWKClient fetches that key set and caches it in-memory
  3. For each incoming request, get_signing_key_from_jwt(token) reads the JWT's kid (key ID) header, finds the matching public key from the cached set, and verifies the RS256 signature
  4. The decoded payload is returned - no HTTP call to Keycloak needed

See security.py.

Audience Validation

A JWT can contain an aud (audience) claim that says which application the token was issued for. When an application checks aud, it rejects tokens that were meant for a different service - even if the signature is valid. This prevents a token issued for Service A from being reused against Service B.

Two different JWTs occur in SDEP, and aud is treated differently in each. Which client authentication method was used does not change this: both client secret and client-signed JWT lead to the same kind of access token.

JWT Sent to aud checked Checked by
Client assertion (client_signed_jwt) /token, to authenticate Yes Keycloak
Access token (Bearer) The API, to authorize the call No -

Client Assertion

With client-signed JWT (private_key_jwt), the client signs a short-lived assertion whose aud is the authorization server's token endpoint. SDEP does not inspect that assertion: it forwards it to Keycloak as client_assertion, and Keycloak validates the signature against the registered public key and the aud against its own token endpoint. An assertion signed for a different token endpoint is rejected with HTTP 401.

This audience binding is required by RFC 7523 and is what stops an intercepted assertion from being replayed against another authorization server. It is also why each environment configures its own CLIENT_SIGNED_JWT_AUDIENCE, matching the public issuer URL of that environment's Keycloak.

Access Token

The bearer token that SDEP itself verifies on each API call is not audience-checked. The reason is practical:

  • Keycloak does not include an aud claim in the client credentials tokens it issues to SDEP clients by default
  • If SDEP started requiring aud, every existing client would be rejected, until the Keycloak configuration is updated to include it

SDEP is the only application in the Keycloak realm, so the aud claim always originates from SDEP itself. Therefore this has no security impact.

The validation guarantees for the access token are therefore:

  • RS256 signature verification using Keycloak JWKS (proves the token was issued by Keycloak and has not been tampered with)
  • Expiry (exp) verification (rejects tokens that are no longer valid)
  • No aud verification (a valid Keycloak token for another service in the same realm would be accepted)

JWKS Key Rotation (5-Minute TTL)

PyJWKClient is configured with cache_jwk_set=True and lifespan=300 (5 minutes). This ensures that when Keycloak rotates or revokes signing keys, SDEP picks up the changes within at most 5 minutes - without requiring a restart. The alternative (@lru_cache) would cache keys indefinitely, meaning rotated or revoked keys would never be refreshed until the process was restarted.

Concern How it's addressed
Performance 99.9% of requests use cached keys - no network call to Keycloak
Key rotation New keys are picked up within 5 minutes
Key revocation Revoked keys stop being trusted within 5 minutes
Thread safety _get_jwks_client() uses double-checked locking to ensure exactly one PyJWKClient instance is created across threads

Request Authentication Flow

Client request with Bearer token
  → OAuth2ClientCredentials extracts the token from the Authorization header
    → verify_bearer_token() calls validate_jwt_token()
      → _get_jwks_client() returns the singleton PyJWKClient
        → client.get_signing_key_from_jwt(token) matches the JWT kid to a cached public key
          → jwt.decode() verifies signature + expiry using that key
            → get_parsed_token() extracts realm_access.roles, client_id, client_name
              → RequireRoles checks roles against endpoint requirements
                → Endpoint handler executes

See auth_dependencies.py.

Audit log (details)

Audit Fields

For each request that matters, capture:

Field Source Description Answers
timestamp Server clock UTC, server default now() When
requestId Generated UUID4 correlation ID -
roles JWT realm_access.roles Verified roles, or null when no token was authenticated (401, or unauthenticated endpoints) Who
resourceType Derived from path Entity type, e.g. area, listing, activity Where
action Derived from method + path Semantic action verb, e.g. create What
httpMethod Request HTTP method (GET, POST, DELETE) What
path Request Request path, e.g. /api/ca/v1/areas Where
httpStatusCode Response HTTP status code Result
statusCode Derived from httpStatusCode OK if httpStatusCode < 400, else NOK Result
durationMs Calculated Request processing time in milliseconds -

Role Extraction - Only From Verified Tokens

The audit middleware reads roles from the JWT payload. The auth dependency (verify_bearer_token) stashes that payload on request.state.jwt_payload, after signature and expiry verification.

  • Tokens that fail verification never reach request.state, so forged tokens cannot pollute the audit trail
  • The middleware does not re-decode the token, avoiding a duplicate signature check per audited request
  • The 401 vs 403 distinction is encoded in the httpStatusCode column
  • The roles column carries the verified role set when one is available, and null otherwise
  • Audience validation remains disabled, as described above: aud is not enforced until Keycloak token configuration supports it
Scenario What happens roles in audit log
Valid JWT, authorized (2xx) Auth dependency verifies the token and stashes the payload on request.state Verified roles from token
Valid JWT, missing required role (403) Token verified by verify_bearer_token; RequireRoles then rejects on insufficient role Verified roles from token
Forged, tampered, or expired JWT (401) Auth dependency rejected the token before the handler ran; no payload on request.state null
No JWT (e.g. /token endpoint) No bearer credentials presented null

Action Mapping

The middleware derives a semantic action and resource type from the HTTP method and request path:

Method Path pattern Resource type Action
POST /api/ca/v*/areas area create
GET /api/ca/v*/areas area list
GET /api/ca/v*/areas/count area count
GET /api/ca/v*/areas/{id} area read
DELETE /api/ca/v*/areas/{id} area delete
GET /api/str/v*/areas area list
GET /api/str/v*/areas/count area count
GET /api/str/v*/areas/{id} area read
POST /api/str/v*/listings/bulk listing create_bulk
POST /api/lsa/v*/listing-screenings/bulk listing screen_bulk
POST /api/str/v*/listing-acknowledgements/bulk listing acknowledge_bulk
GET /api/{str,lsa,ca,lma,sta}/v*/listings listing list
GET /api/{str,lsa,ca,lma,sta}/v*/listings/count listing count
POST /api/str/v*/activities/bulk activity create_bulk
GET /api/ca/v*/activities activity list
GET /api/ca/v*/activities/count activity count
GET /api/sta/v*/activities activity list
GET /api/sta/v*/activities/count activity count
GET /api/ama/v*/activities activity list
GET /api/ama/v*/activities/count activity count
POST /api/auth/v*/token auth token
GET /api/ping system ping

Unmatched paths fall back to action unknown.

Example

| id  | timestamp                     | request_id   | roles                        | resource_type | action | http_method | path             | http_status_code | status_code | duration_ms |
| --- | ----------------------------- | ------------ | ---------------------------- | ------------- | ------ | ----------- | ---------------- | ---------------- | ----------- | ----------- |
| 20  | 2026-03-23 15:03:38.519686+00 | a34e8a0e-... | sdep_write,sdep_ca,sdep_read | system        | ping   | GET         | /api/ping        | 200              | OK          | 1           |
| 21  | 2026-03-23 15:03:39.864974+00 | 7bccb30b-... | sdep_write,sdep_ca,sdep_read | area          | create | POST        | /api/ca/v1/areas | 201              | OK          | 33          |
| 22  | 2026-03-23 15:03:39.947615+00 | f357d78c-... | sdep_write,sdep_ca,sdep_read | area          | create | POST        | /api/ca/v1/areas | 201              | OK          | 27          |
| 23  | 2026-03-23 15:03:40.02963+00  | 02294cf4-... | sdep_write,sdep_ca,sdep_read | area          | create | POST        | /api/ca/v1/areas | 201              | OK          | 18          |

Skip List

The following paths are not audited (high-frequency, low-value):

  • / (root)
  • /favicon.ico (browsers request this automatically; the application does not serve a favicon)
  • /api/docs (landing page)
  • /api/health
  • The /openapi.json and /docs path of every API version (/api/auth/v1, /api/ca/v1 and v2, /api/str/v1 and v2, /api/lsa/v2, /api/lma/v2, /api/ama/v1, /api/sta/v1 and v2), plus /api/openapi.json, /api/ping/openapi.json and /api/ping/docs
  • The list lives in backend/app/security/audit.py (SKIP_PATHS); the API version paths are derived from the domain registry (API_DOMAINS), so a new API version is skipped without an edit there

Retention of the Database

For the database table, expired audit log rows are automatically deleted by a background task that runs every hour.

  • The retention period is configurable via the AUDITLOG_RETENTION environment variable (default: 1 day).
  • Deletion is batched (1,000 rows per batch) to avoid long-running transactions.

The retention logic in audit_retention.py is split into two functions with distinct responsibilities:

  • delete_old_audit_logs does the actual work.
  • audit_log_cleanup_loop is the scheduler that ensures that work runs repeatedly for the lifetime of the application.
Function Responsibility Invocation
delete_old_audit_logs(retention_days) One-shot deletion of rows older than retention_days [1] Each cycle of audit_log_cleanup_loop; also standalone in scripts and tests
audit_log_cleanup_loop(retention_days, interval_seconds) Infinite scheduling loop around delete_old_audit_logs [2] An asyncio.Task in the FastAPI lifespan of main.py [3]

[1] Deletes in batches of 1,000 and returns the total number of deleted rows. A pure async function that runs to completion; it does not loop or sleep.

[2] Runs one deletion, sleeps interval_seconds (default 3,600 s = 1 hour), repeats until cancelled. Exceptions are caught and logged, so one failed cycle does not kill the loop.

[3] Started when the application boots, cancelled (task.cancel()) when it shuts down.

Retention of Stdout

For stdout, retention is assumed to be part of the deployment environment (out of scope of this repo).