🚀 Feature Description
Implement headless, asynchronous Audit Logging across the platform to track all user-initiated actions and administrative events. Instead of building an internal log viewer, this feature focuses purely on generating structured audit events and exporting them directly to external SIEMs (like Elasticsearch, Splunk, or Datadog). All configuration for this integration will be handled strictly via backend environment variables (.env), with no UI overhead.
💡 Motivation
Currently, when managing Kubernetes clusters through the platform, it is difficult to maintain granular traceability of user actions. I'm always frustrated when investigating an incident, as there is no clear way to map an action back to the specific authenticated platform user. Native Kubernetes audit logs often only show the platform's service account, obfuscating the actual human identity.
Furthermore, enterprise security teams do not want to log into yet another dashboard to view audit logs. They need these logs ingested into their centralized SIEM for correlation, long-term retention, and alerting.
🎯 Proposed Solution
Introduce a centralized Audit Log interceptor at the FastAPI layer that captures structured events for every significant request.
Each log entry should contain:
-
Timestamp: When the action occurred.
-
Actor: The email/ID of the authenticated user.
-
Action: The operation performed (e.g., READ, UPDATE, EXEC, DELETE).
-
Target/Resource: The specific resource affected (e.g., Secret/db-credentials, Pod/api-server-xyz).
-
Status: Success or Failure (with HTTP status codes).
-
Source IP: The IP address of the user making the request.
Critical Requirement: The logging mechanism must be entirely non-blocking to the primary request lifecycle. If the configured SIEM endpoint is offline, unreachable, or timing out, the platform must fail-open and continue operating normally without disrupting the user experience or blocking K8s operations.
🔄 Alternatives Considered
-
Storing logs in the platform's internal database with a UI Console: This was considered but rejected. It bloats the primary database, introduces performance bottlenecks, and forces security teams to use a fragmented workflow instead of their dedicated SIEM.
-
Relying solely on Kubernetes API Server Audit Logs: Falls short because the platform acts as a proxy, hiding the actual user logged into the platform's IAM/SSO.
📋 Additional Context
Providing native, frictionless integration with external SIEMs will significantly boost the platform's appeal to enterprise users and compliance teams (SOC 2, ISO 27001). The "fail-open", headless design ensures that adding security visibility doesn't compromise platform reliability or require complex frontend changes.
🎨 UI/UX Considerations
None. This feature is entirely headless.
-
What screens/pages are affected? No UI changes are required.
-
Configuration will be managed entirely by infrastructure administrators via backend .env variables (e.g., AUDIT_SIEM_URL, AUDIT_SIEM_TOKEN, AUDIT_LOG_ENABLED).
🔧 Technical Considerations
If you have technical insights, please describe:
-
API changes needed? Implementation of a FastAPI middleware (e.g., using BaseHTTPMiddleware or global dependencies) to intercept incoming requests, extract the user identity from the auth token, and capture the execution context and response status.
-
Configuration: Use standard environment variable loading (e.g., via pydantic-settings if the project uses it) to read the SIEM destination parameters.
-
Database schema changes? None. Logs are not stored locally in the database.
-
Performance implications? To guarantee non-blocking behavior, the middleware should construct the audit payload and hand it off to FastAPI's native BackgroundTasks (or a dedicated asyncio.Queue worker). This ensures the HTTP POST to the external SIEM happens asynchronously in the background, allowing the platform to return the HTTP response to the client immediately. Proper timeout handling and exception catching must be in place during the SIEM POST request to ensure the background task silently drops the log if the SIEM is unreachable, preventing thread locking or memory leaks.
🚀 Feature Description
Implement headless, asynchronous Audit Logging across the platform to track all user-initiated actions and administrative events. Instead of building an internal log viewer, this feature focuses purely on generating structured audit events and exporting them directly to external SIEMs (like Elasticsearch, Splunk, or Datadog). All configuration for this integration will be handled strictly via backend environment variables (.env), with no UI overhead.
💡 Motivation
Currently, when managing Kubernetes clusters through the platform, it is difficult to maintain granular traceability of user actions. I'm always frustrated when investigating an incident, as there is no clear way to map an action back to the specific authenticated platform user. Native Kubernetes audit logs often only show the platform's service account, obfuscating the actual human identity.
Furthermore, enterprise security teams do not want to log into yet another dashboard to view audit logs. They need these logs ingested into their centralized SIEM for correlation, long-term retention, and alerting.
🎯 Proposed Solution
Introduce a centralized Audit Log interceptor at the FastAPI layer that captures structured events for every significant request.
Each log entry should contain:
Timestamp: When the action occurred.
Actor: The email/ID of the authenticated user.
Action: The operation performed (e.g., READ, UPDATE, EXEC, DELETE).
Target/Resource: The specific resource affected (e.g., Secret/db-credentials, Pod/api-server-xyz).
Status: Success or Failure (with HTTP status codes).
Source IP: The IP address of the user making the request.
Critical Requirement: The logging mechanism must be entirely non-blocking to the primary request lifecycle. If the configured SIEM endpoint is offline, unreachable, or timing out, the platform must fail-open and continue operating normally without disrupting the user experience or blocking K8s operations.
🔄 Alternatives Considered
Storing logs in the platform's internal database with a UI Console: This was considered but rejected. It bloats the primary database, introduces performance bottlenecks, and forces security teams to use a fragmented workflow instead of their dedicated SIEM.
Relying solely on Kubernetes API Server Audit Logs: Falls short because the platform acts as a proxy, hiding the actual user logged into the platform's IAM/SSO.
📋 Additional Context
Providing native, frictionless integration with external SIEMs will significantly boost the platform's appeal to enterprise users and compliance teams (SOC 2, ISO 27001). The "fail-open", headless design ensures that adding security visibility doesn't compromise platform reliability or require complex frontend changes.
🎨 UI/UX Considerations
None. This feature is entirely headless.
What screens/pages are affected? No UI changes are required.
Configuration will be managed entirely by infrastructure administrators via backend .env variables (e.g., AUDIT_SIEM_URL, AUDIT_SIEM_TOKEN, AUDIT_LOG_ENABLED).
🔧 Technical Considerations
If you have technical insights, please describe:
API changes needed? Implementation of a FastAPI middleware (e.g., using BaseHTTPMiddleware or global dependencies) to intercept incoming requests, extract the user identity from the auth token, and capture the execution context and response status.
Configuration: Use standard environment variable loading (e.g., via pydantic-settings if the project uses it) to read the SIEM destination parameters.
Database schema changes? None. Logs are not stored locally in the database.
Performance implications? To guarantee non-blocking behavior, the middleware should construct the audit payload and hand it off to FastAPI's native BackgroundTasks (or a dedicated asyncio.Queue worker). This ensures the HTTP POST to the external SIEM happens asynchronously in the background, allowing the platform to return the HTTP response to the client immediately. Proper timeout handling and exception catching must be in place during the SIEM POST request to ensure the background task silently drops the log if the SIEM is unreachable, preventing thread locking or memory leaks.