-
Notifications
You must be signed in to change notification settings - Fork 12
Description
Feature Description
Add the ability to receive events from external systems — CI/CD pipelines, deployment tools, third-party monitoring services — and normalize them into Logtide log entries, making Logtide the single source of truth for all operational events across a team's infrastructure.
Problem/Use Case
Operational context is scattered across multiple tools. A deployment happens in GitHub Actions, an alert fires in an external uptime monitor, a background job fails in a different service — all of these are operationally significant events, but none of them automatically appear in Logtide. When debugging an incident, engineers have to correlate timelines across multiple UIs manually.
If Logtide could receive events from these external systems and store them as structured log entries, all operational signals would be searchable and correlatable in one place.
Proposed Solution
- Generic webhook receiver endpoint:
POST /api/v1/receivers/:projectId/:receiverIdauthenticated with a per-receiver token (not the project's main API key, to limit blast radius if a receiver token is leaked) - Built-in adapters for common sources with automatic normalization:
- GitHub Actions: workflow run events, deployment status changes
- Uptime Robot / Better Stack: downtime and recovery alerts
- Generic JSON: user-defined field mapping from any arbitrary payload to Logtide log fields (
level,service,message,metadata)
- Receiver management UI: create and delete receivers, view recent received events with their normalized output, copy the receiver URL and token
- Received events are passed through the normal ingestion pipeline — PII masking and parsing rules (see issue [Feature] OTLP Metrics Ingestion #4) apply automatically
Alternatives Considered
- Requiring external tools to use the Logtide ingest API directly: this works for tools that support custom webhook payloads, but most tools have fixed webhook formats that can't be changed.
- Polling external APIs: having Logtide poll GitHub/external services for events. Rejected because it requires storing credentials for external services, adds complexity, and polling latency is worse than push-based webhooks.
Implementation Details (Optional)
- Receiver definitions stored in a
receiverstable with: id, project_id, name, adapter_type, token (hashed), field_mapping JSON, created_at - Each adapter is a small transform function:
(rawPayload) => LogtideLogEntry. New adapters can be added without touching the core receiver infrastructure - The receiver endpoint should return quickly (202 Accepted) and process the payload asynchronously via a BullMQ job, consistent with the rest of the ingestion pipeline
- Token authentication for receivers should use a separate prefix (e.g.
lr_) to distinguish receiver tokens from project API keys (lp_) in logs and UIs
Priority
- Critical - Blocking my usage of LogTide
- High - Would significantly improve my workflow
- Medium - Nice to have
- Low - Minor enhancement
Target Users
- DevOps Engineers
- Developers
- Security/SIEM Users
- System Administrators
- All Users
Additional Context
This feature pairs naturally with the health monitoring feature (issue #3). A team might use native Logtide monitors for their own services and webhook receivers to ingest events from external dependencies they don't control — covering both sides of their operational landscape in a single platform.
Contribution
- I would like to work on implementing this feature