Skip to content

Keeper: restructure OCSF log pipeline routing - #24870

Draft
jbfeldman-dd wants to merge 4 commits into
masterfrom
jonah.feldman/keeper-ocsf
Draft

Keeper: restructure OCSF log pipeline routing#24870
jbfeldman-dd wants to merge 4 commits into
masterfrom
jonah.feldman/keeper-ocsf

Conversation

@jbfeldman-dd

@jbfeldman-dd jbfeldman-dd commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds an OCSF log pipeline for Keeper that normalizes Keeper enterprise audit events into OCSF 1.5.0 classes. Keeper's audit_event field drives routing, and every event is guaranteed to land in a schema-valid class via a fallback chain.

The pipeline consists of an OCSF pre transformations stage (product/vendor metadata, event_code, and shared field mappings applied to all events) followed by per-class sub-pipelines, in ascending class_uid order, with two fallbacks last:

  • Detection Finding [2004] — genuine alerts (KeeperAI risk-level and session-lock detections, BreachWatch high-risk records) with is_alert and risk_level_id derived from severity. Detection-rule configuration changes are deliberately excluded.
  • Account Change [3001] — account lifecycle and setting changes.
  • Authentication [3002] — login / logout / MFA events.
  • Entity Management [3004] — device-approval events and role create/update.
  • User Access Management [3005] — privilege grant/revoke events.
  • Group Management [3006] — team membership and team lifecycle events.
  • Web Resources Activity [6001] — resource sharing and data-access events; web_resources[] is populated from the relevant uid fields via an array-processor.
  • Application Lifecycle [6002]ssh_agent_* and pam_gateway_* events.
  • Datastore Activity [6005] — record CRUD, record-content reads, and file_attachment_* events.
  • Scan Activity [6007] — BreachWatch / discovery scan events.
  • API Activity [6003] (fallback) — any otherwise-unmapped event that carries both username and remote_address.
  • Base Event [0] (fallback, last) — everything else, so no event is dropped.

The class mappings preserve the customer-derived field coverage from the original Fleak-generated mappings, extended to complete sister-event families (e.g. all compliance_report_*, ext_share_*, keeper_ai_* variants) so documented events route consistently rather than only the ones present in customer samples.

Test coverage: the tests file exercises all 12 classes plus both fallbacks with PII-free samples (reserved-range IPs and anonymized emails). All 24 samples validate against OCSF 1.5.0 at Required 100%.

Motivation

Keeper audit logs previously had no OCSF normalization, so they could not participate in OCSF-based security analytics. This pipeline maps Keeper's audit events to standard OCSF classes with an explicit fallback chain that keeps every event schema-valid.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

jbfeldman-dd and others added 2 commits July 29, 2026 14:52
Add OCSF pre-transformations pipeline and sub-pipelines for classes
Entity Management [3004], User Access Management [3005], and File
Hosting Activity [6006]. Flip preserveSource to true on the standard
remappers so vendor fields survive for OCSF sourcing, and add OCSF
facets. Regenerate expected test output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-partition the Keeper OCSF pipeline into a smaller, consistent class set
with explicit fallback routing so every event lands in a valid class.

- Route genuine alerts (keeper_ai risk/session-lock, BreachWatch) to
  Detection Finding [2004] with is_alert and risk_level_id; keep detection-rule
  config changes out of 2004.
- Move device-approval and role create/update events to Entity Management [3004];
  restrict Group Management [3006] to team events.
- Unify sharing and data-access events under Web Resources [6001], populating
  web_resources[] via an array-processor; keep record CRUD, record-content reads,
  and file_attachment_* in Datastore Activity [6005].
- Map ssh_agent_* and pam_gateway_* to Application Lifecycle [6002].
- Eliminate the 1006, 2003, 5019, and 6006 sub-pipelines; re-home their events.
- Add two-tier fallback: unmapped events with username and remote_address to
  API Activity [6003], everything else to Base Event [0].
- Add PII-free synthetic samples covering the new families and both fallbacks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jbfeldman-dd jbfeldman-dd added the qa/required QA is required for this PR and will generate a QA card label Aug 14, 2026
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant