Skip to content

feat: save audit logs to postgres (M2-10911) - #2076

Merged
sricharan-varanasi merged 12 commits into
developfrom
feat/audit-logs-to-postgres
Jun 24, 2026
Merged

feat: save audit logs to postgres (M2-10911)#2076
sricharan-varanasi merged 12 commits into
developfrom
feat/audit-logs-to-postgres

Conversation

@sricharan-varanasi

@sricharan-varanasi sricharan-varanasi commented Jun 18, 2026

Copy link
Copy Markdown
Contributor
  • Tests for the changes have been added
  • Related documentation has been added / updated
  • For new features, QA automation engineers have been tagged
  • OSS packages added to Curious open source credit page

📝 Description

🔗 Jira Ticket M2-10911

Moves audit log storage from OpenSearch to our Postgres database.

Changes include:

  • New audit_logs table (+ migration) using a hybrid layout: a few indexed columns (event_id, event_timestamp, event_action, event_outcome, user_id, applet_ids) plus a JSONB payload holding the full event document.
  • Worker task now writes events to Postgres via AuditLogCRUD instead of OpenSearch. Insert is idempotent (ON CONFLICT (event_id) DO NOTHING) so retries don't duplicate rows.
  • Audit export endpoint now reads from Postgres (same request/response, filters, and access control as before).
  • Removed the live OpenSearch wiring (docker-compose services, startup hook). The OpenSearch client/config/mapping modules are left in place but unused.
  • No data migration - there are no audit logs in prod.

🪤 Peer Testing

Requires services up: docker compose up -d postgres postgres-arb redis rabbitmq and uv run alembic upgrade head.

  • Trigger an audited action (e.g. view answers / export) as an owner.

    Expected outcome: a row appears in audit_logs with the correct event_action, user_id, applet_ids and full payload; event_outcome = success.

  • Hit a data-access endpoint with a non-existent applet id, e.g. GET /answers/applet/<random-uuid>/data.

    Expected outcome: 404, and a failure row in audit_logs with error.type = AppletNotFoundError.

  • Call GET /audit/applets/{applet_id}/events as an owner/manager.

    Expected outcome: returns that applet's events, same shape as before, filtered by date range / paginated.

@sricharan-varanasi sricharan-varanasi added the Do not merge Pull request cannot be merged for some reason label Jun 18, 2026
@sricharan-varanasi sricharan-varanasi changed the title Feat/audit logs to postgres feat: save audit logs to postgres (M2-10911) Jun 18, 2026
@sricharan-varanasi sricharan-varanasi removed the Do not merge Pull request cannot be merged for some reason label Jun 18, 2026

@divbzero divbzero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sricharan-varanasi This looks very good overall. I just had one main suggestion of changing from .any(applet_id) to .contains(applet_id). Please take a look to see if that make sense to you too.

Separately, I also want to try this branch of code locally before we merge the database migration. Thank you!

Comment thread src/apps/audit/db/schemas.py Outdated
Comment thread src/apps/audit/crud.py Outdated
Comment thread src/apps/audit/db/schemas.py

@divbzero divbzero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sricharan-varanasi the changes look good! I’ve spun it up locally and confirmed that audit events are logged to the PostgreSQL database and can be exported successfully. Ready to merge after removing ix_audit_logs_event_action from upgrade/downgrade database migration scripts.

@sricharan-varanasi
sricharan-varanasi merged commit ab40ccc into develop Jun 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants