SaaS Connector - Mixpanel (prototype)

[ddimaria]

Relevant issue(s)

https://www.notion.so/Add-SaaS-Connections-25891edacf2b802e895aefe0a12e3809

Description

This body of work started off an a PoC of Meltano. Performance wasn't very good, so I pivoted to writing this code in Rust, which increased performance by over an order of magnitude. While this is now production-ready, it's intended to be used only by Quadratic as an initial review. Changes need to be made to use this more broadly among Quadratic users.

• Put behind feature flag
• Meltano code for a Mixpanel data pipeline
• Rust Mixpanel client for pulling down event data and converting to daily parquet files
• Once parquet files are created, upload to an object store (s3 or file-system)
• Client code to create a Mixpanel connection and trigger a sync on creation/edit
• API code to create the Mixpanel connection type, and to support the connection service
• Connection service code to:
  • Datafusion connector to read in a directory of parquet files from an object store and query against it
  • sync endpoint to trigger a data sync (currently set at 180 days, but may be adjusted as that takes 8 mins)
  • Background process to pull connections from API hourly and sync

Testing considerations

Get api_secret and project_id from Mixpanel.

QA Wolf tests

Tests need to be added

Dependent docs, links, etc.

Documentation needs to be added. Datafusion supports ANSI SQL, it has some unique features/functions: https://datafusion.apache.org/user-guide/sql/index.html

Demo

[qa-wolf]

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

[github-actions]

Preview - Build, Deploy & Tests-E2E

✅ Build images
✅ Deploy images - Oct 01, 2025 at 08:08 PM UTC - Preview
❌ Tests-E2E - Report

[codecov]

Codecov Report

❌ Patch coverage is 61.37612% with 1336 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.77%. Comparing base (2160fa8) to head (976b0e8).
⚠️ Report is 14 commits behind head on qa.

Files with missing lines	Patch %	Lines
...ratic-rust-shared/src/sql/datafusion_connection.rs	7.42%	486 Missing ⚠️
...uadratic-rust-shared/src/synced/mixpanel/events.rs	28.12%	207 Missing ⚠️
quadratic-connection/src/synced_connection.rs	65.44%	94 Missing ⚠️
...uadratic-rust-shared/src/synced/mixpanel/client.rs	59.28%	57 Missing ⚠️
quadratic-connection/src/sql/datafusion.rs	65.49%	49 Missing ⚠️
...adratic-rust-shared/src/synced/mixpanel/cohorts.rs	0.00%	48 Missing ⚠️
...uadratic-rust-shared/src/synced/mixpanel/funnel.rs	0.00%	48 Missing ⚠️
...uadratic-rust-shared/src/synced/mixpanel/engage.rs	0.00%	43 Missing ⚠️
...adratic-rust-shared/src/synced/mixpanel/revenue.rs	0.00%	42 Missing ⚠️
quadratic-rust-shared/src/synced/mod.rs	86.07%	39 Missing ⚠️
... and 22 more

Additional details and impacted files

@@            Coverage Diff             @@
##               qa    #3408      +/-   ##
==========================================
- Coverage   89.76%   88.77%   -0.99%     
==========================================
  Files         437      453      +16     
  Lines       90097    93422    +3325     
==========================================
+ Hits        80873    82935    +2062     
- Misses       9224    10487    +1263     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.