feat: Add Kedro+Ibis POC with source-faker integration (do not merge)

[aaronsteers]

feat: Add Kedro+Ibis POC with source-faker integration

Summary

This PR adds a standalone proof-of-concept demonstrating Kedro+Ibis integration patterns for scalable data pipelines. The POC is located in kedro-ibis-poc/ directory and implements a three-stage pipeline:

1. Extract: Uses PyAirbyte's source-faker connector to generate test data (users, products, purchases)
2. Staging: Transforms raw tables into cleaned staging views with column selection/renaming
3. Fact: Creates aggregated fact tables (user purchase summaries, product sales summaries)

Key features:

• Deferred Execution: Ibis pushes computation to DuckDB backend
• Universal Interface: Same Ibis code works across 15+ backends
• Official Integration: Uses kedro-datasets[ibis-duckdb] maintained by Kedro team
• Declarative Configuration: OmegaConf variable interpolation for backend reuse

The implementation follows patterns from the Kedro+Ibis blog article and deepyaman/jaffle-shop reference implementation.

Review & Testing Checklist for Human

Critical (3 items) - This POC has not been tested end-to-end:

• Run the pipeline end-to-end: cd kedro-ibis-poc && uv sync && kedro run - verify it completes without errors
• Validate data outputs: Query DuckDB tables to ensure data extraction and transformations work correctly:

  duckdb data/kedro_ibis_poc.duckdb -c "SELECT COUNT(*) FROM raw_users; SELECT * FROM stg_users LIMIT 5; SELECT * FROM fct_user_purchases LIMIT 10;"

• Verify dependency resolution: Ensure uv sync installs all required packages without conflicts (kedro~=0.19.0, airbyte~=0.24.2, ibis-framework[duckdb]~=9.0)

Notes

• POC is standalone and not integrated with existing morph functionality per @aaronsteers request
• Documentation kept minimal with external links per feedback
• Development details moved to CONTRIBUTING.md to separate concerns
• All lint checks passed, but type-check shows pre-existing issues in morph codebase (not related to POC)

Link to Devin run: https://app.devin.ai/sessions/9c3ec40c87694e3ba931dcc52837b7c3
Requested by: @aaronsteers

[devin-ai-integration]

Original prompt from AJ Steers

Received message in Slack channel #ask-devin-ai:

@Devin - Create a POC mockup for a POC on this foundation. Put your work into a new project folder in the morph (airbyte-integration-catalog) repo, but don't bias your work towards the existing content there. Let's treat this as net-new. <https://kedro.org/blog/building-scalable-data-pipelines-with-kedro-and-ibis>
Thread URL: https://airbytehq-team.slack.com/archives/C08BHPUMEPJ/p1759257854719199?thread_ts=1759257854.719199

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch devin/1759258209-kedro-ibis-poc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}