Skip to content

Commit f9bc82b

Browse files
committed
source-monday: refactor GraphQL code into modular architecture with performance and memory improvements
• Restructure monolithic graphql.py (1109 lines) into specialized modules: - graphql/activity_logs.py for activity log operations - graphql/boards.py for board management - graphql/items/ package with items.py and item_cache.py - graphql/query_executor.py for centralized query execution with IncrementalJsonProcessor - graphql/constants.py for shared constants • Implement IncrementalJsonProcessor for memory-efficient streaming of large GraphQL responses • Add item cache system for efficient items backfill operations and reduced API calls • Enhance API client with improved error handling, complexity management, and debugging capabilities • Improve code quality, API compliance, and maintainability through modular design • Update models and resources to support modular architecture • Add utility functions for common operations • Update test snapshots to reflect architectural changes This refactoring addresses data discrepancy (missing data) issues and maintainability concerns while significantly improving performance and memory efficiency for data synchronization operations.
1 parent aaa2085 commit f9bc82b

File tree

17 files changed

+4510
-375
lines changed

17 files changed

+4510
-375
lines changed

source-monday/poetry.lock

Lines changed: 2645 additions & 14 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

source-monday/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ authors = ["Justin Smith <justin@estuary.dev>"]
77
[tool.poetry.dependencies]
88
estuary-cdk = { path = "../estuary-cdk", develop = true }
99
pydantic = "^2"
10+
aiostream = "^0.7.0"
1011
python = "^3.11"
1112

1213
[tool.poetry.group.dev.dependencies]

0 commit comments

Comments
 (0)