Speedup UpdateGroup Jobs

[ethany-nv]

Description

This PR optimizes our UpdateGroup job by:

• minimizing the amount of data the worker is fetching when calling Workflow.fetch_from_db
• replacing pydantic.create_model() with types.SimpleNamespace, which is much more lightweight and supports .attribute access

Here is the benchmark analysis, running a workflow with 50 tasks:

Category	Current Avg	New Avg	Change	Current Median	New Median	Change
UpdateGroup (SCHEDULING)	1.553s	0.864s	-44.4%	1.543s	0.851s	-44.8%
UpdateGroup (INITIALIZING)	2.222s	1.556s	-30.0%	2.191s	1.563s	-28.7%
UpdateGroup (RUNNING)	2.223s	1.705s	-23.3%	2.210s	1.670s	-24.4%
UpdateGroup (COMPLETED)	2.575s	1.779s	-30.9%	2.451s	1.526s	-37.7%

Total UpdateGroup time: Current=428.7s vs New=300.1s (-30.0%)

Issue #411

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

Summary by CodeRabbit

• Chores
  • Optimized internal data handling mechanisms for improved performance and reduced memory overhead.
  • Enhanced workflow data retrieval to streamline background job processing with more efficient resource utilization.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d7f1e831-8d2e-4fc6-935a-2f9d8e636e3d

📥 Commits

Reviewing files that changed from the base of the PR and between 9cc68cd and 6be6c5d.

📒 Files selected for processing (2)

• src/utils/connectors/postgres.py
• src/utils/job/jobs.py

---

📝 Walkthrough

Walkthrough

Two utility modules are updated to optimize data handling. The PostgreSQL connector replaces dynamic Pydantic model creation with SimpleNamespace objects for fetched rows. The jobs module adds an optional parameter to a workflow fetch call to control group data retrieval.

Changes

Cohort / File(s)	Summary
PostgreSQL Connector Optimization src/utils/connectors/postgres.py	Replaces dynamic Pydantic model construction with SimpleNamespace objects for fetched rows when return_raw is False, reducing overhead. Adds types import to support SimpleNamespace creation. Preserves existing error handling and cursor management logic.
Workflow Fetch Refinement src/utils/job/jobs.py	Updates Workflow.fetch_from_db() call to include fetch_groups=False parameter, enabling lightweight workflow retrieval for timeout calculations without loading related group data.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A rabbit hops through the data stream,
SimpleNamespace—a lighter dream!
No Pydantic models weighing us down,
Fetch groups optional, let workflows fly 'round.
Efficiency gains, both nimble and lean! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Speedup UpdateGroup Jobs' accurately describes the main objective of the changeset, which is to optimize UpdateGroup job performance through workflow fetch optimization and replacing pydantic models with SimpleNamespace objects.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ethany/speedup_update_group

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}