Skip to content

Extract shared WarehouseResource from raw psycopg2 usage #46

@24c02

Description

@24c02

Problem

Every asset that touches postgres does its own os.getenv("WAREHOUSE_COOLIFY_URL") + psycopg2.connect(). There are 10+ files doing this independently with no shared connection management.

Proposal

Create a WarehouseResource(ConfigurableResource) that:

  • Takes connection_url via EnvVar("WAREHOUSE_COOLIFY_URL")
  • Exposes a get_connection() context manager
  • Gets wired into Definitions once and injected via type annotations

Then migrate existing assets to use it instead of raw psycopg2.

Files to migrate

  • defs/airtable_raw_all_bases/definitions.py
  • defs/airtable_audit_logs/definitions.py
  • defs/loops_campaign_and_metrics_export/definitions.py
  • defs/ysws_programs_sync/definitions.py
  • defs/unified_ysws_db/definitions.py
  • defs/agh_fulfillment_zenventory/definitions.py
  • defs/zenventory_inventory_airtable_sync/definitions.py
  • defs/slack_users_sync/definitions.py
  • defs/slack/definitions.py
  • defs/ducklake/definitions.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions