Merge pull request #16 from dagster-io/colton/integrations-audit-via-docs

[dagster-integrations,dagster-conventions] audit via documentation reference

Author: cmpadden

CHANGELOG.md

@@ -8,14 +8,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **dagster-integrations**: Added comprehensive audit summary document (AUDIT_SUMMARY.md)
+  - Documents all 82+ integration verifications
+  - Lists critical fixes applied and remaining work
 
 ### Changed
+- **dagster-integrations**: Updated storage.md integration examples
+  - Improved Postgres section with ConfigurableResource pattern and instance storage clarification
+  - Updated Delta Lake to use correct `DeltaTableResource` API
+  - Enhanced Redshift example with `RedshiftClientResource`
+  - Improved DataHub example with correct `DatahubRESTEmitterResource`
+  - Updated LakeFS to show custom resource pattern with lakefs-client SDK
 
 ### Deprecated
 
 ### Removed
+- **dagster-integrations**: Removed fabricated integrations from storage.md
+  - Removed MongoDB section (dagster-mongo package doesn't exist)
+  - Removed Atlan section (Dagster+ cloud CLI feature, not a Python library)
+  - Removed Secoda section (no library integration exists)
 
 ### Fixed
+- **dagster-integrations**: Fixed critical PostgresResource hallucinations across multiple files
+  - storage.md: Replaced fabricated dagster-postgres resource API with custom ConfigurableResource pattern
+  - Fixed 5 support level misclassifications:
+    - dagster-looker: Community-supported → Dagster-supported
+    - dagster-sigma: Community-supported → Dagster-supported
+    - dagster-polars: Dagster-supported → Community-supported
+    - dagster-wandb: Dagster-supported → Community-supported
+    - dagster-papertrail: Community-supported → Dagster-supported
+- **dagster-conventions**: Fixed PostgresResource hallucinations in resources.md
+  - Updated factory pattern example with custom PostgresResource definition
+- **dagster-conventions**: Fixed PostgresResource hallucinations in testing.md
+  - Updated test examples with custom ConfigurableResource implementations
+- **dagster-integrations**: Fixed other.md Polars integration
+  - Removed non-existent `PolarsDataFrame` import
+  - Corrected support level to Community-supported
 
 ### Security
 

plugins/dagster-conventions/skills/dagster-conventions/references/resources.md

@@ -275,6 +275,26 @@ SNOWFLAKE_WAREHOUSE=PROD_WH
 
 ```python
 from typing import Literal
+from dagster import ConfigurableResource
+import psycopg2
+
+# Define custom Postgres resource (dagster-postgres only provides instance storage)
+class PostgresResource(ConfigurableResource):
+    """Custom Postgres resource for asset data access."""
+    host: str
+    database: str
+    user: str
+    password: str
+    port: int = 5432
+
+    def get_connection(self):
+        return psycopg2.connect(
+            host=self.host,
+            port=self.port,
+            user=self.user,
+            password=self.password,
+            database=self.database
+        )
 
 def create_database_resource(
     env: Literal["dev", "staging", "prod"]

plugins/dagster-conventions/skills/dagster-conventions/references/testing.md

@@ -613,6 +613,27 @@ def test_materialization_with_mocked_resource():
 ### Test with Real Services
 
 ```python
+from dagster import ConfigurableResource
+import psycopg2
+
+# Define custom Postgres resource (dagster-postgres only provides instance storage)
+class PostgresResource(ConfigurableResource):
+    """Custom Postgres resource for asset data access."""
+    host: str
+    port: int = 5432
+    database: str
+    user: str
+    password: str
+
+    def get_connection(self):
+        return psycopg2.connect(
+            host=self.host,
+            port=self.port,
+            user=self.user,
+            password=self.password,
+            database=self.database
+        )
+
 def test_database_integration():
     """Integration test with actual database."""
     postgres_resource = PostgresResource(
@@ -622,9 +643,9 @@ def test_database_integration():
         user="test_user",
         password="test_pass",
     )
-    
+
     result = state_population_database(postgres_resource)
-    
+
     assert len(result) > 0
 ```
 
@@ -666,6 +687,26 @@ services:
 ```python
 # tests/e2e/conftest.py
 import pytest
+from dagster import ConfigurableResource
+import psycopg2
+
+# Define custom Postgres resource (dagster-postgres only provides instance storage)
+class PostgresResource(ConfigurableResource):
+    """Custom Postgres resource for asset data access."""
+    host: str
+    port: int = 5432
+    database: str
+    user: str
+    password: str
+
+    def get_connection(self):
+        return psycopg2.connect(
+            host=self.host,
+            port=self.port,
+            user=self.user,
+            password=self.password,
+            database=self.database
+        )
 
 @pytest.fixture(scope="session")
 def postgres_resource():

plugins/dagster-integrations/skills/integrations-index/references/ai.md

@@ -143,7 +143,7 @@ def train_model(context):
 ---
 
 ### Weights & Biases (W&B)
-**Package:** `dagster-wandb` | **Support:** Dagster-supported
+**Package:** `dagster-wandb` | **Support:** Community-supported
 
 ML experiment tracking and visualization platform with advanced collaboration features.
 

plugins/dagster-integrations/skills/integrations-index/references/bi.md

@@ -5,7 +5,7 @@ Business intelligence and visualization platforms for data exploration, dashboar
 ---
 
 ### Looker
-**Package:** `dagster-looker` | **Support:** Community-supported
+**Package:** `dagster-looker` | **Support:** Dagster-supported
 
 Google's business intelligence platform for creating interactive dashboards and SQL-based analytics.
 
@@ -107,7 +107,7 @@ def refresh_powerbi_dataset(powerbi: PowerBIResource):
 ---
 
 ### Sigma
-**Package:** `dagster-sigma` | **Support:** Community-supported
+**Package:** `dagster-sigma` | **Support:** Dagster-supported
 
 Cloud-native analytics and BI platform with spreadsheet-like interface for data exploration.
 

plugins/dagster-integrations/skills/integrations-index/references/monitoring.md

@@ -91,7 +91,7 @@ def track_metric(prometheus: PrometheusResource):
 ---
 
 ### Papertrail
-**Package:** `dagster-papertrail` | **Support:** Community-supported
+**Package:** `dagster-papertrail` | **Support:** Dagster-supported
 
 Send logs to Papertrail for centralized log management.
 

plugins/dagster-integrations/skills/integrations-index/references/other.md

@@ -46,7 +46,7 @@ def events() -> EventDataFrame:
 ---
 
 ### Polars
-**Package:** `dagster-polars` | **Support:** Dagster-supported
+**Package:** `dagster-polars` | **Support:** Community-supported
 
 Fast DataFrame library with columnar storage and lazy evaluation, often 5-10x faster than pandas.
 
@@ -58,7 +58,6 @@ Fast DataFrame library with columnar storage and lazy evaluation, often 5-10x fa
 
 **Quick start:**
 ```python
-from dagster_polars import PolarsDataFrame
 import polars as pl
 
 @dg.asset