Skip to content

Commit c872dae

Browse files
committed
Fix: Standardize date_local format between AQS and Envista data
- Convert Envista date_local from ISO timestamp to YYYY-MM-DD format in transform_env.py - Update calculate_aqi.py cutoff_date to use timezone-naive datetime for comparison - Ensures consistent date format when AQS and Envista data are consolidated
1 parent 22f59a4 commit c872dae

2 files changed

Lines changed: 5 additions & 1 deletion

File tree

src/envista/transformers/calculate_aqi.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,9 @@ def pm25_to_aqi_new(concentration: float) -> int | Any:
5252
def pm25_to_aqi_with_date_check(row: pd.Series) -> int | Any:
5353
"""Apply appropriate AQI calculation based on date."""
5454
concentration = row["arithmetic_mean"]
55+
# Parse date_local without timezone info and compare with naive datetime
5556
date_local = pd.to_datetime(row["date_local"])
56-
cutoff_date = pd.to_datetime("2024-05-06 00:00:00-0800")
57+
cutoff_date = pd.to_datetime("2024-05-06")
5758

5859
if date_local < cutoff_date:
5960
return pm25_to_aqi_old(concentration)

src/envista/transformers/transform_env.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,9 @@ def transform_env_daily(year: str, raw_daily_files: list[Path], unique_monitors:
6161
"data_channels_valid":"validity_indicator",
6262
"stations_tag":"site_code"})
6363

64+
# Standardize date_local format to match AQS (YYYY-MM-DD)
65+
transformed_df["date_local"] = pd.to_datetime(transformed_df["date_local"]).dt.strftime("%Y-%m-%d")
66+
6467
# Add and populate columns to match AQS schema
6568
transformed_df["poc"] = "99" # Dummy value to distinguish from AQS data
6669
transformed_df["parameter_code"] = "88502" # True AQS code for non-regulatory PM2.5 data

0 commit comments

Comments
 (0)