-
Notifications
You must be signed in to change notification settings - Fork 63
Description
Relevant to #819 and ultimately #6 .
gen_hydro projects are linked to horizons in inputs_project_hydro_operational_characteristics. The way hydro data is read from the database is to pick the intersection of these horizons, with those specified in the inputs_temporal_horizons_timepoints table :
gridpath/gridpath/project/operations/operational_types/common_functions.py
Lines 824 to 835 in ad4ffd1
| -- Limit to bt-horizons from this temporal scenario ID | |
| JOIN ( | |
| SELECT DISTINCT balancing_type_horizon, horizon | |
| FROM inputs_temporal_horizon_timepoints | |
| WHERE temporal_scenario_id = {subscenarios.TEMPORAL_SCENARIO_ID} | |
| AND subproblem_id = {subproblem} | |
| AND stage_id = {stage} | |
| ) as hrz_tbl | |
| ON ( | |
| balancing_type_project = balancing_type_horizon | |
| AND prj_tbl.horizon = hrz_tbl.horizon | |
| ) |
When horizons are missing from the temporal table, the intersection is also the null set and therefore the tab files aren't even written:
gridpath/gridpath/project/operations/operational_types/common_functions.py
Lines 446 to 449 in ad4ffd1
| # Only write if we have data | |
| data_list = [row for row in data.fetchall()] | |
| if data_list: | |
| with open(out_file, append_mode, newline="") as f: |
This results in a file not found error:
ERROR:root:[Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' and FileNotFoundError: [Errno 2] No such file or directory: '../scenarios_etc-dev/2horizons_w_hydro_w_balancing_types/inputs/hydro_conventional_horizon_params.tab' .
I see three equally valid ways to handle this:
- Make sure all empty files are written, with (perhaps?) the result that the hydro projects are never simulated. This aligns with existing practice of silently dropping mismatched horizons.
- Make mismatched horizons an error, and use LEFT JOIN instead of INNER JOIN while fetching the hydro opchar data. This entails a lot of changes throughout the test suite.
- Keep existing behaviour as by-design.
I am in the process of writing validation to catch this case, but before that wanted to understand if current operational behaviour is intended.