Skip to content

Commit a131f12

Browse files
test: give the debug cases and annual integration their own predbat instance (#4530)
The debug regression cases planned against the shared instance, and read_debug_yaml only restores the attributes its dump actually carries - so anything the dump omits inherited whatever the previous test left behind. That made the golden plans depend on test ordering, and made the plan produced inside the suite differ from the one `--debug <case>` produces standalone. Both have caused real confusion: the goldens have had to be regenerated simply because the registry order changed, and a case that passes alone could fail in the suite. Each case now gets a freshly created instance, built the same way the standalone path builds one. The three now agree exactly - verified that predbat_debug_pre_saving1 produces an identical plan standalone, running debug_cases alone, and in the full suite. Doing that removed a mask. debug_cases had been overwriting most of the shared instance immediately before annual_integration ran, and annual_integration never sets up the state it plans against; left with the ambient state instead it went from 34s to 463s - silently, still passing. It gets a fresh instance for the same reason, which also makes it faster than it was before any of this (55s -> 34s). Full suite 144.77s -> 119.86s, and it no longer depends on what ran before what. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 9ae81c1 commit a131f12

1 file changed

Lines changed: 25 additions & 5 deletions

File tree

apps/predbat/unit_test.py

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -220,8 +220,14 @@
220220

221221

222222
def run_debug_cases(my_predbat):
223-
"""
224-
Run debug case files from the cases directory
223+
"""Run debug case files from the cases directory.
224+
225+
my_predbat is deliberately unused: each case gets a freshly created instance instead. read_debug_yaml
226+
only restores the attributes its dump actually carries, so on a shared instance anything the dump omits
227+
inherits whatever the previous test happened to leave behind - which made these cases depend on test
228+
ordering, and made the plan produced here differ from the one `--debug <case>` produces standalone.
229+
Neither is a property a golden regression test can afford. Building the instance the same way the
230+
standalone path does makes the two agree and makes the result independent of what ran before.
225231
"""
226232
failed = False
227233
print("**** Running debug case files ****")
@@ -235,8 +241,9 @@ def run_debug_cases(my_predbat):
235241
pathname = os.path.dirname(filename)
236242
if basename == "random_scenarios.yaml":
237243
continue # Skip the random scenarios template file
238-
test_failed = run_single_debug(basename, my_predbat, filename, pathname + "/" + basename + ".expected.json")
239-
total_calculate_plan_time += getattr(my_predbat, "last_calculate_plan_time", 0.0)
244+
case_predbat = create_predbat()
245+
test_failed = run_single_debug(basename, case_predbat, filename, pathname + "/" + basename + ".expected.json")
246+
total_calculate_plan_time += getattr(case_predbat, "last_calculate_plan_time", 0.0)
240247
case_count += 1
241248
if test_failed:
242249
print(f"**** Debug case {basename}: FAILED ****")
@@ -251,6 +258,19 @@ def run_debug_cases(my_predbat):
251258
return failed
252259

253260

261+
def run_annual_integration_isolated(my_predbat):
262+
"""Run the annual integration test against a freshly created instance.
263+
264+
my_predbat is unused, for the same reason run_debug_cases ignores it: this test plans a year of
265+
sampled days against whatever state the shared instance is carrying, and never sets that state up
266+
itself. It used to be shielded by debug_cases running immediately before it and overwriting most of
267+
the instance from a debug dump; once debug_cases stopped mutating the shared instance, the ambient
268+
state it inherited instead made it 13x slower (34s -> 463s) without ever failing, which is exactly
269+
the kind of coupling a test suite should not have.
270+
"""
271+
return test_annual_integration(create_predbat())
272+
273+
254274
def create_predbat():
255275
my_predbat = PredBat()
256276
my_predbat.states = {}
@@ -499,7 +519,7 @@ def main():
499519
("annual_store", test_annual_store, "Annual run store tests", False),
500520
("annual_costs", test_annual_costs, "Annual install cost and payback model tests", False),
501521
("tariff_catalogue", test_tariff_catalogue, "Tariff catalogue tests", False),
502-
("annual_integration", test_annual_integration, "Annual prediction integration tests", True),
522+
("annual_integration", run_annual_integration_isolated, "Annual prediction integration tests", True),
503523
("load_ml", test_load_ml, "ML Load Forecaster tests (MLP, training, persistence, validation)", True),
504524
]
505525

0 commit comments

Comments
 (0)