Skip to content

test: give the debug cases and annual integration their own predbat instance - #4530

Merged
springfall2008 merged 1 commit into
mainfrom
test/isolate-debug-cases
Aug 15, 2026
Merged

test: give the debug cases and annual integration their own predbat instance#4530
springfall2008 merged 1 commit into
mainfrom
test/isolate-debug-cases

Conversation

@springfall2008

Copy link
Copy Markdown
Owner

Fixes the order-sensitivity that has already forced two golden regenerations and produced several confusing failures.

The problem

run_debug_cases planned against the shared my_predbat, and read_debug_yaml only restores the attributes its dump actually carries — so anything the dump omits inherited whatever the previous test left behind.

Two consequences, both real:

  • The goldens depended on test ordering. Reordering TEST_REGISTRY changed which state leaked in, which changed the plan, which broke predbat_debug_pre_saving1 and forced a regeneration that had nothing to do with the planner.
  • The suite and the CLI disagreed. --debug cases/<case>.yaml standalone produced a different plan from the same case run inside the suite, so reproducing a suite failure by hand didn't reproduce the plan.

test_single_debug.py already carried a hand-maintained list of attributes to reset for exactly this reason — dynamic_load_baseline, battery_rate_max_export, rate_max_base, rate_export_max_forward. That is whack-a-mole; the next omitted attribute starts it again.

The fix

Each case gets a freshly created instance, built the same way the standalone path builds one.

Verified predbat_debug_pre_saving1 now produces a byte-identical plan in all three contexts:

  • --debug cases/predbat_debug_pre_saving1.yaml standalone
  • ./run_all --test debug_cases
  • the full ./run_all

Previously the first differed from the last.

What that uncovered

Removing the leak removed a mask. debug_cases had been overwriting most of the shared instance immediately before annual_integration ran, and annual_integration never sets up the state it plans against. Left with the ambient state instead, it went from 34s to 463s — silently, still passing.

It gets a fresh instance for the same reason. That also makes it faster than it was before any of this: 55s → 34s, because it is no longer planning against whatever a previous test happened to leave.

Effect

before after
full ./run_all 144.77s 119.28s
annual_integration 55s 34.10s
debug_cases ~25s 24.76s

Faster, and no longer dependent on what ran before what.

./run_all and ./run_all --quick both pass; pre-commit clean.

Note

The per-attribute resets in test_single_debug.py are now redundant, since a fresh instance starts from the same defaults they were restoring. I have left them: they are harmless, and removing them is a separate change that wants its own verification.

🤖 Generated with Claude Code

…nstance

The debug regression cases planned against the shared instance, and read_debug_yaml only restores the
attributes its dump actually carries - so anything the dump omits inherited whatever the previous test
left behind. That made the golden plans depend on test ordering, and made the plan produced inside the
suite differ from the one `--debug <case>` produces standalone. Both have caused real confusion: the
goldens have had to be regenerated simply because the registry order changed, and a case that passes
alone could fail in the suite.

Each case now gets a freshly created instance, built the same way the standalone path builds one. The
three now agree exactly - verified that predbat_debug_pre_saving1 produces an identical plan standalone,
running debug_cases alone, and in the full suite.

Doing that removed a mask. debug_cases had been overwriting most of the shared instance immediately
before annual_integration ran, and annual_integration never sets up the state it plans against; left
with the ambient state instead it went from 34s to 463s - silently, still passing. It gets a fresh
instance for the same reason, which also makes it faster than it was before any of this (55s -> 34s).

Full suite 144.77s -> 119.86s, and it no longer depends on what ran before what.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 15, 2026 12:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes order-sensitive state leakage in the unit-test harness by ensuring tests that load partial debug dumps (and tests that implicitly depend on prior state) run against freshly created PredBat instances, matching the standalone --debug <case> execution path.

Changes:

  • Update run_debug_cases to create a new PredBat instance per debug YAML case instead of reusing the shared my_predbat.
  • Introduce run_annual_integration_isolated and route annual_integration through it so the integration test runs against a fresh instance rather than inherited ambient state.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread apps/predbat/unit_test.py
@springfall2008
springfall2008 merged commit a131f12 into main Aug 15, 2026
3 checks passed
@springfall2008
springfall2008 deleted the test/isolate-debug-cases branch August 15, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants