Skip to content

simulator.md

Latest commit

 

History

History
84 lines (70 loc) · 4.58 KB

simulator.md

File metadata and controls

84 lines (70 loc) · 4.58 KB
Raw

Plan-Faithfulness Simulator (R == P)

Audience: agents and developers changing the optimizer, intent classification, or inverter control mapping. For the human-facing overview, see docs/DEVELOPMENT.md → Testing.

The single most valuable verification tool in this codebase. Every savings number the optimizer reports is a plan (P) — computed from the chosen battery actions. But the inverter is driven by coarse modes (grid_first / load_first / battery_first + charge/discharge rates), and a mode is a policy, not a power setpoint. So the plan can claim things the hardware can't deliver. Plan-only tests (run_scenario, scenario expected_results) cannot see this gap.

The simulator (core/bess/simulation/) closes the loop: it takes the control commands derived from a plan and computes the realized economics (R) of executing them. The invariant is:

R == P — executing the optimizer's plan through the inverter must reproduce the planned economics (to within the DP's SoE/power-grid resolution). A larger gap is a real control-fidelity bug, not noise.

This is pure simulation (no hardware, no mock-HA), so it runs in unit tests.

Core API (core/bess/simulation/)

from core.bess.simulation.inverter_simulator import (
    derive_control_command,  # plan period (intent + power [+ intra_period_discharge_allowed]) -> ControlCommand
    simulate,                # execute commands on conditions -> realized PeriodData + cost
)
from core.bess.simulation.verification import (
    verify_plan_faithfulness,   # optimize -> derive -> simulate -> (P, R, per-period deltas)
    ab_compare,                 # realized-cost delta between two command sequences (exact)
    realized_under_solar_error, # optimize on FORECAST solar, execute on ACTUAL solar (robustness)
)

In scenario tests, use the helper run_scenario_realized(scenario) (in tests/helpers.py) — it returns (result, realized_cost) so you can assert R == P alongside the normal plan checks. test_scenarios.py does this for every scenario.

When to use it — REQUIRED for any optimizer or control change

If you touch the DP (dp_battery_algorithm.py), the intent classification (strategic_intent.py), or the control mapping (inverter_controller.py), you MUST verify R == P still holds — a passing plan-only test means nothing if the plan isn't executable. Add/keep an R == P assertion for the affected scenarios.

What it has already caught (why it exists)

  • Phantom solar-export over-crediting (#145): the optimizer booked export of surplus a load_first "store" period actually stores → ~8–16% inflated savings on sunny days. Invisible to plan-only tests.
  • Discharge can't be paced for home support (#147): LOAD_SUPPORT dumps the battery greedily on deep evening peaks → realizes ~3–4% worse than planned.
  • Shadow-price discharge gate was invisible to the simulator (#388): derive_control_command had no way to reach the same intra_period_discharge_gate production uses for SOLAR_EXPORT/SOLAR_STORAGE (#187/#319) — an R == P test for gate-related changes was structurally unsatisfiable until this was fixed. Since #526 the parameter is intra_period_discharge_allowed: bool | None, not the old shadow_price/buy_price pair. The DP decides the gate where it owns the value function and records it on pd.decision.intra_period_discharge_allowed; pass that value through to exercise the gate. None means "this caller is not exercising the gate" and leaves it closed — deliberately distinct from False, which is the DP deciding against opening it. Do not reintroduce a scalar here: a shadow_price of 0.0 cannot be told apart from one that was never computed, which is the defect #526 removed. Production also applies this gate to LOAD_SUPPORT on TOU/register platforms (#384, reverted by #393, re-landed by #520) — the simulator deliberately does not mirror that, because a 15-min point-forecast simulator has no sub-period excess load to cover and so can only model the gate's cost, never its benefit. See docs/agents/bess-knowledge.md's SOLAR_EXPORT discharge gate section.

Known gaps and xfail

A real, unfixed R != P gap should be xfail'd with a tracking issue (so it stays visible), never hidden under a loose tolerance. See the DISCHARGE_PACING_SCENARIOS set in test_scenarios.py (→ #147). The grid-resolution residual (max(0.5 SEK, 1% of |P|) in test_scenarios.py) is the only tolerance — it reflects the DP's 0.1 kWh SoE grid, not a fudge factor.