fix: stop planning a sub-floor solar export the inverter will absorb (#630)

CHANGELOG.md

@@ -22,6 +22,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Fixed
 
+- **A tiny solar surplus is no longer planned as an export the inverter will absorb** — below the export the plan can express, the battery charged anyway and ran fuller than planned, spilling the difference later. ([#630](https://github.com/johanzander/bess-manager/issues/630))
 - **The setup wizard no longer locks you out of an inverter platform it failed to auto-detect** — every platform stays selectable, and a re-scan keeps the one you picked. ([#621](https://github.com/johanzander/bess-manager/issues/621))
 - **System no longer gets stuck on "initializing" when many consecutive periods are near-tied** — a long run of volatile prices could make every hourly optimization fail, leaving no schedule at all. ([#624](https://github.com/johanzander/bess-manager/issues/624))
 - **Grid charging now reaches the planned amount instead of stopping just short** — the charge rate is written as a whole percent, and rounding it down meant the battery charged slightly less than the plan counted on.

core/bess/action_selector.py

@@ -17,7 +17,8 @@
 evaluators for speed (P1 permits this explicitly: they estimate V over a
 whole state grid at once and never emit an action). They consume this
 module's candidate-space definitions -- `_residual_cover_p`,
-`_discharge_is_unexecutable` -- and the platform lattice from
+`_discharge_is_unexecutable`, `_solar_export_bypass_is_unexecutable` -- and
+the platform lattice from
 `execution_model.PlatformCapabilities` rather than restating them, so the
 action space itself has one definition even where the evaluation loop does
 not.
@@ -31,6 +32,7 @@
 from core.bess.dp_battery_algorithm import (
     POWER_TOLERANCE_KW,
     PeriodFlows,
+    _ac_flows,
     _compute_reward,
     _effective_ac_cap_kwh,
     _soe_floor,
@@ -116,6 +118,60 @@ def _discharge_is_unexecutable(
     )
 
 
+def _solar_export_bypass_is_unexecutable(
+    solar_production: float,
+    home_consumption: float,
+    battery_settings: BatterySettings,
+    dt: float,
+) -> bool:
+    """Is the SOLAR_EXPORT-below-max bypass (#313) one no inverter can carry
+    out as commanded this period? (#630)
+
+    The bypass holds SoE exactly and lets the period's own solar surplus
+    export instead of passively charging. Nothing commands that directly --
+    it is delivered by `classify_strategic_intent` labelling the period
+    SOLAR_EXPORT, which is the intent that writes charge rate 0 and so stops
+    the inverter absorbing the surplus. That label needs
+    `grid_exported > FLOW_NOISE_FLOOR_KWH`; at or below it the period falls
+    through to IDLE, whose command is `load_first` at charge rate 100 --
+    which absorbs. So the plan books export revenue on energy the hardware
+    puts in the battery, and the battery runs fuller than planned until it
+    hits a bound and spills the difference (#630: +0.0016 SEK over one
+    quarterly day, from a single 0.0034 kWh surplus).
+
+    This is the charge-side twin of `_residual_cover_p`'s first gate, which
+    excludes discharges the same classifier would call IDLE (the #282 shape),
+    and it gates on the same constant for the same reason -- one threshold,
+    so the candidate space and the classifier cannot drift apart.
+
+    Withholding the candidate is always safe: plain IDLE is offered
+    unconditionally, and wherever the bypass diverts nothing (no surplus, or
+    a battery already at `max_soe`) the two coincide exactly -- the same
+    duplication `_tie_margin` already documents.
+
+    Scalar per period, not per state: with no battery flow at all,
+    `_ac_flows` reads only solar, load and the AC cap, so the export this
+    candidate produces does not depend on SoE. The backward passes evaluate
+    it once for the whole bypass column rather than per grid state.
+
+    The AC cap is derived here rather than accepted as an argument, even
+    though all three call sites already hold the identical value. That
+    repeated call is deliberate, for the reason `_period_flows`' docstring
+    gives: a cap passed in is a cap a caller can get wrong, and gating this
+    candidate under a different cap than `_period_flows` prices it under is
+    the reward-vs-flows divergence P4 exists to remove. One derivation per
+    period is not a cost worth reopening that seam for.
+    """
+    _, grid_exported, _ = _ac_flows(
+        solar_production,
+        home_consumption,
+        0.0,
+        0.0,
+        _effective_ac_cap_kwh(battery_settings, dt),
+    )
+    return grid_exported <= FLOW_NOISE_FLOOR_KWH
+
+
 def _residual_cover_p(
     home_consumption: float,
     solar_production: float,
@@ -641,8 +697,11 @@ def consider(power: float, forced_next_soe: float | None = None) -> None:
     # rationale. Bypasses _state_transition (whose power=0 branch always
     # charges as much as room/rate permit) to force next_soe == soe
     # directly, then reuses the same _compute_reward call every other
-    # candidate uses.
-    consider(0.0, forced_next_soe=soe)
+    # candidate uses. Withheld where the classifier would call the period
+    # IDLE rather than SOLAR_EXPORT, since nothing then commands the hold
+    # (#630).
+    if not _solar_export_bypass_is_unexecutable(solar, home, battery_settings, dt):
+        consider(0.0, forced_next_soe=soe)
 
     # Discharge -- exact breakpoint enumeration (Finding 1/2/3/5).
     for p in _discharge_candidates(
@@ -673,12 +732,16 @@ def consider(power: float, forced_next_soe: float | None = None) -> None:
             c for c in candidates if c.grid_imported <= effective_import_cap + 1e-9
         ]
 
-    # The SOLAR_EXPORT-below-max candidate holds soe exactly unchanged, so it
-    # is feasible at every state, and the import-cap filter above cannot empty
-    # a non-empty list (its threshold is floored at the minimum grid_imported
-    # any candidate achieves, so that candidate always survives) --
-    # `candidates` is never empty and an IndexError below would be a real bug,
-    # not a case to defend against.
+    # Plain IDLE is offered unconditionally and holds soe within bounds at
+    # every state, so it is always feasible, and the import-cap filter above
+    # cannot empty a non-empty list (its threshold is floored at the minimum
+    # grid_imported any candidate achieves, so that candidate always
+    # survives) -- `candidates` is never empty and an IndexError below would
+    # be a real bug, not a case to defend against.
+    #
+    # This used to name the SOLAR_EXPORT-below-max candidate instead. That
+    # stopped being the guarantee when #630 made the bypass conditional; IDLE
+    # is the unconditional one, and always was.
     argmax_index = 0
     best_value = float("-inf")
     for index, candidate in enumerate(candidates):

core/bess/dp_battery_algorithm.py

@@ -1243,7 +1243,10 @@ def _run_dynamic_programming(
     # for the reward/transition primitives, so a top-level import would be
     # circular -- the same arrangement pwl_window_dp already has with this
     # file.
-    from core.bess.action_selector import _residual_cover_p
+    from core.bess.action_selector import (
+        _residual_cover_p,
+        _solar_export_bypass_is_unexecutable,
+    )
 
     # Set defaults if not provided
     if solar_production is None:
@@ -1385,27 +1388,40 @@ def _run_dynamic_programming(
         # to preserve headroom for above-cap solar: ac_flows_grid caps the
         # exported surplus and the DP weighs the clipped remainder against
         # the value of keeping the room (no separate HOLD action needed).
-        zeros_col = np.zeros_like(soe_col)
-        reward_bypass, grid_imported_bypass = _compute_reward_grid(
-            zeros_col,
-            soe_col,
-            soe_col,
-            home_consumption=home_consumption[t],
-            battery_settings=battery_settings,
-            dt=dt,
-            current_buy_price=buy_price[t],
-            current_sell_price=sell_price[t],
-            solar_production=solar_production[t],
-            import_cap_kwh=import_cap_kwh,
-        )
-        value_bypass = reward_bypass.reshape(-1) + V[t + 1][np.arange(n_states)]
-        if effective_import_cap is not None:
-            bypass_feasible = (
-                grid_imported_bypass.reshape(-1)
-                <= effective_import_cap.reshape(-1) + 1e-9
+        #
+        # Withheld entirely where the classifier would call the period IDLE
+        # rather than SOLAR_EXPORT (#630) -- there the bypass is not a
+        # commandable action at all, and plain IDLE is what the hardware
+        # does. Unlike the discharge mask above, this one DOES apply here:
+        # that exception rests on the coarse lattice having no exact-cover
+        # point nearby, so an in-band action is the better proxy. Nothing
+        # analogous holds here -- the bypass is an exact action, not a
+        # lattice approximation, and its executable neighbour (plain IDLE)
+        # is already in the action set at every state.
+        if not _solar_export_bypass_is_unexecutable(
+            solar_production[t], home_consumption[t], battery_settings, dt
+        ):
+            zeros_col = np.zeros_like(soe_col)
+            reward_bypass, grid_imported_bypass = _compute_reward_grid(
+                zeros_col,
+                soe_col,
+                soe_col,
+                home_consumption=home_consumption[t],
+                battery_settings=battery_settings,
+                dt=dt,
+                current_buy_price=buy_price[t],
+                current_sell_price=sell_price[t],
+                solar_production=solar_production[t],
+                import_cap_kwh=import_cap_kwh,
             )
-            value_bypass = np.where(bypass_feasible, value_bypass, -np.inf)
-        V[t, :] = np.maximum(V[t, :], value_bypass)
+            value_bypass = reward_bypass.reshape(-1) + V[t + 1][np.arange(n_states)]
+            if effective_import_cap is not None:
+                bypass_feasible = (
+                    grid_imported_bypass.reshape(-1)
+                    <= effective_import_cap.reshape(-1) + 1e-9
+                )
+                value_bypass = np.where(bypass_feasible, value_bypass, -np.inf)
+            V[t, :] = np.maximum(V[t, :], value_bypass)
 
         # Residual load-cover candidate (#466 follow-up): one extra
         # O(n_states) column discharging exactly this period's forecast net

core/bess/pwl_window_dp.py

@@ -17,6 +17,7 @@
     PeriodInputs,
     _discharge_is_unexecutable,
     _residual_cover_p,
+    _solar_export_bypass_is_unexecutable,
     select_action,
 )
 from core.bess.dp_battery_algorithm import (
@@ -258,6 +259,17 @@ def _pwl_candidate_values_at(
     # delta -> battery_charged=0, so grid_exported reflects the full
     # surplus). With the AC cap set, this candidate is also what defers
     # charging to preserve headroom for above-cap solar.
+    #
+    # Withheld where the classifier would call the period IDLE rather than
+    # SOLAR_EXPORT (#630): nothing commands the hold there, so it is not an
+    # action this pass may value. Plain IDLE (power=0, already in the main
+    # grid above) is what the hardware does instead, which is why dropping
+    # the column cannot leave a row without a finite action.
+    if _solar_export_bypass_is_unexecutable(
+        solar_production[t], home_consumption[t], battery_settings, dt
+    ):
+        return value.max(axis=1)
+
     zeros_col = np.zeros_like(soe_col)
     reward_bypass, grid_imported_bypass = _compute_reward_grid(
         zeros_col,

core/bess/tests/integration/test_plan_faithfulness.py

@@ -191,25 +191,15 @@ def test_scenarios_are_plan_faithful_realized_equals_planned():
     # curtailment-affected export volume shifts slightly with the plan);
     # was +0.0203 under the 0.2 kW / 0.05 kWh grid.
     "regression_2026_08_08_143843": +0.0214,
-    # Sub-grid-step passive solar charging at IDLE. Period 32 has a 0.0034 kWh
-    # solar surplus -- an order of magnitude below SOE_STEP_KWH (0.025), so the
-    # DP's snapped SOE trajectory cannot represent absorbing it and plans it as
-    # export. The command it derives (load_first, charge_rate_pct=100) makes
-    # the inverter absorb it anyway, so the battery runs slightly fuller than
-    # planned. The drift is costless until period 64, where the battery reaches
-    # max_soe: being 0.0033 kWh fuller leaves that much less room for solar,
-    # which is exported instead, for a +0.0016 SEK realized-vs-planned gap.
-    #
-    # NOT caused by #624's window bisection, and measured rather than assumed:
-    # with `detect_tie_windows` patched to return no windows -- no PWL solve,
-    # no splice, no bisection -- this fixture's gap is bit-identical at
-    # +0.001555. Two reviewers independently attributed it to the bisection
-    # seam; the seam's own measured contribution is 0.000000 SEK.
-    #
-    # Goes to zero when the DP's IDLE flow model credits passive charging
-    # below one SOE step, or when the trajectory stops being snapped. This
-    # entry is the record of a real defect, not a tolerance for one.
-    "regression_2026_08_17_624": +0.0016,
+    # `regression_2026_08_17_624`'s +0.0016 entry lived here between #629 and
+    # #630 and is now gone: that fixture's gap is +0.000000. The cause was not
+    # the SOE-grid snapping this entry described -- the forward replay carries
+    # continuous SoE, and period 32's SoE was held bit-exactly, which snapping
+    # would not produce. The DP was choosing the SOLAR_EXPORT-below-max bypass
+    # (#313) for a surplus too small to classify SOLAR_EXPORT, so the derived
+    # IDLE command absorbed what the plan exported. See
+    # `_solar_export_bypass_is_unexecutable` and
+    # `test_subfloor_solar_export_is_never_planned` below (#630).
 }
 
 
@@ -248,6 +238,77 @@ def test_realized_matches_planned_across_all_fixtures():
     )
 
 
+def test_subfloor_solar_export_is_never_planned():
+    """Regression for issue #630.
+
+    The DP offers a "hold SoE, export this period's own solar surplus"
+    candidate (the SOLAR_EXPORT-below-max bypass, #313). It is only executable
+    where `classify_strategic_intent` will actually label the period
+    SOLAR_EXPORT, because that intent is what writes charge rate 0 and stops
+    the inverter absorbing the surplus. Below the classifier's
+    `FLOW_NOISE_FLOOR_KWH`, the period falls through to IDLE instead, whose
+    command is `load_first` at charge rate 100 -- so the plan books export
+    revenue on energy the hardware puts in the battery. Same shape as #282 on
+    the discharge side, which `_residual_cover_p` already gates against.
+
+    The scenario is built to make the bypass the DP's honest choice, so the
+    plan is wrong for a real economic reason rather than by accident:
+
+    - period 0's surplus is 0.0099 kWh, just under the 0.01 kWh floor
+    - periods 1-2 carry enough surplus to fill the battery to `max_soe`
+      regardless, so the marginal value of storing period 0's surplus is only
+      the export it displaces later (sell 0.2), not its evening value
+    - selling it now at 0.5 therefore beats storing it, by more than the
+      cycle cost
+    - a larger discharge-and-refill at period 0 is unprofitable
+      (0.5 sell - 0.4 cycle - 0.2 displaced export < 0), so the DP has no
+      reason to reach for a discharge candidate instead
+
+    `terminal_value_per_kwh` is pinned to 0.0 so the leftover-SoE term cannot
+    quietly change which candidate wins as prices are tuned.
+
+    Measured on the pre-fix code: R=2.404206 P=2.401236, gap +0.002970 SEK --
+    three times `PLAN_EXECUTION_TOLERANCE_SEK`.
+    """
+    from core.bess.tests.helpers import run_scenario_realized
+
+    scenario = {
+        "buy_price": [1.2, 1.2, 1.2, 3.0, 3.0, 3.0],
+        "sell_price": [0.5, 0.2, 0.2, 0.2, 0.2, 0.2],
+        "solar_production": [0.5099, 2.0, 2.0, 0.0, 0.0, 0.0],
+        "home_consumption": [0.5, 0.5, 0.5, 6.0, 6.0, 6.0],
+        "terminal_value_per_kwh": 0.0,
+        "battery": {
+            "max_soe_kwh": 20.0,
+            "min_soe_kwh": 2.2,
+            "max_charge_power_kw": 10.0,
+            "max_discharge_power_kw": 10.0,
+            "efficiency_charge": 0.97,
+            "efficiency_discharge": 0.97,
+            "cycle_cost_per_kwh": 0.40,
+            "initial_soe": 19.0,
+        },
+    }
+
+    result, realized = run_scenario_realized(scenario)
+    planned = result.economic_summary.battery_solar_cost
+
+    # The outcome that matters: executing this plan costs what it said it would.
+    assert abs(realized - planned) <= PLAN_EXECUTION_TOLERANCE_SEK, (
+        f"plan books a cost its own execution does not reproduce: "
+        f"R={realized:.6f} P={planned:.6f} gap={realized - planned:+.6f}"
+    )
+
+    # The mechanism, so a future regression is diagnosable and not just red:
+    # period 0 must absorb its sub-floor surplus, because that is what the
+    # command derived for it does.
+    p0 = result.period_data[0].energy
+    assert p0.grid_exported == pytest.approx(0.0, abs=1e-9), (
+        f"period 0 plans a {p0.grid_exported:.5f} kWh export below the "
+        f"classifier's noise floor -- the derived command absorbs it instead"
+    )
+
+
 def _battery(initial_soe):
     return {
         "max_soe_kwh": 20.0,