fix: stop planning a sub-floor solar export the inverter will absorb (#630)

CHANGELOG.md

@@ -22,6 +22,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Fixed
 
+- **A tiny solar surplus is no longer planned as an export the inverter will absorb** — below the export the plan can express, the battery charged anyway and ran fuller than planned, spilling the difference later. ([#630](https://github.com/johanzander/bess-manager/issues/630))
 - **Grid charging now reaches the planned amount instead of stopping just short** — the charge rate is written as a whole percent, and rounding it down meant the battery charged slightly less than the plan counted on.
 - **Growatt VPP no longer briefly executes the previous period's power command when switching modes** — enabling remote control commits immediately, so the power target is now written before it, and cleared on release. ([#593](https://github.com/johanzander/bess-manager/issues/593))
 - **The battery now covers house load exactly instead of exporting a few Wh and committing the inverter** — a period whose load fell between two discharge steps was planned as a small export, which forces `grid_first` at a fixed rate and imports any load spike. ([#352](https://github.com/johanzander/bess-manager/issues/352))

core/bess/action_selector.py

@@ -17,7 +17,8 @@
 evaluators for speed (P1 permits this explicitly: they estimate V over a
 whole state grid at once and never emit an action). They consume this
 module's candidate-space definitions -- `_residual_cover_p`,
-`_discharge_is_unexecutable` -- and the platform lattice from
+`_discharge_is_unexecutable`, `_solar_export_bypass_is_unexecutable` -- and
+the platform lattice from
 `execution_model.PlatformCapabilities` rather than restating them, so the
 action space itself has one definition even where the evaluation loop does
 not.
@@ -31,6 +32,7 @@
 from core.bess.dp_battery_algorithm import (
     POWER_TOLERANCE_KW,
     PeriodFlows,
+    _ac_flows,
     _compute_reward,
     _effective_ac_cap_kwh,
     _soe_floor,
@@ -116,6 +118,60 @@ def _discharge_is_unexecutable(
     )
 
 
+def _solar_export_bypass_is_unexecutable(
+    solar_production: float,
+    home_consumption: float,
+    battery_settings: BatterySettings,
+    dt: float,
+) -> bool:
+    """Is the SOLAR_EXPORT-below-max bypass (#313) one no inverter can carry
+    out as commanded this period? (#630)
+
+    The bypass holds SoE exactly and lets the period's own solar surplus
+    export instead of passively charging. Nothing commands that directly --
+    it is delivered by `classify_strategic_intent` labelling the period
+    SOLAR_EXPORT, which is the intent that writes charge rate 0 and so stops
+    the inverter absorbing the surplus. That label needs
+    `grid_exported > FLOW_NOISE_FLOOR_KWH`; at or below it the period falls
+    through to IDLE, whose command is `load_first` at charge rate 100 --
+    which absorbs. So the plan books export revenue on energy the hardware
+    puts in the battery, and the battery runs fuller than planned until it
+    hits a bound and spills the difference (#630: +0.0016 SEK over one
+    quarterly day, from a single 0.0034 kWh surplus).
+
+    This is the charge-side twin of `_residual_cover_p`'s first gate, which
+    excludes discharges the same classifier would call IDLE (the #282 shape),
+    and it gates on the same constant for the same reason -- one threshold,
+    so the candidate space and the classifier cannot drift apart.
+
+    Withholding the candidate is always safe: plain IDLE is offered
+    unconditionally, and wherever the bypass diverts nothing (no surplus, or
+    a battery already at `max_soe`) the two coincide exactly -- the same
+    duplication `_tie_margin` already documents.
+
+    Scalar per period, not per state: with no battery flow at all,
+    `_ac_flows` reads only solar, load and the AC cap, so the export this
+    candidate produces does not depend on SoE. The backward passes evaluate
+    it once for the whole bypass column rather than per grid state.
+
+    The AC cap is derived here rather than accepted as an argument, even
+    though all three call sites already hold the identical value. That
+    repeated call is deliberate, for the reason `_period_flows`' docstring
+    gives: a cap passed in is a cap a caller can get wrong, and gating this
+    candidate under a different cap than `_period_flows` prices it under is
+    the reward-vs-flows divergence P4 exists to remove. One derivation per
+    period is not a cost worth reopening that seam for.
+    """
+    _, grid_exported, _ = _ac_flows(
+        solar_production,
+        home_consumption,
+        0.0,
+        0.0,
+        _effective_ac_cap_kwh(battery_settings, dt),
+    )
+    return grid_exported <= FLOW_NOISE_FLOOR_KWH
+
+
 def _residual_cover_p(
     home_consumption: float,
     solar_production: float,
@@ -641,8 +697,11 @@ def consider(power: float, forced_next_soe: float | None = None) -> None:
     # rationale. Bypasses _state_transition (whose power=0 branch always
     # charges as much as room/rate permit) to force next_soe == soe
     # directly, then reuses the same _compute_reward call every other
-    # candidate uses.
-    consider(0.0, forced_next_soe=soe)
+    # candidate uses. Withheld where the classifier would call the period
+    # IDLE rather than SOLAR_EXPORT, since nothing then commands the hold
+    # (#630).
+    if not _solar_export_bypass_is_unexecutable(solar, home, battery_settings, dt):
+        consider(0.0, forced_next_soe=soe)
 
     # Discharge -- exact breakpoint enumeration (Finding 1/2/3/5).
     for p in _discharge_candidates(
@@ -673,12 +732,16 @@ def consider(power: float, forced_next_soe: float | None = None) -> None:
             c for c in candidates if c.grid_imported <= effective_import_cap + 1e-9
         ]
 
-    # The SOLAR_EXPORT-below-max candidate holds soe exactly unchanged, so it
-    # is feasible at every state, and the import-cap filter above cannot empty
-    # a non-empty list (its threshold is floored at the minimum grid_imported
-    # any candidate achieves, so that candidate always survives) --
-    # `candidates` is never empty and an IndexError below would be a real bug,
-    # not a case to defend against.
+    # Plain IDLE is offered unconditionally and holds soe within bounds at
+    # every state, so it is always feasible, and the import-cap filter above
+    # cannot empty a non-empty list (its threshold is floored at the minimum
+    # grid_imported any candidate achieves, so that candidate always
+    # survives) -- `candidates` is never empty and an IndexError below would
+    # be a real bug, not a case to defend against.
+    #
+    # This used to name the SOLAR_EXPORT-below-max candidate instead. That
+    # stopped being the guarantee when #630 made the bypass conditional; IDLE
+    # is the unconditional one, and always was.
     argmax_index = 0
     best_value = float("-inf")
     for index, candidate in enumerate(candidates):

core/bess/dp_battery_algorithm.py

@@ -1243,7 +1243,10 @@ def _run_dynamic_programming(
     # for the reward/transition primitives, so a top-level import would be
     # circular -- the same arrangement pwl_window_dp already has with this
     # file.
-    from core.bess.action_selector import _residual_cover_p
+    from core.bess.action_selector import (
+        _residual_cover_p,
+        _solar_export_bypass_is_unexecutable,
+    )
 
     # Set defaults if not provided
     if solar_production is None:
@@ -1385,27 +1388,40 @@ def _run_dynamic_programming(
         # to preserve headroom for above-cap solar: ac_flows_grid caps the
         # exported surplus and the DP weighs the clipped remainder against
         # the value of keeping the room (no separate HOLD action needed).
-        zeros_col = np.zeros_like(soe_col)
-        reward_bypass, grid_imported_bypass = _compute_reward_grid(
-            zeros_col,
-            soe_col,
-            soe_col,
-            home_consumption=home_consumption[t],
-            battery_settings=battery_settings,
-            dt=dt,
-            current_buy_price=buy_price[t],
-            current_sell_price=sell_price[t],
-            solar_production=solar_production[t],
-            import_cap_kwh=import_cap_kwh,
-        )
-        value_bypass = reward_bypass.reshape(-1) + V[t + 1][np.arange(n_states)]
-        if effective_import_cap is not None:
-            bypass_feasible = (
-                grid_imported_bypass.reshape(-1)
-                <= effective_import_cap.reshape(-1) + 1e-9
+        #
+        # Withheld entirely where the classifier would call the period IDLE
+        # rather than SOLAR_EXPORT (#630) -- there the bypass is not a
+        # commandable action at all, and plain IDLE is what the hardware
+        # does. Unlike the discharge mask above, this one DOES apply here:
+        # that exception rests on the coarse lattice having no exact-cover
+        # point nearby, so an in-band action is the better proxy. Nothing
+        # analogous holds here -- the bypass is an exact action, not a
+        # lattice approximation, and its executable neighbour (plain IDLE)
+        # is already in the action set at every state.
+        if not _solar_export_bypass_is_unexecutable(
+            solar_production[t], home_consumption[t], battery_settings, dt
+        ):
+            zeros_col = np.zeros_like(soe_col)
+            reward_bypass, grid_imported_bypass = _compute_reward_grid(
+                zeros_col,
+                soe_col,
+                soe_col,
+                home_consumption=home_consumption[t],
+                battery_settings=battery_settings,
+                dt=dt,
+                current_buy_price=buy_price[t],
+                current_sell_price=sell_price[t],
+                solar_production=solar_production[t],
+                import_cap_kwh=import_cap_kwh,
             )
-            value_bypass = np.where(bypass_feasible, value_bypass, -np.inf)
-        V[t, :] = np.maximum(V[t, :], value_bypass)
+            value_bypass = reward_bypass.reshape(-1) + V[t + 1][np.arange(n_states)]
+            if effective_import_cap is not None:
+                bypass_feasible = (
+                    grid_imported_bypass.reshape(-1)
+                    <= effective_import_cap.reshape(-1) + 1e-9
+                )
+                value_bypass = np.where(bypass_feasible, value_bypass, -np.inf)
+            V[t, :] = np.maximum(V[t, :], value_bypass)
 
         # Residual load-cover candidate (#466 follow-up): one extra
         # O(n_states) column discharging exactly this period's forecast net

core/bess/pwl_window_dp.py

@@ -17,6 +17,7 @@
     PeriodInputs,
     _discharge_is_unexecutable,
     _residual_cover_p,
+    _solar_export_bypass_is_unexecutable,
     select_action,
 )
 from core.bess.dp_battery_algorithm import (
@@ -258,6 +259,17 @@ def _pwl_candidate_values_at(
     # delta -> battery_charged=0, so grid_exported reflects the full
     # surplus). With the AC cap set, this candidate is also what defers
     # charging to preserve headroom for above-cap solar.
+    #
+    # Withheld where the classifier would call the period IDLE rather than
+    # SOLAR_EXPORT (#630): nothing commands the hold there, so it is not an
+    # action this pass may value. Plain IDLE (power=0, already in the main
+    # grid above) is what the hardware does instead, which is why dropping
+    # the column cannot leave a row without a finite action.
+    if _solar_export_bypass_is_unexecutable(
+        solar_production[t], home_consumption[t], battery_settings, dt
+    ):
+        return value.max(axis=1)
+
     zeros_col = np.zeros_like(soe_col)
     reward_bypass, grid_imported_bypass = _compute_reward_grid(
         zeros_col,

core/bess/tests/integration/test_plan_faithfulness.py

@@ -229,6 +229,77 @@ def test_realized_matches_planned_across_all_fixtures():
     )
 
 
+def test_subfloor_solar_export_is_never_planned():
+    """Regression for issue #630.
+
+    The DP offers a "hold SoE, export this period's own solar surplus"
+    candidate (the SOLAR_EXPORT-below-max bypass, #313). It is only executable
+    where `classify_strategic_intent` will actually label the period
+    SOLAR_EXPORT, because that intent is what writes charge rate 0 and stops
+    the inverter absorbing the surplus. Below the classifier's
+    `FLOW_NOISE_FLOOR_KWH`, the period falls through to IDLE instead, whose
+    command is `load_first` at charge rate 100 -- so the plan books export
+    revenue on energy the hardware puts in the battery. Same shape as #282 on
+    the discharge side, which `_residual_cover_p` already gates against.
+
+    The scenario is built to make the bypass the DP's honest choice, so the
+    plan is wrong for a real economic reason rather than by accident:
+
+    - period 0's surplus is 0.0099 kWh, just under the 0.01 kWh floor
+    - periods 1-2 carry enough surplus to fill the battery to `max_soe`
+      regardless, so the marginal value of storing period 0's surplus is only
+      the export it displaces later (sell 0.2), not its evening value
+    - selling it now at 0.5 therefore beats storing it, by more than the
+      cycle cost
+    - a larger discharge-and-refill at period 0 is unprofitable
+      (0.5 sell - 0.4 cycle - 0.2 displaced export < 0), so the DP has no
+      reason to reach for a discharge candidate instead
+
+    `terminal_value_per_kwh` is pinned to 0.0 so the leftover-SoE term cannot
+    quietly change which candidate wins as prices are tuned.
+
+    Measured on the pre-fix code: R=2.404206 P=2.401236, gap +0.002970 SEK --
+    three times `PLAN_EXECUTION_TOLERANCE_SEK`.
+    """
+    from core.bess.tests.helpers import run_scenario_realized
+
+    scenario = {
+        "buy_price": [1.2, 1.2, 1.2, 3.0, 3.0, 3.0],
+        "sell_price": [0.5, 0.2, 0.2, 0.2, 0.2, 0.2],
+        "solar_production": [0.5099, 2.0, 2.0, 0.0, 0.0, 0.0],
+        "home_consumption": [0.5, 0.5, 0.5, 6.0, 6.0, 6.0],
+        "terminal_value_per_kwh": 0.0,
+        "battery": {
+            "max_soe_kwh": 20.0,
+            "min_soe_kwh": 2.2,
+            "max_charge_power_kw": 10.0,
+            "max_discharge_power_kw": 10.0,
+            "efficiency_charge": 0.97,
+            "efficiency_discharge": 0.97,
+            "cycle_cost_per_kwh": 0.40,
+            "initial_soe": 19.0,
+        },
+    }
+
+    result, realized = run_scenario_realized(scenario)
+    planned = result.economic_summary.battery_solar_cost
+
+    # The outcome that matters: executing this plan costs what it said it would.
+    assert abs(realized - planned) <= PLAN_EXECUTION_TOLERANCE_SEK, (
+        f"plan books a cost its own execution does not reproduce: "
+        f"R={realized:.6f} P={planned:.6f} gap={realized - planned:+.6f}"
+    )
+
+    # The mechanism, so a future regression is diagnosable and not just red:
+    # period 0 must absorb its sub-floor surplus, because that is what the
+    # command derived for it does.
+    p0 = result.period_data[0].energy
+    assert p0.grid_exported == pytest.approx(0.0, abs=1e-9), (
+        f"period 0 plans a {p0.grid_exported:.5f} kWh export below the "
+        f"classifier's noise floor -- the derived command absorbs it instead"
+    )
+
+
 def _battery(initial_soe):
     return {
         "max_soe_kwh": 20.0,

docs/agents/bess-knowledge.md

@@ -502,6 +502,17 @@ battery has room.  With the cap set:
   the DP chooses it ahead of the above-cap window.  These periods classify as
   **SOLAR_EXPORT** with `battery_action = 0` and a not-full battery, meaning
   "deliberately holding for later overflow".
+- **The bypass is only offered where that classification actually lands
+  (#630)**: nothing commands "hold and export" directly — it is delivered by
+  the SOLAR_EXPORT label writing `charge_rate=0`.  That label needs
+  `grid_exported > 0.01 kWh` (`FLOW_NOISE_FLOOR_KWH`); below it the period
+  falls through to **IDLE**, whose command is `load_first` at charge rate
+  100, which absorbs the surplus instead.  Planning the export anyway meant
+  the battery ran fuller than planned until it hit a bound and spilled the
+  difference.  `_solar_export_bypass_is_unexecutable` in `action_selector.py`
+  withholds the candidate there, so a sub-floor surplus is planned as
+  absorbed — which is what the hardware does.  Charge-side twin of
+  `_residual_cover_p`'s LOAD_SUPPORT gate, on the same constant.
 - **Hardware mapping**: SOLAR_EXPORT blocks passive charging (#313), stopping
   `load_first` from filling the battery from surplus solar. On a genuinely
   full battery this is a no-op. The mechanism differs by platform: register-