bug: discharge gate compares an undiscounted shadow price against buy*eta -- 5.3% too strict, wrong on exact ties

[bess-product-owner]

Found while diagnosing @Frank-Leysen's 17 Aug debug bundle against #520. This is a third defect at the same seam as #571 and #579, and unlike those it is still live on main.

Summary

intra_period_discharge_gate tests

buy_price_t * battery_settings.efficiency_discharge >= shadow_price

(core/bess/dp_battery_algorithm.py:958-960). That is correct only if shadow_price is denominated per kWh of SoE. In the discharge-limited regime it is not — it comes back undiscounted, i.e. per kWh delivered. The two sides of the comparison are therefore in different units, and the gate is a factor 1/η too strict: it demands the current price beat the competing future price by 5.3% before the battery may cover a load spike, and it is wrong on exact ties.

Root cause

SOE_STEP_KWH = 0.025 and POWER_STEP_KW = 0.1 (core/bess/dp_constants.py:62,65), so the SoE grid step is exactly POWER_STEP_KW × 0.25h — deliberate, per the comment at line 29. The SoE→delivery conversion carries η (dp_battery_algorithm.py:599, :1289).

So the value function is a staircase whose riser is one full delivery step, flat once every 1/(1-η) = 20 cells. _value_slope_below (dp_battery_algorithm.py:1562-1587) takes a one-cell backward difference, so it lands on a riser 19 times out of 20 and reports the undiscounted price.

Measured on a clean single-period synthetic (buy 0.40, η 0.95, floor 1.8):

slopes i=1..45: [0.0, 0.4 x18, 0.0, 0.4 x19, 0.0, 0.4 x5]
flat cells at i = [1, 20, 40]      # exactly every 20, as 1/(1-0.95) predicts
mean slope       0.38 = eta x buy  # the TRUE per-kWh-of-SoE value
one-cell reading 0.40 = buy        # what the gate actually receives

With η = 0.5 the same probe gives [0.0, 0.4, 0.0, 0.4, ...] — every other cell, averaging 0.20 = η × buy. The bias is structural, not noise.

Why η should cancel entirely

Covering ΔE from the battery consumes ΔE/η of SoE. That SoE would later have delivered ΔE anyway, so the opportunity cost is ΔE · p_future and the correct test is simply

buy_now >= p_future

with η cancelling on both sides. The implemented form applies η to the buy side while the shadow side is already undiscounted, which is the 5.3% over-strictness.

Evidence from the bundle

Debug export: https://gist.github.com/Frank-Leysen/dead2ff1a5954a913ecfabb13534eee0 (10.1.0b9, Growatt MOD 5000TL3-XH, EUR).

Replayed from Prediction Snapshots, the 02:00 run splits its gate-closed periods into two regimes:

p8-p19 : shadow 0.3757/0.3814  vs buy*eff 0.3673/0.3624  -> closed on a real forward differential
p25-p33: shadow == buy_t EXACTLY (0.42279, 0.43114, 0.43518) -> closed purely by the 0.95 factor
p20-p24: gate True (buy*eff 0.3787 > shadow 0.3757)
p35+   : shadow 0.0, gate False (bottom-of-grid, #526 "absence is not permission" -- correct)

Periods 28–33 are 07:00–08:30, the reporter's boiler window. There the marginal kWh's best future use is this very period, so shadow == buy_t bit-exactly, and buy_t × 0.95 >= buy_t is false by construction. The correct comparison is an exact tie and >= is inclusive — the gate should have opened.

Confirmed reproducible on current main: replaying the 02:00 run's exact horizon gives shadow=0.3756876, gate=False, action=-0.125, bit-identical to the b9 bundle. So this is not a b9 artifact and #579 does not fix it.

Scope of the failure

Not globally broken. Across all 35 runs in the bundle the gate is open in 1179/1476 (80%) of LOAD_SUPPORT periods. The failure is confined to the overnight, fully-committed, discharge-limited regime — which is exactly where users notice it, because it is the regime where the battery visibly holds charge while the house imports. In this bundle it closed for 16 consecutive periods, 04:00–07:45.

Cost

Small, and filed as correctness rather than money, like #571. Valuing each closed gate against the corrected test: p8 loses 0.0011 EUR; p22 and p29/p30 are exact ties worth 0.000. Night total ~0.1 eurocent. The battery ended at 1.95 kWh against a 1.80 floor, so most of the night's 0.523 kWh import was unavoidable and the DP's rationing was directionally right as prices rose 0.3867 → 0.4352.

A hardware command should not depend on a unit mismatch the user cannot observe, regardless of what it costs — and the same seam can bite harder where the value gap is wider.

Why this blocks the #520 VPP half

The VPP design (docs/superpowers/specs/2026-08-23-vpp-load-tracking-design.md) maps the gate's decision rather than its rate, so it would inherit this biased authorization verbatim — VPP would faithfully hold in exactly the cases where the gate is wrong to close. Fixing this first keeps the bias from reaching a second platform.

Not to be confused with

• bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571 / fix: price the discharge gate off the cell the battery is actually in #579 — np.round snapping making V locally non-concave / pricing off the cell below. Fixed, shipped in v10.1.0. Different defect, same seam; it does not address the units mismatch.
• shadow_price == 0.0 is ambiguous between "worth nothing" and "never computed", and the discharge gate opens unconditionally there #526 — shadow_price == 0.0 at bottom-of-grid. Correctly handled; p35+ in this bundle show shadow 0.0, gate False as intended.

Refs #520, #524, #526, #571, #579, #384, #393, #147.

[bess-agent]

Implementation started; branch fix/issue-683-gate-units (local worktree, unpushed).

Fix and RED test are done and green. _value_of_delivering_below prices the delivery rather than the cell — delivering SOE_STEP_KWH costs SOE_STEP_KWH/eta of SoE, so the interpolant is read across exactly the span the discharge consumes and the staircase averages out by construction instead of 19 times in 20. The gate becomes buy_price_t >= shadow_price, both sides now per delivered kWh.

Behaviour changes only in the staircase regime. Where V is smooth the new estimator returns old/eta, and buy >= shadow is algebraically identical to the old buy * eta >= shadow — so the decision is unchanged there. That is why only four tests moved rather than half the suite.

Evidence the test discriminates (RED verified before the fix): test_gate_opens_when_buying_costs_at_least_the_marginal_value failed on all three fixtures — 63 / 23 / 20 wrongly-closed periods. The middle is the reporter's own 13 Aug bundle, where every closed period was of this kind.

Not yet done, deliberately handed to a fresh session:

• four pre-existing tests still encode the old units (one is bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571's within-cell constancy guard, which this fix breaks on purpose — measured behaviour is now continuous and monotone, where bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571's defect was a discontinuous mid-cell step; owner approved re-pinning it to continuity + monotonicity)
• corpus gate-flip count and cost delta measured, goldens re-pinned deliberately
• slow suite, code review, local verification, docs check (shadow_price's denomination changed and is surfaced in the debug export), CHANGELOG

One thing for whoever picks this up: a fixture generated from the 17 Aug bundle does not reproduce the defect (0 gate-closed periods, 0 ties) because that export starts at 08:37 heading into a solar day. It was removed rather than kept as decoration. The evidence lives in the three corpus fixtures that do reach the branch.

[bess-agent]

Resuming implementation from step 5 (four pre-existing tests still encoding the old units).