Exact piecewise-linear value-function propagation for the DP backward induction (Approach 2 of #276)

[johanzander]

Summary

Follow-up from #276 (see design doc docs/superpowers/specs/2026-07-12-dp-continuous-action-reformulation-design.md) and #282 (Approach 1, merged/pending as PR #284). Approach 1 (hardware-aware breakpoint enumeration in Step 2) closed the Step-2-vs-V mismatch, but directly tested against #275's own "Worse" reproduction scenario, it made zero measurable difference — the SOE held past midnight was bit-identical to Option B alone. Root cause: Step 1's value function V (computed once by backward induction, on a fixed SOE_STEP_KWH grid, then linearly interpolated) itself carries approximation error that no amount of exact Step 2 search against that same V can recover from.

This issue is Approach 2: rework _run_dynamic_programming's backward induction to track V as an exact piecewise-linear function of SoE — a small number of genuine (soe, value, slope) breakpoints derived recursively from the next period's breakpoints plus that period's own reward breakpoints — instead of a sampled grid. If V itself has no approximation error, there's no gap left for Step 2 to inherit, regardless of search method.

Why this is needed (not just Approach 1 again)

#282's postmortem: even with an exact Step 2 search, using the same (grid-approximated) V, the DP still holds the same amount of charge on #275's defining reproduction. This confirms the bottleneck is V itself, not Step 2's fidelity to it — exactly what the original design doc predicted Approach 1 alone might not fully close.

Scope

• Prototype the backward recursion: represent V[t, :] as a list of breakpoints instead of a SOE_STEP_KWH-spaced array.
• Derive V[t, :]'s breakpoints from V[t+1, :]'s breakpoints plus the period's own reward breakpoints (physical charge/discharge bounds, the BATTERY_EXPORT_THRESHOLD_KWH self-throttle crossing) — reusing Replace DP per-period grid search with closed-form breakpoint enumeration (Approach 1 of #276) #282's hardware-percent-grid action space so any implementation stays R==P-safe by construction, not just accurate.
• Measure whether the number of genuine breakpoints (not grid artifacts) stays small enough for this to be tractable at production horizons (96–192 periods), especially combined with perf: vectorize DP backward-induction hot loop in dp_battery_algorithm.py #236's vectorization.
• Re-run DP holds charge for a known-worse tomorrow price instead of a known-better today price (real 2-day horizon branch) #275's own "Worse" reproduction scenario as the acceptance test: does the SOE held past midnight actually drop toward the floor now?

Out of scope for the initial investigation

• Implementation — start with a design doc + prototype validating feasibility and measuring the real effect on DP holds charge for a known-worse tomorrow price instead of a known-better today price (real 2-day horizon branch) #275's reproduction, per project convention (see 2026-07-06 and 2026-07-12 docs), before committing to a full rewrite.

References

• Investigate continuous-action/exact-piecewise-linear reformulation of the DP to eliminate discretization interpolation error #276 — original investigation issue (Approach 1/2 options)
• Replace DP per-period grid search with closed-form breakpoint enumeration (Approach 1 of #276) #282 / PR fix: hardware-aware breakpoint enumeration for DP discharge/charge search (#282) #284 — Approach 1, does not resolve DP holds charge for a known-worse tomorrow price instead of a known-better today price (real 2-day horizon branch) #275
• DP holds charge for a known-worse tomorrow price instead of a known-better today price (real 2-day horizon branch) #275 — original reported symptom (Frank, Belpex H integration? #126), still open
• docs/superpowers/specs/2026-07-12-dp-continuous-action-reformulation-design.md — original options analysis

[johanzander]

Investigation paused — evidence suggests this isn't needed

Before building the exact piecewise-linear V machinery, ran a cheap direct test: swept SOE_STEP_KWH from 0.05 down to 0.0025 (20x finer, effectively approaching what an exact V would give) against #275's own "Worse" reproduction. The held-charge behavior this issue was meant to fix doesn't change with grid resolution at all — ruling out discretization/interpolation error as the actual cause.

A deeper trace then showed the reported symptom itself doesn't hold up: the overnight reserve drains to within 0.003 kWh of the floor before solar arrives each day; the later "export at a worse price" is fresh next-day solar with no earlier selling opportunity, not the carried-over charge. Full writeup: #275 (comment)

Status: pausing this investigation. Approach 2 would be significant engineering effort (a genuine backward-induction redesign — the exact piecewise-linear envelope prototype started here already hit real implementation complexity even on a 4-period toy example) for a mechanism that doesn't appear to be the actual root cause. Revisit only if #275 is re-confirmed as a real bug through a different mechanism.

[johanzander]

Correction and final status: not needed, closing

My earlier comment on this issue said the investigation was being "paused pending re-diagnosis." That undersold the actual conclusion — corrected picture, full detail at #275 (comment):

The real discretization defect #275 reported was already fixed in v9.9.0b13 (Option B, PR #279) — a genuine, verified 27% reduction in excess held charge. This issue (Approach 2: exact piecewise-linear value-function tracking) was scoped to eliminate what was assumed to be remaining discretization error. Direct testing shows there isn't any: a 20x finer grid doesn't change the residual, and the residual has been directly verified (Bellman-optimality check + realized-cost financial comparison against the "hold less" alternative) to be the financially optimal amount, not a defect.

Building the exact-V machinery this issue scoped would not improve on the current, already-optimal behavior. Recommend closing as not needed.

[johanzander]

Closing — not needed. Direct investigation (PR #286) shows the residual behavior this issue aimed to eliminate is already financially optimal, not discretization error. See #275 for the full proof.