feature: computed overnight reserve floor (Min SOC + historical night use) so one knob covers evening export + night load

[Frank-Leysen]

Context

Live Growatt MOD 5000TL3-XH / 15 kWh / Belgium Dynamic (Belpex H + Luminus), tester @Frank-Leysen. Related discussion lived on #126; opening a dedicated issue per your request.

Today BESS has a single Min SOC knob. In practice we need two different floors:

1. Hard floor (Min SOC) — never go below this (hardware / safety / end of overnight).
2. Evening export / midnight planning floor — how low evening BATTERY_EXPORT may go so enough energy remains for overnight house load until morning PV covers consumption.

With only one knob we keep hitting a bad trade-off (reported on #126, e.g. 17 Jul):

Min SOC setting	What happens
Low (e.g. 12–15%)	Strong evening export, but overnight can run too thin → avoidable grid import on Belgian buy≈0.35 vs sell≈0.12–0.16
High (e.g. 47%)	Night reserve OK, but DP often holds far above Min SOC through the peak (we saw ~74% flat) → evening arbitrage largely wasted

So: one Min SOC cannot express "export hard into the evening peak, but keep X% reserved for overnight load support until solar returns."

Observed (today's numbers as example)

• Battery: 15 kWh
• User Min SOC: 12% → 1.8 kWh hard floor
• Overnight battery use (last night → this morning, until PV starts covering load): ~2.8 kWh
• Implied planning reserve after evening export: 1.8 + 2.8 = 4.6 kWh ≈ 30.7% SOC

Separately, before tomorrow's day-ahead prices land (~14:00), the dashboard Net Grid Cost can look overly optimistic because the plan still assumes export down toward Min SOC; after prices, midnight SOC often rises (overnight reserve) and the headline shrinks. A visible computed floor would make that behaviour easier to understand even when the horizon is incomplete.

Desired

Keep one user setting: Min SOC (hard floor).

Have BESS compute an overnight planning floor from history, e.g. last 7 nights:

planning_floor_kWh = min_soc_kWh + expected_overnight_battery_use_kWh (+ optional safety margin)
planning_floor_soc = planning_floor_kWh / capacity

Use planning_floor_soc in:

• evening BATTERY_EXPORT stop / midnight target
• schedule / Savings / Net Grid Cost so the UI does not imply "export all the way to Min SOC" when overnight reserve is required

Important nuance: overnight the battery may still discharge from planning_floor down to Min SOC for house load. The computed value is not a second hardware Min SOC (writing ~31% to the inverter would stop discharge too early and force expensive night import).

expected_overnight_battery_use ideally = energy needed until net PV covers house load (not a fixed clock). Prefer median or P75 of recent nights + a small configurable margin, so boiler/EV/heatwave outliers do not dominate.

Proposal — question for Johan

Does this design make sense to you as a product approach?

We are not asking to replace the DP with rules. The idea is a transparent constraint / terminal target derived from Min SOC + recent overnight reality, so:

• users keep a single Min SOC
• graphs and economics immediately respect the real overnight buffer
• we avoid the manual Min SOC yo-yo (15% vs 47%) that fights the optimiser

If the DP already intends to hold this reserve via the load forecast: great — then the ask is mainly to surface that computed floor and to guarantee it when the forecast / pre-14:00 horizon underestimates overnight need (without double-counting on top of a correct load model).

Happy to stay guinea pig on Growatt + Belpex H and supply debug bundles for any prototype (e.g. 7-day median overnight kWh + margin %).

Thanks — curious what you think of the approach.

[johanzander]

Hi Frank — thanks for the detailed writeup, and for the sustained testing on this.

Before I design a new floor/reserve concept, I want to make sure we've actually pinned down the problem, since this is a case where it's easy to jump to a solution shape (a second SOC knob) before confirming the mechanism it's meant to fix. I'm happy to add the feature if the evidence supports it — I just want to be certain first.

What I found checking your two most recent "good progress" logs (b21, 21 Jul and b22, 22 Jul):

In both nights, overnight grid import did happen, but not the way the "one knob can't do two jobs" framing assumes:

Night	Evening export drained SOE	Overnight imports	SOE at each import (floor = 2.25 kWh)	Cost
20→21 Jul	15.0→6.1 kWh	0.20 + 0.10 + 0.10 kWh	5.4, 3.45, 3.30 kWh	~0.15 SEK
21→22 Jul	15.0→7.8 kWh	0.20 + 0.20 + 0.20 kWh	5.7, 3.6, 3.45 kWh	~0.22 SEK

Every import happened with 3.3-5.7 kWh of headroom still above the floor — the battery was nowhere near empty. The imports are small (0.1-0.2 kWh), land at roughly the same clock times each night (~00:15, ~02:00, ~05:30-07:15) regardless of how much was exported that evening, and cost about 0.15-0.22 SEK/night total. At the point SOE actually gets closest to the floor (~06:00), import is 0.00 in both nights — solar/low load carries it through fine.

That doesn't match "sold cheap in the evening, forced to rebuy expensive later because the overnight estimate was too low." If that were the mechanism, we'd expect the import to show up right as SOE approaches the floor, sized to whatever the forecast missed by. Instead it looks like brief demand spikes inside a single 15-minute period that exceed whatever discharge rate was scheduled for that period — while there's still plenty of energy sitting in the battery to have covered it.

That's a different problem, and a computed floor wouldn't fix it. A floor changes how much energy is available; it does nothing about whether the battery can respond fast enough within a period to a load spike that shows up between period boundaries. If the mechanism is "the schedule commits to a power level for 15 minutes and a spike inside that window outruns it," then raising the reserve just means there's more energy sitting there uselessly — the same spike still isn't served in time.

That's what makes me think this might actually be the same class of problem as #375 (VPP closed-loop grid-following regulator) — a sub-period trim that reacts to live grid flow between period boundaries rather than waiting for the next 15-minute schedule update. I know #375 is scoped to EMS/VPP-controlled platforms and you're on Growatt TOU/cloud-schedule, so it may not apply to your setup directly as designed — but the underlying gap (schedule is right on average, individual moments inside a period aren't) looks like the same root cause either way.

Before we scope anything: can you help me confirm this reading? If you can grab a debug export from the moment one of these small overnight imports happens (not a summary after the fact), that'd let us check the actual load/discharge-rate values in that exact 15-minute window and confirm whether it's a within-period response gap rather than a which-forecast-was-used gap.

[johanzander]

Correction to my numbers above: the currency is EUR, not SEK — I mislabeled it. So the overnight import costs were ~€0.15 (20→21 Jul) and ~€0.22 (21→22 Jul), not SEK. The kWh figures and the argument are unchanged.

[johanzander]

Correction and update on the diagnosis above.

Withdrawing the #375 reference — that's specifically about a VPP power-setpoint closed loop, and you're on solax_modbus_growatt_min in TOU mode, not VPP mode. It doesn't apply here, and wouldn't have solved this even if it did.

The actual mechanism, confirmed against your b21/b22 logs: LOAD_SUPPORT and BATTERY_EXPORT currently share one code path (_map_intent_to_rates) that caps the discharge-rate register at the DP's planned average power for the 15-minute period, not at what the load might spike to. For BATTERY_EXPORT that's correct — it has no natural backstop, so an open ceiling would just oversell beyond the arbitrage plan. But LOAD_SUPPORT is different: it runs through load_first mode, which is already self-limiting to the real house deficit — opening its ceiling can never cause it to export or overshoot beyond actual need, only to spend reserve covering a real spike. The two intents don't need the same cap, and right now they get one.

Worth noting: capping LOAD_SUPPORT to the plan average isn't an oversight — it's the fix for a previous bug (#147), where a hardcoded 100% rate caused the battery to drain early on high-consumption days and blow past reserve planned for a later, pricier period. So the fix here isn't "just open it back up" — that regresses #147. What's missing is a way to open it up only when there's genuine unneeded headroom, which is exactly your situation (3+ kWh above the floor with nothing later in the horizon needing it).

There's already a precedent for this kind of decision in the codebase: the SOLAR_EXPORT shadow-price gate, which opens extra discharge only when the DP's own shadow price says the marginal stored kWh isn't worth reserving. We're going to build the equivalent gate for LOAD_SUPPORT — same idea, applied to this intent — so it can serve a real load spike from reserve precisely when the reserve isn't earmarked for anything, and stay capped when it is. Looking into a fix now.

[Frank-Leysen]

Hi Johan — thanks for digging into the logs and for the corrections.

Agreed with your reading. Looking back at our live nights (including the recurring boiler reheat pattern we reported on #352), the small overnight imports consistently happened with SOC well above Min SOC and EMS discharge capped below house load — i.e. a within-period response / rate-cap issue, not "evening export drained the overnight estimate too low."

So yes: a computed overnight floor would not have fixed those imports. Happy to park #381 as a product idea until/unless we see a real "sold cheap → forced expensive rebuy near the floor" case with evidence.

On the LOAD_SUPPORT shadow-price gate: that matches what we've been seeing. Looking forward to testing it on Growatt TOU (solax_modbus / Grid First–Load First). We'll watch that it doesn't regress the #147 early-drain behaviour on high-consumption days.

Debug at the moment of import: next time one of these small overnight imports shows up live, I'll grab a debug export immediately (plus dashboard screenshot) rather than a later summary, so you can see load vs planned discharge rate in that exact 15-minute window. Will post it here and/or on #352 as appropriate.

Thanks again for the careful diagnosis.

[Frank-Leysen]

Live example — morning of 29 Jul 2026 (9.9.0b26): evening BATTERY_EXPORT left too little overnight reserve

Growatt MOD 5000TL3-XH / 15 kWh / Luminus Dynamic (Belpex H).

Yesterday (28 Jul) — good day economically, but aggressive evening export

User totals: Import 0.8 kWh @ 0.29 EUR · Export −12.9 kWh @ −2.12 EUR · Net −12.1 kWh @ −1.83 EUR.

From debug schedule (28 Jul evening):

Time	Intent	SOE (kWh)
19:45	SOLAR_EXPORT → full	15.0 (100%)
20:00–21:30	BATTERY_EXPORT	15.0 → 5.7
21:45–23:45	LOAD_SUPPORT	5.7 → ~4.3 at midnight

So ~9.3 kWh left the battery to the grid in ~1.5 h of evening export, then overnight load support continued from a thin reserve.

This morning (~05:00) — floor hit while still dark

SOC / SOE	11% / 1.6 kWh
BESS Min SOC / min_soe	12% / 1.8 kWh (reserved_capacity: 1.8)
Hardware discharge stop	12%
Strategic intent	Standby / IDLE
EMS discharge rate	0%
Home load	~570 W
Battery	~250 W residual
Grid import	~330 W @ ~0.37 €/kWh

Overnight trajectory (29 Jul actuals):

00:00  LOAD_SUPPORT  4.3 → 4.1 kWh   (~29% at midnight)
…
03:00  IDLE          2.6 → 2.4
04:30  IDLE          2.0 → 1.8       (at min_soe)
04:45  IDLE          1.8 → 1.6       (below floor → Standby + import)

Import here is expected once the floor is hit — not the #352/#393 “ample SOC but discharge gated” pattern. The planning question is whether evening export should have stopped higher so LOAD_SUPPORT could cover until morning PV (~06:00+).

Why this belongs on #381

With a single Min SOC knob (12%), BESS correctly treats that as the hard floor and exports down toward it when evening sell prices are attractive. What we lack is a separate overnight reserve / midnight planning floor (e.g. Min SOC + expected night kWh until PV covers load), so evening BATTERY_EXPORT does not consume the energy still needed for the dark hours.

Rough need for this night: midnight → ~06:00 at ~0.4–0.6 kW house ≈ 2.5–3.5 kWh above the 1.8 kWh floor → target midnight SOE closer to ~4.5–5.5 kWh (~30–37%), not ~4.3 kWh with continued drawdown to 1.6 before dawn.

Evidence

Debug export (05:04 CEST): https://gist.github.com/Frank-Leysen/85e2ace2dd6ebc5a859c0161d30c850d

Happy to add screenshots if useful. No ask to change #352 for this — different mechanism (reserve sizing, not discharge-rate gate).

[johanzander]

Thanks for the live example and the fast debug export — this let us actually
pin down the mechanism instead of guessing.

Conclusion: no algorithm defect, and this doesn't change our "park the
floor idea" decision. Root cause is a one-night consumption forecast miss,
not a bad decision or a control problem. Breaking it down into the three
things that can look identical from the outside but have very different
implications:

1. Was the 20:00 export decision optimal given what the DP knew at the
   time? Yes. Given its own forecast (~0.4 kW / 0.1 kWh per 15-min
   overnight), draining toward the floor was the correct economic call —
   night sell price was well below buy price and nothing later in the
   horizon needed the reserve. No defect in the shadow-price/floor logic.
2. Was the forecast itself wrong? Yes — this is the actual root cause.
   Planned overnight consumption was 2.0 kWh (00:00–04:45); actual metered
   consumption was 2.49 kWh, a +25% miss. That ~0.5 kWh gap at the morning
   buy price accounts for essentially the entire ~€0.15–0.20 variance.
3. Was there an execution/control problem (inverter not delivering the
   commanded rate)? No. LOAD_SUPPORT runs load-following (100% discharge,
   load_first) with no fixed rate to violate — it tracked real load almost
   exactly the whole night. The earlier fixed-rate BATTERY_EXPORT segment
   that evening also matched its plan almost exactly, confirming the
   inverter executed commands faithfully. Floor protection did exactly what
   it should once SOE dropped below the 1.8 kWh floor — it flipped to IDLE
   at the next re-plan.

So: correct algorithm, correct control, and a real but ordinary consumption
forecast miss for that specific night — not a systematic gap. At this
magnitude (~€0.15–0.20 on a night that still cleared several euros of
savings), it doesn't justify a second reserve knob, and it isn't the same
class of issue as the earlier b21/b22 within-period rate-cap cases either.

One idea did surface from this: since underestimating overnight consumption
is more costly than overestimating it (a missed forecast near the floor
risks an expensive import, while an overly conservative forecast just delays
using that energy a bit), there may be a case for planning the no-PV
overnight tail against a slightly conservative consumption estimate rather
than the mean forecast. That's a real potential improvement, but it needs
data across many nights/users to know if it's worth building — posted as
an idea for later: #418.

Appreciate you sticking with this through several rounds of diagnosis —
it's a good example of "sold cheap, needed later" needing hard evidence
before jumping to a design change.