Skip to content

fix: model Growatt hardware solar behavior in DP optimizer - #71

Closed
pookey wants to merge 1 commit into
johanzander:mainfrom
pookey:feat/hardware-mode-solar-modeling
Closed

fix: model Growatt hardware solar behavior in DP optimizer#71
pookey wants to merge 1 commit into
johanzander:mainfrom
pookey:feat/hardware-mode-solar-modeling

Conversation

@pookey

@pookey pookey commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Summary

The DP optimizer's energy model did not match how the Growatt inverter actually handles solar in its operating modes, causing incorrect schedules:

  • battery_first mode: Hardware absorbs ALL excess solar, but the DP only modeled the requested charge power — allowing it to plan impossible cross-temporal arbitrage (export solar cheap, buy grid expensive)
  • Discharge override: When solar >= consumption, the Growatt overrides discharge commands and charges from excess solar instead. The DP had no knowledge of this and planned discharge during sunny periods
  • Dashboard mismatch: create_decision_data() derived intent and batteryAction from the raw DP power, ignoring hardware overrides — the dashboard showed "Powering Home" with phantom discharge during solar-covered periods

Changes

  • _state_transition, _compute_reward, _build_period_data: three-way branching (charge / effective-discharge / idle-or-override) matching actual Growatt hardware behavior
  • _compute_idle_solar_charging: shared helper for implicit solar charging (excess solar clamped by charge rate and available capacity)
  • _create_idle_schedule: propagates SOE with implicit solar charging instead of flat SOE
  • create_decision_data: derives strategic intent and battery_action_kwh from post-override energy flows (energy_data.battery_charged/battery_discharged) instead of raw DP power
  • Updated 5 scenario test expected values
  • Added test_battery_first_solar_charging.py (8 tests) and test_idle_solar_charging.py (16 tests)

Test plan

  • All 277 unit tests pass
  • All 50 integration tests pass (3 skipped)
  • Verify on dashboard: during SOLAR_STORAGE periods, excess solar goes to battery (not exported)
  • Verify on dashboard: during periods where solar covers load, no phantom discharge shown
  • Verify schedule bar shows correct mode labels (IDLE not LOAD_SUPPORT when solar covers load)

🤖 Generated with Claude Code

@pookey
pookey marked this pull request as draft April 16, 2026 11:33
@johanzander

johanzander commented Apr 16, 2026 via email

Copy link
Copy Markdown
Owner

Ian P. Christian @claude
fix: model Growatt hardware solar behavior in DP optimizer
aa6c622 
The DP optimizer's energy model did not match how the Growatt inverter
actually handles solar in its operating modes, causing three issues:

1. **battery_first mode ignored implicit solar charging**: When the DP
   selected a small charge action (e.g. 0.4 kW), it modeled only that
   amount going to the battery. But in battery_first mode, hardware
   absorbs ALL excess solar. This allowed the optimizer to plan
   cross-temporal arbitrage (export solar at low sell price, buy grid
   later at higher buy price) that could never happen on real hardware.

2. **Discharge override not modeled**: When solar >= consumption, the
   Growatt overrides any discharge command — excess solar charges the
   battery instead. The DP had no knowledge of this, planning discharge
   during sunny periods where it would never execute.

3. **Decision layer used raw DP power instead of effective flows**:
   `create_decision_data()` classified intent and battery_action from
   the raw DP power parameter, ignoring hardware overrides. The dashboard
   showed "Powering Home" with -1.2 kWh discharge during periods where
   solar covered the load and no discharge would occur.

Changes:
- `_state_transition`: charging branch uses max(dp_stored, solar_stored);
  discharge only effective when solar < consumption; else branch does
  implicit solar charging
- `_compute_reward` and `_build_period_data`: same three-way branching
  (charge/effective-discharge/idle-or-override) with consistent energy
  balance
- `_compute_idle_solar_charging`: extracted as shared helper for implicit
  solar charging calculation (excess solar, clamped by rate and capacity)
- `_create_idle_schedule`: now propagates SOE with implicit solar charging
- `create_decision_data`: intent derived from energy_data flows
  (battery_discharged/battery_charged) not raw power; battery_action_kwh
  derived from effective flows
- Updated 5 scenario test expected values
- Added test_battery_first_solar_charging.py (8 tests)
- Added test_idle_solar_charging.py (16 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@pookey
pookey force-pushed the feat/hardware-mode-solar-modeling branch from 98ccd1d to aa6c622 Compare April 17, 2026 11:58
@pookey pookey closed this Apr 20, 2026
ridax67 pushed a commit to ridax67/bess-manager that referenced this pull request Jul 15, 2026
@johanzander @claude
release: v9.9.0b14 (johanzander#71)
462e5df 
* Fix stale health-check banner and InfluxDB placeholder-config log spam (#217)

Two bugs from #201's debug bundle:

1. The dashboard health banner only refreshed at startup, after a
   settings save, or after the setup wizard — never periodically. A
   sensor blip at exactly one of those moments left the banner stuck
   showing an error indefinitely, even after sensors recovered, with
   no way for the user to clear it short of an unrelated settings save
   or a full restart. Added a public refresh_health_check() wrapper on
   BatterySystemManager, a 5-minute scheduler job, and a manual
   "Recheck now" button (POST /api/system-health/recheck) so the
   banner self-corrects without user guesswork.

2. get_sensor_data_batch/get_power_sensor_data_batch only checked for
   empty-string InfluxDB config, not the shipped placeholder
   credentials, so the periodic sensor-collection job kept attempting
   real HTTP connections to the default (unconfigured) InfluxDB URL
   every ~15 minutes for any user who never set it up. Both now use
   the existing is_influxdb_configured() check, which also gained a
   bucket-emptiness check it was missing.

Also converts two pre-existing call sites (settings-save refresh,
setup-wizard completion) from the private _run_health_check() to the
new public wrapper, for consistency with the no-private-cross-class-
calls pattern the wrapper itself establishes.

* docs: backfill missing changelog entries for #207, #211, #217 (#220)

These PRs shipped fixes but didn't touch CHANGELOG.md, leaving gaps
between what's documented and what actually merged since 9.8.1. Also
converts (#N) references to full markdown links, since plain #N text
only auto-links correctly when read within this same repo.

* fix: unify Battery/Home settings startup/PATCH paths, drop *_STORE_TO_API registries (#219) (#224)

BatterySettings.update()/HomeSettings.update() no longer translate camelCase
- snake_case (the settings store's native format) is now the canonical shape
for both the startup path and the PATCH /api/settings path, mirroring the
identical fix already applied to Price settings in #197/#216.

Root cause: the startup path filtered store fields through hand-maintained
BATTERY_STORE_TO_API/HOME_STORE_TO_API translation dicts before calling
update_settings(), while the PATCH handler already bypassed them and passed
the raw snake_case store dict directly - working only because
_camel_to_snake() was a no-op on already-snake_case keys. A field present in
the store but missing from the registry (e.g. charging_power_rate,
efficiency_charge, efficiency_discharge) was silently dropped at startup
while continuing to work via PATCH - the same bug class fixed for Price
in #197 after Belgian user #126's spot_multiplier revert-on-restart bug.

Code review (medium effort) surfaced and fixed three issues before this
was verified:
- build_system_settings() filtered the battery store section to known
  BatterySettings fields (to exclude the non-dataclass temperature_derating
  key) but passed the home section through completely unfiltered - a stray
  key surviving a partial migration would crash the whole app at startup.
  Added the same _HOME_DATACLASS_FIELDS filter for home.
- _BATTERY_DATACLASS_FIELDS (new) duplicated api.py's pre-existing
  _BATTERY_MODEL_ATTRS - identical dataclass-field derivation computed in
  two files with no test asserting them equal. api.py now imports the one
  definition from api_conversion.py instead of recomputing it.
- BatterySettings.update()/HomeSettings.update() used hasattr(), which also
  matches method names (not just fields) and violates the repo's explicit
  "never use hasattr" rule - replaced with an explicit dataclass-field
  membership check.

Verified locally against a real running backend (podman-compose CI stack):
startup applies the fixture correctly, PATCH persists and echoes battery/
home settings, values survive a container restart, charging_power_rate/
efficiency_charge/efficiency_discharge (previously silently dropped) now
correctly reach BSM, and a stray pre-migration key injected into the store
no longer crashes startup - confirmed to crash without the home filter fix.

Closes #219

* fix: detect Solcast via entity registry unique_id instead of entity_id (#223)

Solcast detection previously matched entity_id substrings ("forecast_today"),
which fails when Home Assistant translates entity names to the user's
language. This was already fixed and shipped on beta but never ported to
main because the only call site never passed the entity registry through.

- Add SOLCAST_SUFFIX_MAP mapping unique_id keys to BESS sensor keys
- Pass entity_registry to discover_optional_sensors for Solcast lookup
- Add solcast_solar to the debug bundle exporter's registry filters so
  future locale-detection bug reports are actually diagnosable
- Add Solcast entity_registry entries to mock-HA scenarios that previously
  relied on the removed substring fallback

Split from #218; items 1 and 2 depend on the spot_multiplier feature port
and are tracked in #221.

Closes #218

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: unify Battery/Home/Price settings startup/PATCH paths, fix live efficiency-field bug (#197) (#216)

* fix: unify price settings startup/PATCH paths, drop PRICE_STORE_TO_API (#197)

PriceSettings.update() translated camelCase to snake_case on every call,
which made it a no-op for the PATCH /api/settings path (already snake_case)
but load-bearing for the startup path, which went through a separate
hand-maintained PRICE_STORE_TO_API registry to build a camelCase dict.
Two divergent paths to the same sink, kept in sync only by an accidental
no-op quirk — a field added to PriceSettings without a matching registry
entry would silently vanish on restart while still working via PATCH
(the root cause class behind #126's spot_multiplier revert-on-restart bug).

Make snake_case the single canonical format: PriceSettings.update() no
longer translates camelCase, and PRICE_STORE_TO_API (a translation dict)
is replaced by PRICE_REQUIRED_FIELDS (a presence-validation-only set) kept
in sync with the dataclass by a new structural test. The one remaining
camelCase producer (POST /api/setup/complete's live-apply path) is updated
to build snake_case keys directly.

Battery and Home settings have the same historical dual-path pattern but
are intentionally left untouched here, matching the issue's scope — a
candidate follow-up, not a live bug.

* fix: extend settings unification to Battery and Home, fix live efficiency-field bug (#197)

Battery and Home settings had the identical architecture flaw already
fixed for Price: BatterySettings.update()/HomeSettings.update() translated
camelCase on every call (a no-op for PATCH's already-snake_case dicts, but
load-bearing for the startup path's separate BATTERY_STORE_TO_API/
HOME_STORE_TO_API translation registries). Both now take snake_case only,
matching Price; _camel_to_snake is now dead code and removed.

Investigating Battery surfaced a live bug on main (not beta-only, unlike
the Price example): BATTERY_STORE_TO_API only listed 7 of BatterySettings's
10 fields. charging_power_rate/efficiency_charge/efficiency_discharge are
user-editable via the Settings page and persisted to the store, but
build_system_settings() only ever built its startup output from those 7
keys — so a user's efficiency settings applied live via PATCH, then
silently reverted to class defaults (97%/95%/40%) on every restart.

Fix: BATTERY_REQUIRED_FIELDS (the original 7, presence-validated at
startup, unchanged semantics) is now paired with BATTERY_MODEL_ATTRS
(derived live from the dataclass, all 10 fields — relocated from a local
_BATTERY_MODEL_ATTRS in api.py, now the single source of truth for both
the startup and PATCH paths). build_system_settings() filters the full
battery store section through BATTERY_MODEL_ATTRS instead of only the
required 7, so all persisted fields survive a restart, while
temperature_derating (a non-model key in the same store section, applied
separately at BSM construction) is correctly filtered out rather than
crashing BatterySettings.update().

Home has no equivalent gap (its one excluded field, min_valid, is
genuinely not store-backed, like Price's min_profit/use_actual_price) —
HOME_REQUIRED_FIELDS does double duty as both the required-fields check
and the passthrough filter.

Test coverage per section (Battery/Home, mirroring Price): update()
rejects camelCase, build_system_settings() output is snake_case, a
structural test ties the required-fields registry to the dataclass, and a
round-trip test on a real BatterySystemManager asserts the startup and
PATCH paths converge. The Battery round-trip specifically asserts
charging_power_rate/efficiency_charge/efficiency_discharge survive the
startup path — the regression test for the live bug above — plus a
temperature_derating passthrough test.

Full suite verified: 975 fast + 326 slow tests pass, quality-check.sh
clean. Locally verified via podman-compose (docker-compose.ci.yml, now
usable after #214 added podman-compose to requirements-dev.txt): PATCHed
efficiency/charging-power-rate and phase_count/default_hourly values
survive a live container restart; a stray temperature_derating value
survives restart without crashing startup.

* fix: filter stale pre-migration keys from Home PATCH path, matching startup (#197/#219)

The merged Battery/Home unification (from #216 + #224, reconciled after
both PRs independently touched the same files) left one residual gap: the
PATCH /api/settings handler filtered battery through BATTERY_MODEL_ATTRS
but passed home straight through unfiltered, unlike the startup path
(build_system_settings()), which now filters both through their
respective *_MODEL_ATTRS sets.

A stale pre-migration key ('consumption') can coexist with its renamed
successor ('default_hourly') if a migration was ever interrupted — the
rename in settings_store.py only fires when default_hourly is absent, so
once both exist, neither the rename nor cleanup runs on subsequent loads.
The startup path already tolerates this (filters non-model keys out); the
PATCH path did not, and would raise AttributeError inside
HomeSettings.update() — the exact "startup and PATCH must reach BSM
identically" gap this issue is about, just newly surfaced in the Home
column after the merge.

Fix: PATCH's home branch now filters through HOME_MODEL_ATTRS the same
way battery already does, importing it from api_conversion.py.

* feat: port multiplicative spot-price adjustment (spot_multiplier) to main (#227)

main's ENTSO-e provider (#208) only supported the additive markup model,
same as Nordpool ((spot + markup) * VAT + fees). Contracts that scale the
raw spot price instead — e.g. Belgian Luminus Dynamic (spot * 1.0175 +
fees) * VAT, export spot * 1.018 + compensation — got systematically
wrong prices. Beta already built spot_multiplier/export_spot_multiplier
for this (v9.10.0b1); this ports it to main using beta's commit as a
reference, folding in a wiring gap that the referenced beta history hit
too (spot_multiplier missing from PRICE_STORE_TO_API silently reverting
to 1.0 on every restart) and the setup wizard's _PRICE_MAP allow-list
gap this issue explicitly flagged, plus use_actual_price which has the
identical pre-existing gap.

Fixes found and applied during code review before this PR opened:
- setup_complete()'s live-update path omitted the two new fields, so a
  wizard-configured multiplier only took effect after an addon restart
- two spots (PricingFormSection's provider-switch handler and
  SetupWizardPage's scan-on-mount) unconditionally overwrote a user's
  saved custom multiplier with a generic default

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: set working_dir /app/backend in docker-compose.ci.yml to unshadow live mount (#228)

Dockerfile.dev's WORKDIR is /app, which is also the build-time COPY
destination for backend/*.py. Without an explicit working_dir override,
uvicorn's sys.path[0] resolves to the stale image-baked /app/*.py ahead of
the live-mounted /app/backend/*.py in PYTHONPATH, so editing app.py,
api.py, api_conversion.py etc. and restarting the container silently
keeps running old code. docker-compose.yml's dev setup already works
around this by cd'ing into /app/backend before invoking uvicorn; this
applies the same fix to the CI compose file via working_dir.

Verified locally with podman: confirmed sys.path/cwd, confirmed a live
edit is invisible after restart without the fix and picked up
immediately with it.

Closes #226

* test: de-duplicate Battery/Home/Price test classes in test_settings_contracts.py (#229)

Extract a parametrized TestModelAttrsConsistency (Battery + Home) to
replace the two near-identical classes that differed only in dataclass,
constant names, and exclusion set — Price stays separate since it has no
MODEL_ATTRS constant to check against.

Extract _apply_startup()/_apply_patch() helpers to remove the repeated
_bsm() + update_settings() boilerplate shared by all three
*SettingsRoundTrip classes, keeping the one genuinely different line
(Battery's BATTERY_MODEL_ATTRS PATCH filter) inline per class. Also
routes Price's startup test through the existing _full_options() helper
instead of rebuilding the same dict inline.

No behavior change: same 32 tests, same assertions, verified passing
before and after.

Closes #225

* fix: use correct solar baseline in profitability gate, never substitute a worse fallback (#235)

The DP optimizer's profitability gate compared its schedule against a
solar-blind baseline (solar_only_cost was hardcoded equal to
grid_only_cost), so on high-solar days with negative injection prices
the gate could reject a genuinely good schedule and fall back to an
all-IDLE plan that can never discharge — reproducing the reported
"battery fills to 100%, exports at negative prices, then sits full
through the evening peak" pattern.

Root cause and fix, in order of discovery:

- solar_only_cost is now the real per-period solar-only-no-battery cost
  (reusing EconomicData.solar_only_cost, already computed per period),
  and the gate compares against it instead of the zero-solar baseline.
- The same hardcode existed a second time in
  BatterySystemManager._create_updated_schedule's "today only" summary
  recompute, which runs on every normal (non-prepare-next-day) schedule
  update — not just when the gate mishandles a day. Fixed the same way.
- Testing against 3 real historical fixtures found the all-IDLE
  fallback itself can cost more than the schedule it replaces: it still
  pays wear cost on passively-absorbed solar but never discharges to
  recoup it. Added a guardrail so the gate only substitutes the
  fallback when it's actually cheaper than the rejected schedule.
- Updated docs/agents/bess-knowledge.md's "Profit threshold" section,
  which described the old unconditional-substitution, grid-baseline
  behavior and is the documented source of truth for the AI chat/issue
  analyst.

Note: for the reporter's own configuration (cycle_cost_per_kwh=0.40 vs
their evening price spread), discharging into the evening peak is
genuinely unprofitable even with the fix — verified against the exact
real debug-bundle data. This fix corrects the gate's math and prevents
it from making things worse; it does not force discharge where the
economics don't support it.

Closes #231

* docs: trim changelog verbosity, add missing GitHub Release step to release skill (#230)

- CHANGELOG entries with a PR link should be one-liners, not restate the
  full PR description (found this session while cutting v9.9.0b6 on beta)
- The beta release skill was missing the "create a published GitHub
  Release" step; release-addon.yml only triggers on release:published,
  not on a pushed tag, so following the skill as written would silently
  skip the Docker image build
- Note that beta/main's branch protection no longer requires a manual
  review (removed 2026-07-04 to match main, which has none); merge is
  now gated on CI status checks only

* fix: use currency-appropriate default for battery cycle cost (#237)

BATTERY_CHARGE_CYCLE_COST (0.40) is a SEK value, but non-Swedish
installs got it verbatim since it's hidden under advanced settings and
users rarely change it. Setup discovery now sets cycle_cost_per_kwh
from a currency map (EUR 0.035, GBP 0.031) alongside the existing
locale-detection logic for currency/VAT, and the wizard mirrors this
so the Battery step shows the right value before the user ever saves.

* fix: show sell price alongside buy price in Battery SOC chart tooltip (#238)

* fix: show sell price alongside buy price in Battery SOC chart tooltip

BatteryLevelChart's tooltip only ever read hour.buyPrice, so contracts
where sell/export price diverges sharply from buy price (e.g. Belpex
dynamic contracts going negative at midday) gave no visual explanation
for why the optimizer held or exported. Adds a "Show sell price" toggle
(off by default, persisted like the existing dataResolution preference)
that adds a second tooltip row when enabled.

Closes #232

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: move sell price toggle to Energy Flow chart, not Battery SOC chart

BatteryLevelChart only ever surfaces price in a tooltip on hover; the
actual persistent, always-visible price line users look at lives in
EnergyFlowChart's "Electricity Price" line. Toggling sell price on the
Battery SOC card had no visible effect on the chart users actually
read for price shape. Moves the toggle, tooltip row, and a second
dashed price line to EnergyFlowChart instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: always show sell price in Energy Flow tooltip, gate only the line

The toggle now only controls whether the second dashed price line (and
its legend entry) is drawn on the chart. The tooltip's "Sell Price" row
shows whenever sell price data exists, regardless of the toggle state,
so users always get the number on hover even if they haven't turned on
the visual line.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* feat: add self-resolved health-check recovery notice (#239)

* feat: add self-resolved health-check recovery notice (#215)

A sensor that briefly goes ERROR/WARNING and recovers on its own (e.g. an
HA restart) previously left no trace once the live dashboard banner
self-corrected back to OK — if nobody was watching during the window, the
intermittent issue went unnoticed. Track per-component ERROR/WARNING -> OK
transitions and surface them as a dismissible amber banner once all active
issues have cleared, so a self-resolved problem is never silently lost.

- core/bess/health_recovery_tracker.py: new in-memory HealthRecoveryTracker
  (mirrors the existing RuntimeFailureTracker pattern), bounded to 50 entries.
- core/bess/battery_system_manager.py: _run_health_check now diffs the new
  health-check result against the previous cached one and records/clears
  recoveries per component; new get_health_recoveries()/
  acknowledge_health_recoveries() public wrappers.
- backend/api.py: GET /api/health-recoveries, POST
  /api/health-recoveries/acknowledge.
- frontend: new useHealthRecoveries hook; AlertBanner.tsx gains a third
  "recovered" state (only shown once no active critical/warning issues
  remain) and an expandable "show all" toggle for >3 issues; dismiss is no
  longer available while an issue is actively critical/warning (only the
  recovered notice is dismissible), tightening the live banner to match its
  original intent.
- .claude/skills/verify/SKILL.md: persisted the local mock-HA E2E
  verification recipe (podman-compose setup, live sensor toggling via
  /mock/update_sensor, faketime gotcha) discovered while verifying this
  change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add manual demo script for the health-recovery banner (#215)

Lets a reviewer reproduce the break/fix/observe cycle used to verify
PR #239 with one command instead of copy-pasting curl calls, per request
on the PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: give the health-recovery demo a fully clean baseline

The demo previously ran against ci-normal-day.json as-is, which carries
two pre-existing unrelated errors (Nordpool price-date mismatch,
unconfigured energy-monitoring sensors). Those kept the active-issue
(red, non-dismissible) banner showing after the "fix" step, masking the
actual recovered-state banner this script exists to demonstrate.

Now generates a scratch scenario/settings/options fixture set (today's
date instead of the pinned scenario date, added lifetime energy sensors,
InfluxDB left unconfigured) so Battery Control is the only thing that
ever goes wrong — the recovered banner is now actually visible after
`fix`, not just provable via curl.

Scratch files must live under the repo (.demo-scratch/, gitignored), not
/tmp — the podman machine only shares /Users into its VM, so a /tmp path
mounts as missing even though it exists on the host.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat: show specific failing sensor(s) and consequence in the health banner

The active-issue and recovered banners only named the top-level component
(e.g. "Battery Control") with a generic description, no indication of the
actual failing sensor/entity, and no timestamp for active issues (recovered
ones already had one, just wasn't rendered). Per feedback, address both the
missing specifics and unclear consequence wording.

- core/bess/health_check.py: new describe_failing_checks(component) — names
  the specific sub-check(s)/entity behind a component's ERROR/WARNING (e.g.
  "Battery Charging Power Rate (number.growatt_battery_charging_power_rate)"),
  shared by the recovery tracker and the dashboard summary API instead of
  duplicating the logic.
- core/bess/battery_system_manager.py: _update_health_recoveries now calls
  the shared helper instead of its own private copy.
- backend/api.py: /api/dashboard-health-summary's critical_issues now carry
  a "detail" field in both the degraded-mode and cached-results code paths.
- frontend/AlertBanner.tsx: renders the specific detail alongside each
  active issue and each recovery; adds an "As of {time}" note to the active
  banners; rewords both banners to state the actual consequence ("cannot
  reliably operate or optimize your battery...") instead of a vague "may
  affect system operation"; recovered banner gets a summary line above the
  per-item list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat: add break-multi/fix-multi to the demo script

Lets a reviewer see the multiple-issues case directly in the browser
instead of taking it on faith: breaks 2 sensors in Battery Control and
1 in Energy Monitoring at once (2 different components, one with 2
failing sub-checks), then restores all 3 to show both recoveries
listed together with their own detail/time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: add Playwright regression coverage for the health-recovery banner

Per feedback: new UI controls should get an e2e spec covering their logic,
not just component unit tests. Adds e2e/tests/health-recovery.spec.ts,
which runs in the existing CI e2e job (docker-compose.ci.yml, gated on
backend/core/frontend/e2e changes) — verified locally against the real
stack (all 4 pass; full existing e2e suite of 70 tests still passes too).

Covers: breaking a required sensor surfaces it (with the specific failing
entity) in both /api/dashboard-health-summary and the dashboard DOM with no
dismiss button; fixing it records a recovery with the right previousStatus
and detail; acknowledging clears it; a component erroring again drops its
own stale pending recovery. The "recovered" banner's DOM appearance isn't
asserted here — CI's shared ci-normal-day scenario has its own permanently
pinned pre-existing errors (date-mismatch), so hasCriticalErrors never
reliably clears in that shared stack; that path is verified at the API
level instead, which is deterministic regardless of the scenario's other
baseline state.

Also extends scripts/demo_health_recovery.sh with a `fix-partial` command
(restore only one of two broken components) to manually reproduce the
"one recovered, one still failing" case: the recovery is recorded but
stays hidden behind the still-active banner until everything clears.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: cap DP terminal-value estimate at best in-horizon export price (#251)

The DP's rolling-horizon terminal-value fallback priced leftover battery
charge using only the median buy price. On markets with a wide buy/sell
spread (Belgian ENTSO-e/Belpex), that estimate can exceed the real,
achievable evening export price, so the optimizer held charge to chase a
fictitious future bonus instead of exporting now (#126, #244).

A prior fix (#245) swapped buy price for sell price outright, but that's
structurally biased low for ordinary markets (cycle cost is only ever
charged on charging, never on discharge) and over-drains them — proven by
this PR's Nordic-shaped regression test, the exact gap #245 left untested.

Cap the existing buy-median estimate at the best sell price actually visible
in today's horizon: terminal_value = min(buy_based, sell_cap). The cap is
self-calibrating from data the DP already has — it collapses on wide-spread
contracts without a market-specific threshold, and stays inert on
ordinary/Nordic-shaped markets where the best in-horizon peak already beats
the buy-median estimate. Same fix applied to the duplicated estimator in
simulation/verification.py.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* docs: background the slow suite in implement-issue, add second confirm gate (#250)

Watching the slow suite (~30min) run inline was the main driver of
cache-miss cost in long implement-issue sessions: the full diagnosis/TDD
context sits idle past the 5-min prompt-cache TTL, then gets re-read
uncached on every subsequent turn. Step 6 now dispatches quality-check.sh,
the slow suite, and code review as a single background agent instead, with
a new Step 7 confirm gate before verify/PR so CONFIRMED findings still get
a manual checkpoint.

* chore: forward-port beta-only fixes and carry-forward assets from beta (#253)

* Remove ad hoc DP guardrails in favor of pure backward induction (#59)

* Fix stale health-check banner and InfluxDB placeholder-config log spam (#217)

Two bugs from #201's debug bundle:

1. The dashboard health banner only refreshed at startup, after a
   settings save, or after the setup wizard — never periodically. A
   sensor blip at exactly one of those moments left the banner stuck
   showing an error indefinitely, even after sensors recovered, with
   no way for the user to clear it short of an unrelated settings save
   or a full restart. Added a public refresh_health_check() wrapper on
   BatterySystemManager, a 5-minute scheduler job, and a manual
   "Recheck now" button (POST /api/system-health/recheck) so the
   banner self-corrects without user guesswork.

2. get_sensor_data_batch/get_power_sensor_data_batch only checked for
   empty-string InfluxDB config, not the shipped placeholder
   credentials, so the periodic sensor-collection job kept attempting
   real HTTP connections to the default (unconfigured) InfluxDB URL
   every ~15 minutes for any user who never set it up. Both now use
   the existing is_influxdb_configured() check, which also gained a
   bucket-emptiness check it was missing.

Also converts two pre-existing call sites (settings-save refresh,
setup-wizard completion) from the private _run_health_check() to the
new public wrapper, for consistency with the no-private-cross-class-
calls pattern the wrapper itself establishes.

* docs: backfill missing changelog entries for #207, #211, #217 (#220)

These PRs shipped fixes but didn't touch CHANGELOG.md, leaving gaps
between what's documented and what actually merged since 9.8.1. Also
converts (#N) references to full markdown links, since plain #N text
only auto-links correctly when read within this same repo.

* fix: unify Battery/Home settings startup/PATCH paths, drop *_STORE_TO_API registries (#219) (#224)

BatterySettings.update()/HomeSettings.update() no longer translate camelCase
- snake_case (the settings store's native format) is now the canonical shape
for both the startup path and the PATCH /api/settings path, mirroring the
identical fix already applied to Price settings in #197/#216.

Root cause: the startup path filtered store fields through hand-maintained
BATTERY_STORE_TO_API/HOME_STORE_TO_API translation dicts before calling
update_settings(), while the PATCH handler already bypassed them and passed
the raw snake_case store dict directly - working only because
_camel_to_snake() was a no-op on already-snake_case keys. A field present in
the store but missing from the registry (e.g. charging_power_rate,
efficiency_charge, efficiency_discharge) was silently dropped at startup
while continuing to work via PATCH - the same bug class fixed for Price
in #197 after Belgian user #126's spot_multiplier revert-on-restart bug.

Code review (medium effort) surfaced and fixed three issues before this
was verified:
- build_system_settings() filtered the battery store section to known
  BatterySettings fields (to exclude the non-dataclass temperature_derating
  key) but passed the home section through completely unfiltered - a stray
  key surviving a partial migration would crash the whole app at startup.
  Added the same _HOME_DATACLASS_FIELDS filter for home.
- _BATTERY_DATACLASS_FIELDS (new) duplicated api.py's pre-existing
  _BATTERY_MODEL_ATTRS - identical dataclass-field derivation computed in
  two files with no test asserting them equal. api.py now imports the one
  definition from api_conversion.py instead of recomputing it.
- BatterySettings.update()/HomeSettings.update() used hasattr(), which also
  matches method names (not just fields) and violates the repo's explicit
  "never use hasattr" rule - replaced with an explicit dataclass-field
  membership check.

Verified locally against a real running backend (podman-compose CI stack):
startup applies the fixture correctly, PATCH persists and echoes battery/
home settings, values survive a container restart, charging_power_rate/
efficiency_charge/efficiency_discharge (previously silently dropped) now
correctly reach BSM, and a stray pre-migration key injected into the store
no longer crashes startup - confirmed to crash without the home filter fix.

Closes #219

* fix: detect Solcast via entity registry unique_id instead of entity_id (#223)

Solcast detection previously matched entity_id substrings ("forecast_today"),
which fails when Home Assistant translates entity names to the user's
language. This was already fixed and shipped on beta but never ported to
main because the only call site never passed the entity registry through.

- Add SOLCAST_SUFFIX_MAP mapping unique_id keys to BESS sensor keys
- Pass entity_registry to discover_optional_sensors for Solcast lookup
- Add solcast_solar to the debug bundle exporter's registry filters so
  future locale-detection bug reports are actually diagnosable
- Add Solcast entity_registry entries to mock-HA scenarios that previously
  relied on the removed substring fallback

Split from #218; items 1 and 2 depend on the spot_multiplier feature port
and are tracked in #221.

Closes #218

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: unify Battery/Home/Price settings startup/PATCH paths, fix live efficiency-field bug (#197) (#216)

* fix: unify price settings startup/PATCH paths, drop PRICE_STORE_TO_API (#197)

PriceSettings.update() translated camelCase to snake_case on every call,
which made it a no-op for the PATCH /api/settings path (already snake_case)
but load-bearing for the startup path, which went through a separate
hand-maintained PRICE_STORE_TO_API registry to build a camelCase dict.
Two divergent paths to the same sink, kept in sync only by an accidental
no-op quirk — a field added to PriceSettings without a matching registry
entry would silently vanish on restart while still working via PATCH
(the root cause class behind #126's spot_multiplier revert-on-restart bug).

Make snake_case the single canonical format: PriceSettings.update() no
longer translates camelCase, and PRICE_STORE_TO_API (a translation dict)
is replaced by PRICE_REQUIRED_FIELDS (a presence-validation-only set) kept
in sync with the dataclass by a new structural test. The one remaining
camelCase producer (POST /api/setup/complete's live-apply path) is updated
to build snake_case keys directly.

Battery and Home settings have the same historical dual-path pattern but
are intentionally left untouched here, matching the issue's scope — a
candidate follow-up, not a live bug.

* fix: extend settings unification to Battery and Home, fix live efficiency-field bug (#197)

Battery and Home settings had the identical architecture flaw already
fixed for Price: BatterySettings.update()/HomeSettings.update() translated
camelCase on every call (a no-op for PATCH's already-snake_case dicts, but
load-bearing for the startup path's separate BATTERY_STORE_TO_API/
HOME_STORE_TO_API translation registries). Both now take snake_case only,
matching Price; _camel_to_snake is now dead code and removed.

Investigating Battery surfaced a live bug on main (not beta-only, unlike
the Price example): BATTERY_STORE_TO_API only listed 7 of BatterySettings's
10 fields. charging_power_rate/efficiency_charge/efficiency_discharge are
user-editable via the Settings page and persisted to the store, but
build_system_settings() only ever built its startup output from those 7
keys — so a user's efficiency settings applied live via PATCH, then
silently reverted to class defaults (97%/95%/40%) on every restart.

Fix: BATTERY_REQUIRED_FIELDS (the original 7, presence-validated at
startup, unchanged semantics) is now paired with BATTERY_MODEL_ATTRS
(derived live from the dataclass, all 10 fields — relocated from a local
_BATTERY_MODEL_ATTRS in api.py, now the single source of truth for both
the startup and PATCH paths). build_system_settings() filters the full
battery store section through BATTERY_MODEL_ATTRS instead of only the
required 7, so all persisted fields survive a restart, while
temperature_derating (a non-model key in the same store section, applied
separately at BSM construction) is correctly filtered out rather than
crashing BatterySettings.update().

Home has no equivalent gap (its one excluded field, min_valid, is
genuinely not store-backed, like Price's min_profit/use_actual_price) —
HOME_REQUIRED_FIELDS does double duty as both the required-fields check
and the passthrough filter.

Test coverage per section (Battery/Home, mirroring Price): update()
rejects camelCase, build_system_settings() output is snake_case, a
structural test ties the required-fields registry to the dataclass, and a
round-trip test on a real BatterySystemManager asserts the startup and
PATCH paths converge. The Battery round-trip specifically asserts
charging_power_rate/efficiency_charge/efficiency_discharge survive the
startup path — the regression test for the live bug above — plus a
temperature_derating passthrough test.

Full suite verified: 975 fast + 326 slow tests pass, quality-check.sh
clean. Locally verified via podman-compose (docker-compose.ci.yml, now
usable after #214 added podman-compose to requirements-dev.txt): PATCHed
efficiency/charging-power-rate and phase_count/default_hourly values
survive a live container restart; a stray temperature_derating value
survives restart without crashing startup.

* fix: filter stale pre-migration keys from Home PATCH path, matching startup (#197/#219)

The merged Battery/Home unification (from #216 + #224, reconciled after
both PRs independently touched the same files) left one residual gap: the
PATCH /api/settings handler filtered battery through BATTERY_MODEL_ATTRS
but passed home straight through unfiltered, unlike the startup path
(build_system_settings()), which now filters both through their
respective *_MODEL_ATTRS sets.

A stale pre-migration key ('consumption') can coexist with its renamed
successor ('default_hourly') if a migration was ever interrupted — the
rename in settings_store.py only fires when default_hourly is absent, so
once both exist, neither the rename nor cleanup runs on subsequent loads.
The startup path already tolerates this (filters non-model keys out); the
PATCH path did not, and would raise AttributeError inside
HomeSettings.update() — the exact "startup and PATCH must reach BSM
identically" gap this issue is about, just newly surfaced in the Home
column after the merge.

Fix: PATCH's home branch now filters through HOME_MODEL_ATTRS the same
way battery already does, importing it from api_conversion.py.

* feat: port multiplicative spot-price adjustment (spot_multiplier) to main (#227)

main's ENTSO-e provider (#208) only supported the additive markup model,
same as Nordpool ((spot + markup) * VAT + fees). Contracts that scale the
raw spot price instead — e.g. Belgian Luminus Dynamic (spot * 1.0175 +
fees) * VAT, export spot * 1.018 + compensation — got systematically
wrong prices. Beta already built spot_multiplier/export_spot_multiplier
for this (v9.10.0b1); this ports it to main using beta's commit as a
reference, folding in a wiring gap that the referenced beta history hit
too (spot_multiplier missing from PRICE_STORE_TO_API silently reverting
to 1.0 on every restart) and the setup wizard's _PRICE_MAP allow-list
gap this issue explicitly flagged, plus use_actual_price which has the
identical pre-existing gap.

Fixes found and applied during code review before this PR opened:
- setup_complete()'s live-update path omitted the two new fields, so a
  wizard-configured multiplier only took effect after an addon restart
- two spots (PricingFormSection's provider-switch handler and
  SetupWizardPage's scan-on-mount) unconditionally overwrote a user's
  saved custom multiplier with a generic default

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: set working_dir /app/backend in docker-compose.ci.yml to unshadow live mount (#228)

Dockerfile.dev's WORKDIR is /app, which is also the build-time COPY
destination for backend/*.py. Without an explicit working_dir override,
uvicorn's sys.path[0] resolves to the stale image-baked /app/*.py ahead of
the live-mounted /app/backend/*.py in PYTHONPATH, so editing app.py,
api.py, api_conversion.py etc. and restarting the container silently
keeps running old code. docker-compose.yml's dev setup already works
around this by cd'ing into /app/backend before invoking uvicorn; this
applies the same fix to the CI compose file via working_dir.

Verified locally with podman: confirmed sys.path/cwd, confirmed a live
edit is invisible after restart without the fix and picked up
immediately with it.

Closes #226

* test: de-duplicate Battery/Home/Price test classes in test_settings_contracts.py (#229)

Extract a parametrized TestModelAttrsConsistency (Battery + Home) to
replace the two near-identical classes that differed only in dataclass,
constant names, and exclusion set — Price stays separate since it has no
MODEL_ATTRS constant to check against.

Extract _apply_startup()/_apply_patch() helpers to remove the repeated
_bsm() + update_settings() boilerplate shared by all three
*SettingsRoundTrip classes, keeping the one genuinely different line
(Battery's BATTERY_MODEL_ATTRS PATCH filter) inline per class. Also
routes Price's startup test through the existing _full_options() helper
instead of rebuilding the same dict inline.

No behavior change: same 32 tests, same assertions, verified passing
before and after.

Closes #225

* fix: use correct solar baseline in profitability gate, never substitute a worse fallback (#235)

The DP optimizer's profitability gate compared its schedule against a
solar-blind baseline (solar_only_cost was hardcoded equal to
grid_only_cost), so on high-solar days with negative injection prices
the gate could reject a genuinely good schedule and fall back to an
all-IDLE plan that can never discharge — reproducing the reported
"battery fills to 100%, exports at negative prices, then sits full
through the evening peak" pattern.

Root cause and fix, in order of discovery:

- solar_only_cost is now the real per-period solar-only-no-battery cost
  (reusing EconomicData.solar_only_cost, already computed per period),
  and the gate compares against it instead of the zero-solar baseline.
- The same hardcode existed a second time in
  BatterySystemManager._create_updated_schedule's "today only" summary
  recompute, which runs on every normal (non-prepare-next-day) schedule
  update — not just when the gate mishandles a day. Fixed the same way.
- Testing against 3 real historical fixtures found the all-IDLE
  fallback itself can cost more than the schedule it replaces: it still
  pays wear cost on passively-absorbed solar but never discharges to
  recoup it. Added a guardrail so the gate only substitutes the
  fallback when it's actually cheaper than the rejected schedule.
- Updated docs/agents/bess-knowledge.md's "Profit threshold" section,
  which described the old unconditional-substitution, grid-baseline
  behavior and is the documented source of truth for the AI chat/issue
  analyst.

Note: for the reporter's own configuration (cycle_cost_per_kwh=0.40 vs
their evening price spread), discharging into the evening peak is
genuinely unprofitable even with the fix — verified against the exact
real debug-bundle data. This fix corrects the gate's math and prevents
it from making things worse; it does not force discharge where the
economics don't support it.

Closes #231

* docs: trim changelog verbosity, add missing GitHub Release step to release skill (#230)

- CHANGELOG entries with a PR link should be one-liners, not restate the
  full PR description (found this session while cutting v9.9.0b6 on beta)
- The beta release skill was missing the "create a published GitHub
  Release" step; release-addon.yml only triggers on release:published,
  not on a pushed tag, so following the skill as written would silently
  skip the Docker image build
- Note that beta/main's branch protection no longer requires a manual
  review (removed 2026-07-04 to match main, which has none); merge is
  now gated on CI status checks only

* fix: use currency-appropriate default for battery cycle cost (#237)

BATTERY_CHARGE_CYCLE_COST (0.40) is a SEK value, but non-Swedish
installs got it verbatim since it's hidden under advanced settings and
users rarely change it. Setup discovery now sets cycle_cost_per_kwh
from a currency map (EUR 0.035, GBP 0.031) alongside the existing
locale-detection logic for currency/VAT, and the wizard mirrors this
so the Battery step shows the right value before the user ever saves.

* fix: show sell price alongside buy price in Battery SOC chart tooltip (#238)

* fix: show sell price alongside buy price in Battery SOC chart tooltip

BatteryLevelChart's tooltip only ever read hour.buyPrice, so contracts
where sell/export price diverges sharply from buy price (e.g. Belpex
dynamic contracts going negative at midday) gave no visual explanation
for why the optimizer held or exported. Adds a "Show sell price" toggle
(off by default, persisted like the existing dataResolution preference)
that adds a second tooltip row when enabled.

Closes #232

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: move sell price toggle to Energy Flow chart, not Battery SOC chart

BatteryLevelChart only ever surfaces price in a tooltip on hover; the
actual persistent, always-visible price line users look at lives in
EnergyFlowChart's "Electricity Price" line. Toggling sell price on the
Battery SOC card had no visible effect on the chart users actually
read for price shape. Moves the toggle, tooltip row, and a second
dashed price line to EnergyFlowChart instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: always show sell price in Energy Flow tooltip, gate only the line

The toggle now only controls whether the second dashed price line (and
its legend entry) is drawn on the chart. The tooltip's "Sell Price" row
shows whenever sell price data exists, regardless of the toggle state,
so users always get the number on hover even if they haven't turned on
the visual line.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* feat: add self-resolved health-check recovery notice (#239)

* feat: add self-resolved health-check recovery notice (#215)

A sensor that briefly goes ERROR/WARNING and recovers on its own (e.g. an
HA restart) previously left no trace once the live dashboard banner
self-corrected back to OK — if nobody was watching during the window, the
intermittent issue went unnoticed. Track per-component ERROR/WARNING -> OK
transitions and surface them as a dismissible amber banner once all active
issues have cleared, so a self-resolved problem is never silently lost.

- core/bess/health_recovery_tracker.py: new in-memory HealthRecoveryTracker
  (mirrors the existing RuntimeFailureTracker pattern), bounded to 50 entries.
- core/bess/battery_system_manager.py: _run_health_check now diffs the new
  health-check result against the previous cached one and records/clears
  recoveries per component; new get_health_recoveries()/
  acknowledge_health_recoveries() public wrappers.
- backend/api.py: GET /api/health-recoveries, POST
  /api/health-recoveries/acknowledge.
- frontend: new useHealthRecoveries hook; AlertBanner.tsx gains a third
  "recovered" state (only shown once no active critical/warning issues
  remain) and an expandable "show all" toggle for >3 issues; dismiss is no
  longer available while an issue is actively critical/warning (only the
  recovered notice is dismissible), tightening the live banner to match its
  original intent.
- .claude/skills/verify/SKILL.md: persisted the local mock-HA E2E
  verification recipe (podman-compose setup, live sensor toggling via
  /mock/update_sensor, faketime gotcha) discovered while verifying this
  change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add manual demo script for the health-recovery banner (#215)

Lets a reviewer reproduce the break/fix/observe cycle used to verify
PR #239 with one command instead of copy-pasting curl calls, per request
on the PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: give the health-recovery demo a fully clean baseline

The demo previously ran against ci-normal-day.json as-is, which carries
two pre-existing unrelated errors (Nordpool price-date mismatch,
unconfigured energy-monitoring sensors). Those kept the active-issue
(red, non-dismissible) banner showing after the "fix" step, masking the
actual recovered-state banner this script exists to demonstrate.

Now generates a scratch scenario/settings/options fixture set (today's
date instead of the pinned scenario date, added lifetime energy sensors,
InfluxDB left unconfigured) so Battery Control is the only thing that
ever goes wrong — the recovered banner is now actually visible after
`fix`, not just provable via curl.

Scratch files must live under the repo (.demo-scratch/, gitignored), not
/tmp — the podman machine only shares /Users into its VM, so a /tmp path
mounts as missing even though it exists on the host.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat: show specific failing sensor(s) and consequence in the health banner

The active-issue and recovered banners only named the top-level component
(e.g. "Battery Control") with a generic description, no indication of the
actual failing sensor/entity, and no timestamp for active issues (recovered
ones already had one, just wasn't rendered). Per feedback, address both the
missing specifics and unclear consequence wording.

- core/bess/health_check.py: new describe_failing_checks(component) — names
  the specific sub-check(s)/entity behind a component's ERROR/WARNING (e.g.
  "Battery Charging Power Rate (number.growatt_battery_charging_power_rate)"),
  shared by the recovery tracker and the dashboard summary API instead of
  duplicating the logic.
- core/bess/battery_system_manager.py: _update_health_recoveries now calls
  the shared helper instead of its own private copy.
- backend/api.py: /api/dashboard-health-summary's critical_issues now carry
  a "detail" field in both the degraded-mode and cached-results code paths.
- frontend/AlertBanner.tsx: renders the specific detail alongside each
  active issue and each recovery; adds an "As of {time}" note to the active
  banners; rewords both banners to state the actual consequence ("cannot
  reliably operate or optimize your battery...") instead of a vague "may
  affect system operation"; recovered banner gets a summary line above the
  per-item list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat: add break-multi/fix-multi to the demo script

Lets a reviewer see the multiple-issues case directly in the browser
instead of taking it on faith: breaks 2 sensors in Battery Control and
1 in Energy Monitoring at once (2 different components, one with 2
failing sub-checks), then restores all 3 to show both recoveries
listed together with their own detail/time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: add Playwright regression coverage for the health-recovery banner

Per feedback: new UI controls should get an e2e spec covering their logic,
not just component unit tests. Adds e2e/tests/health-recovery.spec.ts,
which runs in the existing CI e2e job (docker-compose.ci.yml, gated on
backend/core/frontend/e2e changes) — verified locally against the real
stack (all 4 pass; full existing e2e suite of 70 tests still passes too).

Covers: breaking a required sensor surfaces it (with the specific failing
entity) in both /api/dashboard-health-summary and the dashboard DOM with no
dismiss button; fixing it records a recovery with the right previousStatus
and detail; acknowledging clears it; a component erroring again drops its
own stale pending recovery. The "recovered" banner's DOM appearance isn't
asserted here — CI's shared ci-normal-day scenario has its own permanently
pinned pre-existing errors (date-mismatch), so hasCriticalErrors never
reliably clears in that shared stack; that path is verified at the API
level instead, which is deterministic regardless of the scenario's other
baseline state.

Also extends scripts/demo_health_recovery.sh with a `fix-partial` command
(restore only one of two broken components) to manually reproduce the
"one recovered, one still failing" case: the recovery is recorded but
stays hidden behind the still-active banner until everything clears.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add design spec for removing ad hoc DP guardrails

Replaces the cost_basis profitability floor, anti-cycling special case, and
whole-day rejection gate with pure backward induction plus a trivial
idle-vs-DP-cost numerical safety net. Bundles the #240 flow-accounting fix
since it touches the same reward branch. Evidence and reasoning validated
empirically against all 26 pinned fixtures this session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: refine spec after regression-fix investigation and scope feedback

- Add interpolated-V replay recompute as a partial (not full) fix for the
  discretization residual; keep the idle-safety-net for the remainder.
- Scope min_action_profit_threshold removal to the algorithm only; config
  schema removal becomes a separate follow-up issue.
- Add plan-faithfulness/simulator test category to the impact list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add implementation plan for DP guardrail removal

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: drop duplicate regression test from Task 4 per pre-flight review

Task 5's parametrized test already covers the specific fixture Task 4's
version singled out; keeping only one avoids near-verbatim duplicate test
logic.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: remove discharge profitability floor, fix #240 export miscrediting

_compute_reward no longer vetoes a discharge via a cost_basis floor -- IDLE
already makes that comparison correctly through the forward-looking value
function during backward induction (Bellman's principle of optimality).
Also fixes #240: a discharge overshooting home_consumption by less than the
BATTERY_EXPORT threshold (0.1 kWh) is no longer credited as export revenue,
since load-first hardware self-throttles and never actually exports it.

Updates test_surplus_disposition.py test expectations to reflect the removal
of the profitability floor guardrail (tests now verify discharges are no
longer blocked, matching the new behavior).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor: remove dead C grid, dead fallback, and dead stored_period_data

C was the cost-basis-threading grid #234 found dead (loop-order bug meant it
was never read back); stored_period_data was already discarded by the sole
caller before this change, so building a full PeriodData per grid cell in
the hot loop was pure waste. The "no valid action found" fallback branch is
now unreachable since IDLE is always feasible and _compute_reward never
returns -inf (Task 1).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor: replace whole-day rejection gate with idle-cost safety net

The min_action_profit_threshold gate distrusted the DP's own economics
wholesale. Replaced with a trivial comparison against the all-IDLE schedule,
justified purely by SoE-grid discretization noise (verified empirically),
not by an economic threshold. The algorithm no longer reads
min_action_profit_threshold; the config field itself is left in place for a
separate follow-up issue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: remove stale min_action_profit_threshold description from module docstring

* refactor: recompute replay actions from interpolated V instead of grid snap

Step 2 of optimize_battery_schedule used to snap the continuous SoE to the
nearest discretized grid index and trust that cell's stored policy action --
a policy computed for a slightly different state than the one actually
reached. Replaced with a one-step recompute at the true continuous SoE using
the already-known V[t+1, :] (linearly interpolated) as the continuation
value: the same reward+max(V) logic as the backward pass, just applied at
the true state. Reduces (does not eliminate) the SoE-grid discretization
residual the Task 3 safety net guards against. policy is no longer needed by
any caller and is dropped from _run_dynamic_programming's return.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: add DP-never-worse-than-idle regression across all pinned fixtures

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: lock in load_first self-throttling for the #240 boundary case

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: remove tests for the removed min_action_profit_threshold gate

test_action_threshold.py tested the whole-day rejection gate directly, and
test_gate_never_substitutes_a_worse_fallback compared against a
gate-disabled re-run -- both premises no longer apply now that the gate is
gone and the property it checked is unconditionally true by construction
(see the Task 5 regression test).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add Task 8b to fix classify_strategic_intent threshold bug

Discovered mid-Task-8: removing the anti-cycling floor exposed a real
control-fidelity bug where small export-only discharges get misclassified
LOAD_SUPPORT (threshold 0.1 vs the function's own 0.01 elsewhere), which maps
to a hardware mode that cannot execute an export, causing R == P failures up
to 18.7 SEK on quarter-hourly fixtures.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: classify_strategic_intent uses consistent 0.01 kWh export threshold

The BATTERY_EXPORT check used a 0.1 kWh threshold, ten times coarser than
every other flow check in this function (0.01). A discharge with zero home-
deficit coverage and a small (0.01-0.1 kWh) export was misclassified
LOAD_SUPPORT, which maps to load_first -- a mode that can only cover a real
deficit and physically cannot export. When the deficit-based delivery
computed exactly zero, _state_transition's IDLE branch absorbed the entire
solar surplus instead, a much larger unplanned action whose error compounded
for the rest of the horizon. Traced via Task 8's R == P failures on 8 of 9
quarter-hourly fixtures (gaps up to 18.7 SEK).

Also fixes the identical inconsistency in models.py's infer_intent_from_flows
(observational/dashboard-display only, not in the R == P execution path, but
the same threshold mismatch against its own sibling checks).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: scope pytest.mark.slow to the one test that needs it

The module-level pytestmark = pytest.mark.slow applied to every test in
the file regardless of its textual position, silently excluding several
fast unit tests (_compute_reward, _run_dynamic_programming,
classify_strategic_intent) from the fast suite. Move the marker to a
decorator on only the 26-fixture parametrized regression test that
actually needs to run the full optimizer across many scenarios.

* test: regenerate pinned fixture expectations after guardrail removal + classification fix

Three distinct categories of change, bundled since they all touch the same
fixture set and were verified together:

1. 14 fixtures' expected_results: legitimate equal-or-better economics from
   the guardrail-removal redesign (Tasks 1-4). Hand-verified against the
   design doc's empirical validation before updating.

2. realworld_2026_04_29_220919's expected_behavior.intents_present: removed
   LOAD_SUPPORT (BATTERY_EXPORT already listed). The classify_strategic_intent
   threshold fix (d0a42f7) correctly reclassifies 13 periods that have
   battery_to_home=0.0 (zero real home-deficit coverage, solar fully covers
   consumption) from LOAD_SUPPORT to BATTERY_EXPORT. Verified the live intent
   distribution directly: {IDLE: 51, BATTERY_EXPORT: 26, SOLAR_STORAGE: 20,
   GRID_CHARGING: 6, SOLAR_EXPORT: 1} -- zero LOAD_SUPPORT periods.

3. realworld_2026_03_24_225535 and synthetic_2024_08_16_high_spread_with_solar's
   expected_results: two ACCEPTED small regressions (+0.023 SEK and +0.016 SEK
   respectively), not improvements. These are smaller than the design doc's one
   documented residual (0.16 SEK on historical_2025_01_05_no_spread_no_solar)
   and consistent with the same SoE-grid-discretization mechanism it describes
   (Step-2 replay snapping to the nearest 0.1 kWh grid point) -- just manifesting
   on two more fixtures than the doc's original empirical validation happened to
   catch. Recomputed fresh in this session and confirmed unchanged from the
   prior investigation (the classification fix doesn't touch cost calculation).

Full slow scenario suite: 25/26 pass (including R == P plan-faithfulness for
all 25). The one remaining failure, realworld_2026_04_29_195900 (missing
SOLAR_EXPORT periods in intents_present), is a separate, pre-existing bug
confirmed unrelated to both the guardrail removal and the classification
threshold fix (verified via git-stash bisection in a prior task) -- out of
scope here, needs its own investigation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add Task 9b to recalibrate test_solar_export_discharge_gate.py

Discovered mid-Task-9: both slow scenario tests in this file are stale
against the pre-redesign reward model (not the Task 8b threshold fix, ruled
out). Scenario 1's hardcoded shadow_price constant needs updating to match
the documented shadow==sell_price steady-state law; scenario 2's inputs no
longer produce any hold state at all since full-day arbitrage now correctly
dominates them. Both diagnosed and a verified replacement scenario found
before writing this task.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: recalibrate solar-export-gate scenarios for the guardrail redesign

Both scenarios were tuned to the pre-redesign reward model (the removed
discharge-profitability floor). Scenario 1's hardcoded shadow_price constant
(0.7093) was stale -- the DP now correctly converges to shadow == sell_price
in steady state, per the already-documented economic law in
docs/agents/bess-knowledge.md; the one verified finite-horizon transient
period is no longer asserted against a fixed number. Scenario 2's inputs (a
sustained 5x export premium with no future recharge cost) made full-day
arbitrage strictly better than holding, so no SOLAR_EXPORT period existed at
all anymore -- verified the DP's new schedule beats the old hardcoded "hold"
expectation by 16.9 SEK. Replaced with inputs that restore a genuine hold
state (export premium now, but an expensive window right after that makes
preserving stored energy the better choice).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: add Tasks 11-12 from final whole-branch review findings

Task 11 corrects a false "pre-existing" claim (git-stash bisection is
methodologically unsound against committed history) for
realworld_2026_04_29_195900's stale intents_present. Task 12 reconciles
_compute_reward's export threshold with classify_strategic_intent's, found
mismatched (0.1 vs 0.01) after Task 8b only fixed one side. Both diagnosed
and partially verified by the controller before writing these tasks.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: fix realworld_2026_04_29_195900's stale intents_present expectation

Previously believed pre-existing (Tasks 8/9's git-stash bisection was
methodologically unsound -- stash cannot reach behind this branch's own
committed history). Independently re-verified against the true merge-base:
this branch genuinely changed the schedule (SOLAR_EXPORT -> SOLAR_STORAGE +
more active discharge), improving battery_solar_cost by 11.29 SEK (already
reflected in this fixture's expected_results from Task 8) with R == P
holding exactly. Only the separate intents_present list was stale.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: reconcile BATTERY_EXPORT_THRESHOLD_KWH with classification threshold

_compute_reward's export-credit threshold (0.1) no longer matched
classify_strategic_intent's classification threshold (0.01, fixed in Task
8b) -- found by the final whole-branch review. 80 periods across 8 fixtures
sat in the (0.01, 0.1] gap: already correctly classified BATTERY_EXPORT
(executed as real exports via grid_first), but the DP's own reward search
still zeroed their export credit, undervaluing actions it will actually
realize.

Recomputed all 26 fixtures' economics against the new threshold. 4 changed
(22 unaffected):

  - historical_2025_06_02_high_solar_export: battery_solar_cost -5.6755 ->
    -5.7875 SEK (0.1120 SEK better)
  - realworld_2026_04_27_184643: battery_solar_cost -10.7780 -> -11.0672 SEK
    (0.2892 SEK better)
  - realworld_2026_04_11_004719: battery_solar_cost 96.0992 -> 96.1025 SEK
    (0.0033 SEK worse -- accepted small regression)
  - realworld_2026_04_29_195900: battery_solar_cost 5.0938 -> 5.0957 SEK
    (0.0019 SEK worse -- accepted small regression)

The two regressions are both sub-öre, smaller than the already-accepted
residuals from Task 8, and R == P holds comfortably for both (well within
tolerance). This is the exact failure mode the brief anticipated -- the
DP's search resolves a close call differently once the two thresholds
agree -- not a new bug. Net effect across the 4 changed fixtures is
strongly positive: +0.0052 SEK worse vs -0.4012 SEK better. Updated all 4
fixtures' expected_results (including total_charged/total_discharged, which
also shifted since the DP's chosen action sequence changed, not just its
valuation of the already-taken sequence).

Also fixed test_small_discharge_overshoot_not_credited_as_export, which
used a 0.05 kWh overshoot as its "below threshold" example -- that value
was below the old 0.1 threshold but is now above the new 0.01 threshold, so
it broke on this change. Updated it to a genuinely sub-0.01 kWh overshoot
(0.005 kWh) and fixed two stale "0.1 kWh" docstring references.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* fix: action-derived GRID_CHARGING charge_rate display + reduce log verbosity (#62)

get_detailed_period_groups() showed a static 100% charge_rate for
GRID_CHARGING instead of the action-derived value get_period_settings()
already computed, so the debug log's schedule table (and API/frontend)
misreported small top-up charges as full-rate. Extracted a shared
_compute_charge_rate() helper used by both…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pookey @johanzander