fix(output): apply load_scaling_dynamic and manual_load_adjust to today_remaining - #4511
Conversation
…ay_remaining (#4496 follow-up) #4506 fixed today_remaining diverging from the plan's own remaining-load total by applying the flat load_scaling factor, but missed two other multipliers step_data_history() (fetch.py) also applies when building the plan's load_minutes_step: load_scaling_dynamic (per-minute - carries saving-session/free-electricity-event scaling, and any per-window override from rates_import_override or the manual API) and manual_load_adjust (additive, per-minute). Confirmed via a live report on the same issue: a user with a 2-hour "power up" (free electricity) event set to load_scaling: 1.5 via the manual API saw today_remaining still diverge from the plan's own total even after updating to the #4506 fix, since only the flat 1.05 base load_scaling was being applied. Replayed their attached debug.yaml directly - their load_scaling_dynamic dict does carry 1.5 for the 2-hour window as expected; load_today_comparison() never looked at it. After this fix, the same replay's plan-vs-sensor ratio drops from a much larger gap to 1.0076, matching the same small residual seen on the original #4506 replay. New tests confirm today_remaining scales with load_scaling_dynamic (uniformly across the day, mirroring the existing load_scaling test) and reflects a single manual_load_adjust entry additively. Both confirmed to fail without this fix and pass with it.
Follow-up to the previous commit - cspell flagged the handle as an unknown word; reworded rather than adding it to the dictionary.
Raised alongside the #4496 follow-up investigation: the "days_previous_auto enabled - using weighted-bucket historical load forecast..." log line fires unconditionally every cycle purely because days_previous_auto defaults to True, regardless of whether the weighted-bucket forecast is actually used that cycle. Load ML (or any source that sets load_forecast_only) takes precedence and skips it entirely - fetch_sensor_data() already has its own, correctly conditional "Using weighted-bucket historical load forecast over N days" line that only logs when the fallback genuinely runs. Reworded to describe what's enabled/configured, not what happened, so a Load ML user reading the log isn't misled into thinking both forecast sources are being blended every cycle when they aren't - confirmed via code trace, not just the log wording, that no double-application actually occurs.
|
Rolled in a small related fix: the `days_previous_auto` log line in `fetch_config_options()` fired every cycle unconditionally ("...using weighted-bucket historical load forecast...") regardless of whether it was actually used that cycle - Load ML taking precedence skips it entirely, confirmed via the `not self.load_forecast_only` guard in `fetch_sensor_data()`, no double-application occurs. Reworded so it doesn't read as confirming active usage on every cycle. This is what prompted gcoan's original "is predbat double applying in-day load adjustment?" question further up the issue - answer is no, but the log line was genuinely misleading. |
There was a problem hiding this comment.
Pull request overview
This PR is a follow-up fix to align load_today_comparison()’s published predicted/adjusted “today_remaining” values with the plan’s step_data_history() load construction, by incorporating the remaining multipliers (load_scaling_dynamic and manual_load_adjust) that were previously not applied.
Changes:
- Update
load_today_comparison()to applymanual_load_adjust(additive) andload_scaling_dynamic(multiplicative) to future minutes, matchingstep_data_history()behavior. - Add a regression test covering
load_scaling_dynamicandmanual_load_adjusteffects ontoday_remaining. - Clarify
days_previous_autologging to reflect that it’s a fallback when other forecast sources take precedence.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| apps/predbat/output.py | Applies manual_load_adjust and load_scaling_dynamic to future predicted load buckets so today_remaining matches plan load stepping. |
| apps/predbat/tests/test_load_today_comparison.py | Adds regression coverage for load_scaling_dynamic and manual_load_adjust behavior in today_remaining. |
| apps/predbat/fetch.py | Refines a configuration-time log line to avoid implying weighted-bucket forecast is always used each cycle. |
Suppressed comments (1)
apps/predbat/tests/test_load_today_comparison.py:392
- Similarly, the manual_load_adjust case only asserts the predicted sensor. Since load_energy_adjusted is derived from the predicted curve (difference_cap applied), it should also be asserted here to prevent regressions where only one sensor path is corrected.
# manual_load_adjust applied additively at a single future minute
my_predbat.load_scaling_dynamic = {}
manual_adjust_minute = minutes_now + 60
manual_adjust_kwh = 6.0
my_predbat.manual_load_adjust = {manual_adjust_minute: manual_adjust_kwh}
my_predbat.load_today_comparison(load_minutes, load_forecast, {}, import_minutes, minutes_now=minutes_now, step=5, save=True)
manual_adjusted_remaining = my_predbat.dashboard_values[my_predbat.prefix + ".load_energy_predicted"]["attributes"]["today_remaining"]
expected_delta = manual_adjust_kwh * 5 / float(my_predbat.plan_interval_minutes)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| my_predbat.load_today_comparison(load_minutes, load_forecast, {}, import_minutes, minutes_now=minutes_now, step=5, save=True) | ||
| baseline_remaining = my_predbat.dashboard_values[my_predbat.prefix + ".load_energy_predicted"]["attributes"]["today_remaining"] | ||
|
|
||
| # load_scaling_dynamic applied uniformly across every future minute at 2.0x - the whole | ||
| # remaining-today total should double, exactly like the flat load_scaling test does | ||
| my_predbat.load_scaling_dynamic = {minute: 2.0 for minute in range(minutes_now, 24 * 60, 5)} | ||
| my_predbat.load_today_comparison(load_minutes, load_forecast, {}, import_minutes, minutes_now=minutes_now, step=5, save=True) | ||
| dynamic_scaled_remaining = my_predbat.dashboard_values[my_predbat.prefix + ".load_energy_predicted"]["attributes"]["today_remaining"] | ||
|
|
||
| expected_dynamic = round(baseline_remaining * 2.0, 2) | ||
| if abs(dynamic_scaled_remaining - expected_dynamic) > 0.02: | ||
| print(" ERROR: today_remaining with load_scaling_dynamic=2.0 across the day should be ~{} (2x baseline {}), got {}".format(expected_dynamic, baseline_remaining, dynamic_scaled_remaining)) | ||
| failed = True | ||
| else: | ||
| print(" PASS: today_remaining scales with load_scaling_dynamic ({} -> {} at 2x)".format(baseline_remaining, dynamic_scaled_remaining)) |
…rough the day (#4519) load_today_comparison() applied load_scaling/load_scaling_dynamic/manual_load_adjust to future minutes only (#4506, #4511), which correctly fixed today_remaining but also fed load_total_pred - the accumulator the whole-day "today" total is read from - a mix of unscaled past buckets and scaled future ones. As minutes_now advanced, buckets kept flipping from the scaled group to the unscaled one, so the published "today" total (and its chart) drifted by whatever load_scaling wasn't 1.0, even though nothing about the underlying forecast changed. Adds a separate, consistently-scaled accumulator (load_total_pred_day) used only for load_energy_predicted's state/today/today_so_far/today_remaining/results, leaving load_total_pred itself (and the actual-vs-predicted divergence ratio it feeds) untouched.
Summary
Follow-up to #4506 on #4496 - the merged fix applied the flat
load_scalingfactor toload_today_comparison()'s predicted curve (fixing the reported gap for the original reporter), but missed two other multipliersstep_data_history()(fetch.py) also applies when building the plan's ownload_minutes_step:load_scaling_dynamic(per-minute) - carries saving-session/free-electricity-event scaling (load_scaling_saving/load_scaling_free), as well as any per-window override fromrates_import_overrideor the manual API.manual_load_adjust(additive, per-minute).Caught by a second reporter on the same issue after reviewing the #4506 diff directly - they had a 2-hour "power up" (free electricity) event set to
load_scaling: 1.5via the manual API, and correctly identified that the merged fix only accounted for the flat1.05baseload_scaling, not the dynamic per-window override.Verification
Downloaded and replayed their attached
debug.yamldirectly (not just read the code). Confirmedload_scaling_dynamicgenuinely carries1.5for the reported 12:00-14:00 window in their real config. Before this fix,load_today_comparison()never looked at that dict at all. After the fix, the same replay's plan-vs-sensor ratio drops to 1.0076 - matching the same small residual seen on the original #4506 replay (i.e. this closes the gap to the same baseline level as the first fix did for the original reporter).Test plan
test_load_today_comparison.py:today_remainingscales withload_scaling_dynamicapplied uniformly across the remaining day (mirrors the existing flatload_scalingtest), and reflects a singlemanual_load_adjustentry additively at the correct scaleoutput.py, reran, restored)load_today_comparisontests (None-guard, import-exceeds-load, flatload_scaling) pass unchangeddebug.yaml(see above), not just unit tests./run_all --quickpasses./run_pre_commitpasses🤖 Generated with Claude Code