Roll up release - #135
Closed
andig wants to merge 5 commits into
Closed
Conversation
andig
marked this pull request as draft
August 7, 2026 08:13
This was referenced Aug 7, 2026
andig
changed the title
The cost neutral tie breakers, not the physics and not the horizon, drive the requests that exhaust OPTIMIZER_TIME_LIMIT. On a flat tariff the cost optimum is a plateau of equally priced schedules, and the strategy terms are far too small to decide between them, so the search walks the plateau instead. OPTIMIZER_GAP_ABS, one cent, stops the search once the remaining gap is worth less than a cent. OPTIMIZER_STRATEGY_WEIGHT, 3, lifts the strategy terms far enough above the cost terms to actually decide the tie. Neither is enough alone: the gap collapses the plateau walk but leaves the tie to wherever the search stopped, the weight decides the tie but still searches. To support the weight, the objective is assembled as cost_objective plus preference_objective, with a test asserting the split stays exhaustive.
OPTIMIZER_GAP_ABS bounds the whole objective, preferences included, and the preferences are orders below any useful gap: measured over the golden cases the entire weighted preference objective is worth less than one cent on 8 of 19, by up to 149x on 018-high-soc-initial. Inside one objective the gap therefore swallows the tie break, and OPTIMIZER_STRATEGY_WEIGHT cannot buy it back: 3 gives a factor 3 against a shortfall of 3 to 150, and 10 already costs 0.4 percent of real money on 023. So the two are solved in sequence instead. The first stage maximizes cost_objective and may stop OPTIMIZER_GAP_ABS short, which is where the speed comes from. The second maximizes preference_objective under a constraint that keeps the money the first stage found, which is where the strategies are decided. Strategy adherence becomes something the model solves for rather than something that emerges from the ratio between coefficients, so the weight and its cliff table are gone. OPTIMIZER_PREFERENCE_BUDGET replaces the weight and defaults to zero, the cost neutrality the strategies are documented to have. It is not a knob to reach for: the second stage is indifferent to money below the bound, so every unit granted is spent, measured exactly at budgets of 1 and 100. Both stages share one wall clock, so the second solve cannot double the response time, and anything short of Optimal there falls back to the first stage schedule whole. A timed out tie break costs preference, never money. 019-unexpected-charge-spikes is refreshed. Its money is unchanged to the last digit, only the reported objective_value moves, the s[0] artifact documented in AGENTS.md. test_app gains an atol of a hundredth of a cent, below which the reported value only carries the slack the cost bound needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three things the slow request dump brought out, none of them visible on the golden cases alone. CBC returns a strictly worse schedule and reports it as proven optimal when the first stage solution is handed to it as a MIP start, and it calls the model infeasible over a cost bound that same solution satisfies. Measured on request 4f28656db0ba: preference -0.806 warm against -0.610 cold, where cold matches a single joint solve to the last digit. No warm start, and the infeasible runs that motivated turning presolve off disappear with it, so presolve goes back to its default. Deciding the tie to proven optimality is its own hard problem, as expensive as the cost optimum on exactly the requests this is meant to help: solved cold to the end it puts the median back to 10.05 s over the dump. So the stage gets a quarter of the time limit rather than whatever is left, and what it does not finish is kept anyway. A tie break stopped by the clock still holds an incumbent, and the alternative is the first stage schedule, which is no tie break at all. The improvement and the money are checked here rather than read off the solver status. That check needs a tolerance of its own. CBC treats the cost bound like any other row and misses it by its feasibility tolerance, 1.2e-5 on 024-attenuate-demand-peaks, where rejecting over it threw away a tie break worth four times the peak. It is also what 024 needs: levelling a peak costs round trip efficiency, about 3.5e-5 there, so the strategies are not quite the cost neutral tie breakers the docs claim. 019-unexpected-charge-spikes is back to its stored expectation, no golden case moves. test_app keeps an atol of a hundredth of a cent, which is what the bound gives away. Over 335 unique requests from the slow request dump, at the production limit of 10 s: median 7.65 s to 2.99 s, requests over 10 s 149 to 52, and against main the strategies come out ahead on 120 requests and behind on 79, where the weighted variant of the previous commit is behind on 112 and ahead on 51. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The split costs a second CBC spawn on every request with a strategy set, and buys nothing on a request that was never degenerate: a model that proves the joint objective decided its own tie, in one solve, with no gap and no preference budget to give anything away. Measured on the stored cases, that is a median of 0.114 s against 0.052 s on main, a doubling paid by all traffic to help the one percent that reaches the time limit. So probe first. Solve the whole objective with no gap on a fifth of the time limit; if it proves optimality, stop there. Only a probe that runs out falls back to the money-then-preferences split, which is the case the split was built for. On the stored cases the median returns to main's 0.052 s, and 020-weird-charging-at-night gets faster than either, 0.53 s against 0.93 s, because the probe correctly routes it to the split. sol_status, not status: pulp reports LpStatusOptimal whenever CBC returns any feasible solution, including one it stopped on at the time limit. On a captured request a 2 s and a 30 s run both said Optimal, with objectives of -682466848 and 59714881. Reading status alone made the probe accept every truncated search and quietly cut the whole population off at the probe budget. Over 335 captured slow requests the probe is a loss, as it should be on a population that is entirely hard: median 4.89 s against 3.03 s for the unconditional split, with 214 of 335 falling through. Requests over the limit stay far below main, 51 against 149. Known gap: one request of the 335 comes back Not Solved, where the probe finds no feasible solution and leaves the cost stage too little clock to find one either. A shorter probe does not fix it. The probe's incumbent is kept when it has one, which covers the rest. OPTIMIZER_PROBE_SECONDS tunes the budget, 0 restores the unconditional split. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The second stage decides whether to keep its result by reading the variables back, and a solve that ran out of clock before it found an integer solution leaves the relaxation in them. That point scores better on the preferences than any real schedule can, because it is one the model forbids: the binaries land between 0 and 1 and every rule they gate stops holding. So the tie break kept it, overwrote the first stage schedule, and the response went out with the charge power gate ignored in a third of its slots and a status of Feasible. Reported in #127, where an EV with c_min=4140 charged at a fifth of that in 14 of 192 slots, z_c sitting between 0.12 and 0.36. It reproduces on the split path whenever the preference stage finds nothing before its slice runs out. Integrality is now checked where the other two acceptance conditions already are, for the reason stated there: a solver that reports the wrong status must not be able to spend money, and it must not be able to hand back a schedule the model does not allow either. A second check sits right before the result is built, standing in for every stage above deciding correctly. A schedule that breaks the model is worse than no schedule, since it looks like an answer and the caller charges a battery by it, so a fractional point is reported as no schedule at all rather than returned. One pass over the integer variables against a solve measured in seconds. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
andig
force-pushed
the
rollup/two-stage-plus-130
branch
from
August 8, 2026 14:12
0397ee4 to
6ab54bd
Compare
Member
Author
|
All merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rollup branch off
maincombining:perf/two-stage-solve— two stage solve, cost objective first, preferences secondMerge conflicts resolved in two places, both from the same overlap: #130 rewrote the leveling terms that
perf/two-stage-solvehad moved into the preference objective.src/optimizer/optimizer.py— the peak and level deviation terms from Drop the step to step ramp from the leveling strategies #130 now add topreference, notobjective, so they stay in the second stage.docs/comparison_objective_terms.md— kept both the tier 3 two stage paragraph and Drop the step to step ramp from the leveling strategies #130's ramp → level deviation wording.Full test suite passes (124 passed).