Skip to content

Improves nextcheck result and fixes nextcheck bug#123

Open
merschformann wants to merge 15 commits intodevelopfrom
merschformann/improve-nextcheck-analysis
Open

Improves nextcheck result and fixes nextcheck bug#123
merschformann wants to merge 15 commits intodevelopfrom
merschformann/improve-nextcheck-analysis

Conversation

@merschformann
Copy link
Member

@merschformann merschformann commented Feb 11, 2026

Description

This PR means to improve nextcheck output by distinguishing plannable and executable moves, i.e., a plannable move can be executed AND improves the solution, while an executable move can be executed but does not improve the solution.
Furthermore, fixes a bug in nextcheck where planning_makes_objective_worse was flipped. It now accurately expresses whether the best move for a plan unit improves the objective or not.
At last, we add objective_deltas that list the delta / improvement of each objective term defined for all best moves found.

Changes

  • Introduces has_executable_best_move for a plan unit indicating whether the best move for a plan unit could be executed, i.e., did not violate any constraint.
  • Introduces was_executable for a move on a vehicle indicating whether the respective move was executable, i.e., did not violate any constraint.
  • Fixes a bug where planning_makes_objective_worse was incorrectly stating whether a move made the objective worse.
  • Introduces objective_deltas showing the delta of each objective term for all best moves found (only available in high verbosity).
  • Fixes breaking changes in nextmv Python dependency for tests to work again.

Preview

The new features can to some extent already be seen in the golden file test expectation update (tests/check/input.json.golden) part of this PR.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the nextcheck output schema to distinguish between executable vs plannable best moves, fixes an inverted planning_makes_objective_worse interpretation, and updates Python test scaffolding for nextmv>=1.0.0.

Changes:

  • Add has_executable_best_move / was_executable flags to report moves that satisfy constraints even if they don’t improve the objective.
  • Add objective_deltas (high verbosity) to show per-objective-term estimated deltas for candidate moves.
  • Update Python golden-test runner and dependency versions to align with nextmv API changes.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
check/check.go Computes new executable/plannable flags and emits objective_deltas; adjusts when planning_makes_objective_worse is set.
check/schema/schema.go Extends JSON schema with new fields for executability and per-term objective deltas.
tests/check/input.json.golden Updates expected nextcheck JSON output to include the new fields.
src/tests/solve_golden/main.py Updates test harness to new nextmv API (Option, load, write).
requirements.txt Bumps nextmv dependency to >=1.0.0.
.github/workflows/python-test.yml Updates the Python CI matrix (3.9 → 3.10).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

merschformann and others added 2 commits February 12, 2026 00:34
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

check/check.go:369

  • End-of-loop aggregation doesn’t appear to handle the “executable but not plannable” case described in the PR:
  • PlanningMakesObjectiveWorse is only set when hasWorseMove becomes true (which currently requires executing an estimated-improving move that then doesn’t plan). If all executable best moves are non-improving (bestMove.IsImprovement() == false), this flag stays false even though the best executable move worsens the objective.
  • The summary path treats HasPlannableBestMove == false as “no move” and increments PlanUnitsHaveNoMove, even when HasExecutableBestMove is true.
    Consider basing the “no move” summary on HasExecutableBestMove (or introducing a separate counter), and setting PlanningMakesObjectiveWorse whenever an executable move exists but no plannable/improving move exists (using the best executable move’s value/verification logic).
			if !hasImprovingMove && hasWorseMove {
				m.output.PlanUnits[solutionPlanUnitIdx].PlanningMakesObjectiveWorse = true
				m.output.Summary.NumberOfPlanUnitsMakingObjectiveWorse++
			}
			if m.output.PlanUnits[solutionPlanUnitIdx].HasPlannableBestMove {
				m.output.Summary.PlanUnitsBestMoveFound++
			} else {
				m.output.Summary.PlanUnitsHaveNoMove++
				constraints := make(map[string]int, len(m.solution.Model().Constraints()))
				m.output.PlanUnits[solutionPlanUnitIdx].Constraints = constraints
				for _, constraint := range planUnitsObserver.Constraints() {
					name := fmt.Sprintf("%v", constraint)
					if _, ok := constraints[name]; !ok {
						constraints[name] = 0
					}
					constraints[name]++
				}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@merschformann merschformann marked this pull request as ready for review February 13, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments