Agent-bilevel: horizon-faithful plan capture + explorer fixes (solves plain domino 5/5 across 5 seeds)#42
Merged
Conversation
Stop forward simulation at the first failing option (stop_on_failure) and reject plans whose goal is only reached beyond CFG.horizon low-level steps, so capture no longer reports goals the real closed-loop rollout never reaches. Surfaces NOT EXECUTABLE so the agent shortens the plan instead of stopping on a false positive.
…plore-task tools Point the interactive tools at the explore task (not a stale test task), recover tool-validated capture, and seed the info-gain search.
Add agent_sdk_reasoning_effort setting plumbed through the local/docker sandboxes, and allow including built-in tools in the solve-prompt tool names.
yichao-liang
added
the
solved: domino 5/5 across 5 seeds
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✅ Result: full method solves plain domino in 5/5 tasks across 5/5 seeds (25/25), no oracle samplers
Clean re-cut of #41 off current
masterso the commit list reflects only the new work (4 commits, ~400 lines across 11 files) instead of the 276-commit list #41 inherited from squash-merge history. Same net diff; supersedes #41.Plan capture mirrors the real executor
stop_on_failure=True): abort at the first failing option (0-action collision / not-initiable / env failure) instead of forward-simulating past it.CFG.horizonlow-level steps is reportedNOT EXECUTABLEand not captured, so the agent shortens the plan instead of stopping on a false positive.Explorer fixes
Config / tooling
agent_sdk_reasoning_effortsetting, plumbed through local/docker sandboxes.horizonraised 400 → 500.Tests
tests/agent_sdk/test_bilevel_sketch_samplers.pytests/explorers/test_agent_bilevel_explorer.py