Skip to content

test(util): regression coverage for parallel tool_calls - #479

Open
Rubyglask wants to merge 1 commit into
aliasrobotics:mainfrom
Rubyglask:test/sanitize-message-list-parallel-tool-calls
Open

test(util): regression coverage for parallel tool_calls#479
Rubyglask wants to merge 1 commit into
aliasrobotics:mainfrom
Rubyglask:test/sanitize-message-list-parallel-tool-calls

Conversation

@Rubyglask

Copy link
Copy Markdown

Summary

Adds regression tests for cai.util.terminal.sanitize_message_list (exported as
fix_message_list) covering assistant turns that issue multiple parallel
tool_calls
whose responses arrive out of order. This is test-only — no
runtime change.

Why

sanitize_message_list runs on every model request. Its second-pass sequence
check compares each tool message only against its immediate predecessor, so
when an assistant emits 2+ tool calls a sibling tool response can be judged
out-of-order and reordered without the loop index ever advancing — an unbounded
loop that presents as an indefinite hang (issues #469, #401, #410; earlier fix
attempts #411/#424/#449). main already fixes this: the reorder loop now tracks
processed positions and always makes forward progress. This PR adds coverage so
that guard can't be dropped again in a future refactor (roughly how the loop
slipped in originally).

What's already covered vs. what this adds

tests/cli/test_tool_visualization.py::test_fix_message_list_no_duplicate_tool_responses
already exercises an in-order 3-way parallel turn and asserts no
dropped/duplicated responses. What it doesn't cover — and what this adds:

  • out-of-order / scrambled sibling responses (the shapes that actually
    triggered the loop; in-order never did),
  • explicit termination guards (@pytest.mark.timeout), so a regressed build
    fails fast instead of hanging the whole suite,
  • a long-history case (200 parallel pairs),
  • the renamed sanitize_message_list symbol and its fix_message_list alias.

Tests — tests/util/test_sanitize_message_list.py

  • test_parallel_tool_calls_terminate_and_stay_valid — 6 shapes: in-order,
    reversed, 3-way scrambled, two interleaved assistants, the CAI hangs indefinitely when a single turn issues multiple tool_calls (not covered by #370 idle-timeout) #469 two-calls-in-
    one-turn shape, and a repeated sibling response. Asserts termination, a valid
    tool/assistant ordering, and that no tool response is dropped. It deliberately
    does not assert order-preservation — main legitimately reorders even
    already-valid input, and each tool response only has to follow its owning
    assistant.
  • test_long_history_of_parallel_pairs_terminates — 200 reversed parallel pairs.
  • test_tool_message_without_owning_assistant_is_repaired — the index-0 repair
    branch.
  • test_fix_message_list_alias_is_preserved — the public alias contract.

All pass on main. The parallel cases time out (fail) if the reorder loop's
position guard is removed, so the suite pins the fix in place.

Notes

Pure tests, no new dependencies. Happy to adjust placement or naming.

Guards sanitize_message_list / fix_message_list against the parallel tool_calls reorder hang (see aliasrobotics#469). Test-only, no runtime change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant