Skip to content

fix: add delegation confirmation to prevent LLM confusion loops#2285

Open
mikeangstadt wants to merge 1 commit into
rtk-ai:developfrom
mikeangstadt:fix/llm-delegation-confusion-2280
Open

fix: add delegation confirmation to prevent LLM confusion loops#2285
mikeangstadt wants to merge 1 commit into
rtk-ai:developfrom
mikeangstadt:fix/llm-delegation-confusion-2280

Conversation

@mikeangstadt
Copy link
Copy Markdown

Problem

When RTK runs a delegated command like ruff check, the output shows the full command in the bash tool result (e.g. $ rtk ruff check ... --fix) followed by the filtered output (Ruff: No issues found). LLMs misinterpret the $ rtk prefix as a typo for the underlying tool (ruff), causing them to re-execute the command directly in an infinite loop.

Solution

After every successful filtered command, RTK now prints a delegation confirmation line:

rtk: <tool> done

This makes it unambiguous that rtk was intentional and the command successfully completed. The LLM sees both names in the same line and understands the delegation was correct.

Example output before

$ rtk ruff check src/main.py --fix
Ruff: No issues found

Example output after

$ rtk ruff check src/main.py --fix
Ruff: No issues found
rtk: ruff done

Testing

  • Unit tests on filter functions pass unchanged (the change is in the runner, not the filters)
  • The rtk: <tool> done line only appears for commands that succeed (exit_code == 0)
  • Commands using tee_label or no_trailing_newline paths are unaffected

Fixes #2280


Collaborated & Orchestrated by ClosedLoop.AI | www.closedloop.ai · github.com/rtk-ai

RTK rewrites (e.g. ruff check) output only the tool's filtered result
(e.g. 'Ruff: No issues found'), but the surrounding shell command
($ rtk ruff check ...) confuses LLMs into thinking they mistyped
'rtk' instead of 'ruff', causing infinite re-execution loops.

Add 'rtk: <tool> done' confirmation after every successful filtered
command so the LLM sees both rtk and the tool name in the same output
line, confirming the delegation was intentional and completed.

Fixes rtk-ai#2280
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RTK rewrites confuse LLM resulting in loops

1 participant