Skip to content

fix(registry): only rewrite last pipe segment; prevent data corruption in pipes/redirects#2274

Open
youbamj wants to merge 1 commit into
rtk-ai:developfrom
youbamj:fix/rewrite-last-pipe-segment
Open

fix(registry): only rewrite last pipe segment; prevent data corruption in pipes/redirects#2274
youbamj wants to merge 1 commit into
rtk-ai:developfrom
youbamj:fix/rewrite-last-pipe-segment

Conversation

@youbamj
Copy link
Copy Markdown
Contributor

@youbamj youbamj commented Jun 5, 2026

Summary

Fixes critical data corruption bugs where RTK incorrectly filters piped or redirected commands, breaking downstream tools and producing malformed output files.

Real Bugs Fixed

Bug 1: Large JSON truncated, breaks jq

$ rtk curl https://jsonplaceholder.typicode.com/posts | jq
jq: parse error: Unfinished string at EOF at line 14, column 0

Root cause: RTK filters curl output, truncating large JSON responses. The truncated data is piped to jq, which fails to parse incomplete JSON.

Bug 2: Git diff corrupted when redirected to file

Before (RTK-filtered patch - BROKEN):

README.md | 2 ++
 1 file changed, 2 insertions(+)

--- Changes ---

README.md
  @@ -2,6 +2,8 @@
  +Change
  +
   <p align="center">
     <strong>High-performance CLI proxy that reduces LLM token consumption by 60-90%</strong>
   </p>
  +2 -0

After this fix (real patch - CORRECT):

diff --git a/README.md b/README.md
index d91e21b..dabe2cd 100644
--- a/README.md
+++ b/README.md
@@ -2,6 +2,8 @@
   <img src="https://avatars.githubusercontent.com/u/258253854?v=4" alt="RTK - Rust Token Killer" width="500">
 </p>
 
+Change
+
 <p align="center">
   <strong>High-performance CLI proxy that reduces LLM token consumption by 60-90%</strong>
 </p>

Result: git apply /tmp/rtk-test.patch fails with "No valid patches in input" on the RTK version, but succeeds on the real version.

Algorithm

  1. Find last | in entire command
  2. Everything before last | → stays raw (intermediate pipe segments)
  3. Everything after last | → normal rewrite rules:
    • Check for file redirect (>, >>, &>) → skip entirely
    • Check if supported → prepend rtk if yes
  4. Input redirect < is fine (reading from file, output still to terminal)

Examples

Input Output Reason
git log | cat && git stash git log | rtk cat && rtk git stash Only last segments get rtk
cargo test 2>&1 | head > file None Redirected to file, skip
cat < file.txt rtk read - < file.txt Input redirect is OK
find . | xargs grep pattern find . | xargs rtk grep pattern xargs is transparent
curl ... | jq None Pipe chain stays raw

Changes

  • src/discover/registry.rs (+198, -62 lines)
    • Refactored rewrite_compound() with last-pipe detection
    • Extracted rewrite_segments() helper
    • Added has_stdout_redirect() (checks >, >>, &> only)
    • Fixed < input redirect to not block rewrite
    • Added xargs to SHELL_PREFIX_BUILTINS
    • Added 14 new/updated tests

Testing

cargo test        # 2016 passed, 0 failed
cargo fmt --all   # clean
cargo clippy --all-targets  # clean

Breaking Change

This changes the behavior of rtk rewrite for piped commands. Previously all commands in a pipe got rtk prefix; now only the last segment does. This is the correct Unix behavior but may surprise users relying on the old (broken) behavior.

Implement Unix philosophy: when commands are piped, only the last
segment outputs to terminal. All commands before the last pipe stay
raw; only the portion after gets RTK processing.

Changes:
- rewrite_compound(): find last | globally, leave before raw,
  process after with normal per-segment logic
- has_stdout_redirect(): only block on > >> &> (not < input redirect)
- cat < file.txt: special case → rtk read - < file.txt
- xargs: add as transparent prefix (xargs grep → xargs rtk grep)
- Add tests for pipe chains with &&/||/; operators
- Add tests for input redirect (<) behavior

Fixes stdout passthrough for piped/redirected commands.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant