Skill injection matches the whole Bash command string: heredoc bodies, read-only commands, and even grepping the skill's own name force a MANDATORY injection

[fffokazaki]

Summary

pretooluse-skill-inject.mjs matches skill trigger patterns against the entire raw Bash command string, with no attempt to exclude non-code regions. As a result, a skill is force-injected (with a MANDATORY: ... You must run the Skill(...) tool banner) whenever its trigger word appears anywhere in the command text — including inside heredoc bodies, string literals, and even a grep for the skill's own name.

Version: vercel-plugin@0.24.0, Claude Code on macOS (darwin 25.5.0).

Root cause

hooks/pretooluse-skill-inject.mjs:300

const toolTarget = toolName === "Bash" ? toolInput.command || "" : toolInput.file_path || "";

The whole command string becomes the match target, and trigger patterns are word-boundary regexes (\bagent-browser\b etc.). Nothing distinguishes "the command does this" from "the command contains this text".

Reproduction

All four of these fired during a single session of ordinary backend work (a Next.js API route + tests). None of them involved a browser, a dev server, or Vercel env mutation.

1. Heredoc body — a string literal in a test fixture

cat > src/lib/__tests__/foo.test.ts <<'TS'
it('does not serve on localhost', () => {
  vi.stubEnv('NEXT_PUBLIC_APP_URL', 'http://localhost:3000')
  expect(resolve()).toEqual({ served: false })
})
TS

→ injected agent-browser ("dev server started, verify it visually"). Nothing was started; localhost:3000 is test data being written to disk.

2. Read-only listing

vercel env ls --scope myteam

→ injected env-vars. This command only lists variable names and targets; it prints no values and changes nothing.

3. Self-referential grep — the clearest case

grep -rlE "agent-browser" "$PLUGIN_DIR/hooks" 2>/dev/null | head -3

→ injected agent-browser. I was grepping the plugin's own hook directory to investigate this bug, and the investigation triggered the bug.

4. Background task completion notice

A <task-notification> for a finished background command whose text contained the word "workflow" injected the workflow skill (Vercel Workflow DevKit) via the UserPromptSubmit path, in a session with no workflow code.

Example of the emitted metadata for case 3:

{"version":1,"toolName":"Bash","matchedSkills":["agent-browser"],"injectedSkills":["agent-browser"],
 "reasons":{"agent-browser":{"trigger":"full","reasonCode":"pattern-match"}}}

Why this matters

The injected block is not a hint — it is worded as a hard directive:

MANDATORY: Your training data for these libraries is OUTDATED and UNRELIABLE. ... You MUST open and read the official docs linked below BEFORE writing ANY code. ... You must run the Skill(agent-browser) tool.

So each false positive costs real context budget and pushes the agent toward an irrelevant tool. Because the strongest signal (MANDATORY) is attached to the least reliable trigger (substring presence), the practical outcome is that an agent learns to ignore the banner — which also degrades the true positives this feature exists for.

Suggested fixes

Roughly in order of value-per-effort:

1. Do not match inside heredoc bodies and quoted strings. A minimal version: strip <<'EOF' ... EOF / <<EOF ... EOF blocks from toolTarget before matching. This alone removes case 1, which is the most common shape (writing test fixtures and docs).
2. Match on the command head, not the whole line. Most true positives are about what is being executed (npm run dev, vercel env pull), so matching the first token(s) of each pipeline segment would be far more precise than a whole-string scan. isDevServerCommand() at :217 already does something closer to this — the generic pattern path does not.
3. Exclude read-only invocations for mutation-oriented skills. vercel env ls / vercel env --help should not trigger the same guidance as vercel env add.
4. Skip when the match is the skill's own name in a path or a search pattern (grep, rg, find, ls arguments) — case 3.
5. Reserve MANDATORY for high-confidence triggers. Consider emitting a one-line "skill available" note for reasonCode: "pattern-match" and keeping the imperative wording for explicit user requests or verified command matches.

Happy to test a patch against the reproductions above if that helps.

[fffokazaki]

Follow-up: filing this issue reproduced the bug three more times.

The gh issue create call wrote the report body via a heredoc. That body contains npm run dev and vercel env pull as examples inside prose. The hook matched them as if I had run them:

{"toolName":"Bash",
 "matchedSkills":["agent-browser-verify","agent-browser","ai-gateway","verification","nextjs","env-vars"],
 "injectedSkills":["agent-browser-verify","verification","ai-gateway"],
 "reasons":{
   "agent-browser-verify":{"trigger":"dev-server-start","reasonCode":"bash-dev-server-pattern"},
   "verification":{"trigger":"dev-server-companion","reasonCode":"dev-server-co-inject"},
   "ai-gateway":{"trigger":"full","reasonCode":"pattern-match"}}}

Three things this adds to the original report:

1. isDevServerCommand() has the same flaw. I suggested in the issue body that matching the command head would be more precise, and pointed at isDevServerCommand() (:217) as the closer-to-correct path. It is not — it also runs DEV_SERVER_PATTERNS.some((re) => re.test(command)) against the whole string, so \bnpm\s+run\s+dev\b matched prose. Fix suggestion 2 should apply there too.

2. A false positive cascades. agent-browser-verify fired on the phantom dev server, which then co-injected verification via dev-server-co-inject. One bad match pulled in a second skill that had no trigger of its own.

3. The hook emitted a stateful marker for an event that never happened: <!-- marker:dev-server-verify iteration="1" max="2" -->. No server was started — a file was written. Anything keyed off that counter is now counting phantom starts.

The general shape: any command that writes text about Vercel — a doc, a test fixture, a commit message, a bug report about this plugin — is indistinguishable from a command that does the thing. cat > file <<'EOF' and git commit -m are the two everyday shapes that hit it, and both are pure text authoring.

Stripping heredoc bodies (fix suggestion 1) would have prevented all three of these, and cases 1 and 3 in the original report.