Skip to content

fix(uniswap-integrations): Opus 5 migration audit - #572

Merged
wkoutre merged 1 commit into
nextfrom
nickkoutrelakos/opus5-uniswap-integrations-v2
Aug 5, 2026
Merged

fix(uniswap-integrations): Opus 5 migration audit#572
wkoutre merged 1 commit into
nextfrom
nickkoutrelakos/opus5-uniswap-integrations-v2

Conversation

@wkoutre

@wkoutre wkoutre commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Opus 5 migration audit of packages/plugins/uniswap-integrations/. Every .md under the plugin was read in full except skills/github-setup/SKILL.md.

skills/github-setup/SKILL.md is deliberately untouched — PR #565 owns it (credential-leak fix). That PR bumps the plugin to 2.6.1 on its own branch; this branch is off next where the plugin was still 2.6.0, so it bumps 2.6.0 → 2.7.0. The version collision is expected and resolves at merge.

Deltas addressed

Broken references (verified against the live MCP tool surfaces, not recalled)

  • skills/refine-linear-task/SKILL.mdallowed-tools declared mcp__linear__linear_search_issues, mcp__linear__linear_update_issue, mcp__linear__linear_add_comment. None of those exist on https://mcp.linear.app/mcp, the server this plugin configures in .mcp.json; they are third-party-server names. Replaced with the real surface (get_issue, list_issues, save_issue, save_comment), and named the specific tool at the two body steps that previously said only "via Linear MCP". Every other Linear reference in this repo already uses the unprefixed form, so this file was the outlier.
  • skills/daily-standup/standup-guide.md:171mcp__github__get_pull_request is not a tool on the GitHub MCP (api.githubcopilot.com/mcp); PR reads go through pull_request_read. Fixed. (mcp__github__search_issues and mcp__linear__list_issues, also referenced in this skill, are real and were left alone.)
  • skills/orchestrate-deployment/deploy-guide.md:4agent-orchestrator-agent was dispatched bare, but it lives in the development-codebase-tools plugin, so it needs the plugin-qualified form development-codebase-tools:agent-orchestrator-agent. Fixed in the frontmatter and in the fallback note.

Invented numbers

  • deploy-guide.md output shape required costs: { estimated: string; // Monthly cost estimate }. Nothing upstream in the skill queries a pricing source, so a model filling that required field fabricates a dollar figure that then reads as measured. Made the field optional, added a required source, and stated explicitly not to derive a figure from recalled rate cards.

Output length (Delta 4)

Effort settings do not shorten authored deliverables; only explicit length instructions do. Both report-generating skills had multi-section unbounded output formats:

  • skills/datadog-cost-tracker/SKILL.md (specifically flagged) — bounded to ~400 words outside tables, with "don't restate the table in prose".
  • skills/investigate-incident/SKILL.md — same defect class, bounded to ~500 words outside quoted evidence.

Delegation (Delta 2)

  • deploy-guide.md Phase 2 mandated parallel dispatch of two agents with no opt-out. Restated as a ceiling (at most one of each) and added a when-NOT-to-delegate rule for phases that finish in a handful of direct tool calls.

Findings in the brief that do not exist in this plugin

Reported rather than papered over:

  1. git add -A in linear-task-and-pr-from-changes.md:308 — that file is packages/plugins/development-pr-workflow/commands/, not this plugin. This plugin has no commands/ directory at all.
  2. mcp__linear__create_issue in 2 commands — same: both occurrences are in development-pr-workflow (linear-task-and-pr-from-changes.md:188, start-linear-task.md:190). The finding is real, the plugin is not this one.
  3. --skip-graphite in start-linear-task.md:406 — also development-pr-workflow.
  4. mcp__nx_mcp__ (underscores) — searched; zero occurrences anywhere in this plugin.

All three files should be picked up by whoever takes development-pr-workflow.

Deltas that produced no changes

  • Delta 1 (verification ceremony) — none present. Every verify/Verify in the plugin is either a grounding instruction (use-datadog's "check pup is installed and authenticated"; the cost-tracker's "retrieve the metric's tag context before concluding a service has zero ingestion") or a real deployment/rollback gate in the three agents. All kept per the brief.
  • Delta 3 (literal following / recall filters) — no discovery-time filters, and the single **IMPORTANT** in the plugin (standup-guide.md:164, "prompt for GitHub username FIRST before fetching any data") is a real ordering constraint, not decoration. use-datadog's "Critical Rules" heading covers genuine constraints (APM durations are nanoseconds, always pass --from). Left alone.
  • Stale model facts — searched for claude-opus-4, claude-sonnet-4, haiku-3, MultiEdit, budget_tokens, and the "~80% cheaper / 5x cheaper" pricing claim. Zero hits. The two model: values present (haiku, claude-haiku-4-5, sonnet) are current and are frontmatter, so untouched per the brief.
  • agents/cicd-agent.md, agents/infrastructure-agent.md, agents/migration-assistant.md — audited in full. All three are generic domain-reference documents with no migration defects. Their frontmatter name: values (cicd-agent, infrastructure-agent, migration-assistant-agent) match how CLAUDE.md and README.md list them, and match how orchestrate-deployment dispatches them.
  • CLAUDE.md / README.md — no documented behavior drifted. The MCP server tables, skill lists, and agent lists all still match plugin.json and disk. Only the root CLAUDE.md version-table row changed.

Test plan

Version sources agreed before the bump (checked rather than assumed, since a sibling plugin was reported drifted):

$ grep -n '"version"' packages/plugins/uniswap-integrations/.claude-plugin/plugin.json
3:  "version": "2.6.0",
$ grep -n "uniswap-integrations" CLAUDE.md
233:| uniswap-integrations       | 2.6.0   |

Both now read 2.7.0. MINOR, because correcting a fabricated MCP tool name changes runtime behavior.

Plugin structure validation:

$ node scripts/validate-plugin.cjs packages/plugins/uniswap-integrations
  ✓ plugin.json: uniswap-integrations v2.7.0
  ✓ skills/: 7 item(s)
  ✓ agents/: 3 item(s)
  ✓ .mcp.json: 11 MCP server(s)
Validation PASSED

Markdown lint (repo config, plugin scope):

$ bunx markdownlint-cli2 --fix "packages/plugins/uniswap-integrations/**/*.md"
markdownlint-cli2 v0.20.0 (markdownlint v0.40.0)
Linting: 200 file(s)
Summary: 0 error(s)

Format:

$ bunx nx format:write --uncommitted

Lefthook pre-commit ran the full gate on the staged set and passed:

✔️ format (1.30 seconds)
✔️ lint (0.13 seconds)
✔️ lint-markdown (1.05 seconds)
✔️ test (0.61 seconds)
✔️ typecheck (0.59 seconds)
✔️ update-lockfile (1.01 seconds)

Confirmed github-setup/SKILL.md is absent from the diff:

$ git diff --name-only origin/next...HEAD
CLAUDE.md
packages/plugins/uniswap-integrations/.claude-plugin/plugin.json
packages/plugins/uniswap-integrations/skills/daily-standup/standup-guide.md
packages/plugins/uniswap-integrations/skills/datadog-cost-tracker/SKILL.md
packages/plugins/uniswap-integrations/skills/investigate-incident/SKILL.md
packages/plugins/uniswap-integrations/skills/orchestrate-deployment/deploy-guide.md
packages/plugins/uniswap-integrations/skills/refine-linear-task/SKILL.md

🤖 Generated with Claude Code

@wkoutre
wkoutre requested a review from a team as a code owner August 5, 2026 20:13
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ai-toolkit-slack-oauth-backend Ready Ready Preview Aug 5, 2026 11:41pm

Request Review

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check ✅

Verdict: Passed

The plugin version was bumped in both packages/plugins/uniswap-integrations/.claude-plugin/plugin.json (2.6.1 → 2.7.0) and the root CLAUDE.md version table, satisfying the blocking requirement. No other docs (README, plugin CLAUDE.md) reference the corrected tool names or schema, so nothing else went stale.


PR #572 fixes several broken/fictitious MCP tool references across uniswap-integrations skills (mcp__github__get_pull_requestmcp__github__pull_request_read; fabricated Linear tool names → real mcp__linear__get_issue/list_issues/save_issue/save_comment), corrects the cross-plugin agent dispatch name for agent-orchestrator-agent, adds output-length guidance to two SKILL.md files, and tightens the deploy-guide cost-estimate schema to require a real pricing source instead of a fabricated figure.

Version bump: present and consistent — plugin.json and the root CLAUDE.md table both show 2.6.1 → 2.7.0. This satisfies the CRITICAL/blocking rule.

Semver level (non-blocking observation): every change in this diff is a correction to broken tool references or added prompt guidance for existing skills — no new skill, agent, command, or MCP server was added. Per this repo's own CLAUDE.md definitions ("Patch: bug fixes... Minor: new skills/agents/commands/features"), this reads more like a patch-level change (e.g., 2.6.2) than a minor one. Not blocking since a bump did occur, but flagged as a suggestion.

README / plugin CLAUDE.md: no stale references to the old (now-fixed) tool names remain in packages/plugins/uniswap-integrations/README.md or CLAUDE.md, so no further updates are required there.

Changelog: none present; informational only per the validator's own rules.

Suggestions (1)

💡 Inline suggestions have been posted as review comments. Click "Commit suggestion" to apply each fix directly.

  • ℹ️ packages/plugins/uniswap-integrations/.claude-plugin/plugin.json: All changes in this PR are corrections to broken/incorrect tool references and minor guidance additions to existing skills—no new skill, agent, command, or MCP server was added. Per the repo's own semver rules in CLAUDE.md (patch = bug fixes, minor = new features/components), a patch bump (2.6.2) would more accurately reflect the change than the minor bump (2.7.0) that was used. Not blocking since a version bump was performed.

🤖 Generated by Claude Documentation Validator | Mode: suggest

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

● Reviewed · 2026-08-05 23:41 UTC · 2 reviews · view run ↗

Approved — one warning in deploy-guide.md, two open questions on the Linear write path.

Corrects fabricated MCP tool names in three uniswap-integrations skills, bounds report output length in two more, makes the deploy cost estimate optional-with-source, and bumps the plugin 2.6.1 → 2.7.0.

Assessment

The tool-name fixes and the anti-fabrication guard on the cost field change runtime behavior for the better, and the version bump is coherent across plugin.json and the root table. The gap is in orchestrate-deployment: the plugin-qualified agent-orchestrator-agent grant landed in deploy-guide.md's frontmatter, which isn't a registered surface — the enforced allowed-tools in SKILL.md still omits it, so the dispatch remains ungranted at runtime.

Suggestions

  • Orchestrator grant landed on an unenforced file. deploy-guide.md is a reference doc; the enforced permission list is orchestrate-deployment/SKILL.md:3, which still grants only cicd-agent and infrastructure-agent. Add the qualified agent-orchestrator-agent entry there for the fix to take effect. view inline thread →
  • Delegation cap contradicts later phases. "At most one cicd-agent per deployment" conflicts with the Phase 3 and Phase 4 dispatches at lines 99/116/131 and 148; scope the ceiling to the Phase 2 split.
  • save_issue/save_comment naming. These names appear nowhere else in the repo; if they're wrong, refine-linear-task fails at write time after user approval. Worth confirming against the live mcp.linear.app tool list, and passing the Phase 1 issue id to save_issue either way.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9333327bec

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


1. **Update Issue Description** via Linear MCP
2. **Add Comment** (optional): Note the refinement
1. **Update Issue Description** via `mcp__linear__save_issue`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the issue id when saving

When the user approves an update, mcp__linear__save_issue is a create-or-update tool: it updates only when the fetched issue id is passed, and otherwise can create a new Linear issue. This step replaces the old update-only wording with save_issue but doesn't tell the skill to call it with the Phase 1 issue id, so a refinement for DEV-123 can duplicate the task while reporting that it was updated. Please make the save step explicitly pass the current issue id.

Useful? React with 👍 / 👎.

> **Note**: This skill uses **agent-orchestrator-agent** from the development-codebase-tools plugin when available. If not installed, the skill will execute infrastructure-agent and cicd-agent sequentially instead of in parallel coordination.
> **Note**: This skill uses **development-codebase-tools:agent-orchestrator-agent** when that plugin is installed. If not, execute infrastructure-agent and cicd-agent sequentially instead of in parallel coordination.

**Delegate only when the work warrants it.** The agent split below is a ceiling, not a quota: at most one infrastructure-agent and one cicd-agent per deployment. When a phase finishes in a handful of direct tool calls (reading an existing workflow file, a single `kubectl rollout status`, a dry-run with no infrastructure changes), do it inline rather than spawning an agent for it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ warning · correctness — "at most one infrastructure-agent and one cicd-agent per deployment" contradicts the rest of this guide: cicd-agent is dispatched in Phase 2 (line 78), again in Phase 3 for the chosen strategy (lines 99/116/131), and again in Phase 4 for observability (line 148). A model reading the cap as written has to drop one of those instructed dispatches. Scope it to the Phase 2 split, which is the fan-out this PR is actually bounding.

Suggested change
**Delegate only when the work warrants it.** The agent split below is a ceiling, not a quota: at most one infrastructure-agent and one cicd-agent per deployment. When a phase finishes in a handful of direct tool calls (reading an existing workflow file, a single `kubectl rollout status`, a dry-run with no infrastructure changes), do it inline rather than spawning an agent for it.
**Delegate only when the work warrants it.** The Phase 2 split below is a ceiling, not a quota: at most one infrastructure-agent and one cicd-agent for that phase. When a phase finishes in a handful of direct tool calls (reading an existing workflow file, a single `kubectl rollout status`, a dry-run with no infrastructure changes), do it inline rather than spawning an agent for it.

---
description: Refine and enhance Linear task descriptions. Use when user says "refine this Linear task", "improve task description", "make this task clearer", "enhance Linear issue", or needs to improve clarity, completeness, and actionability of Linear issues.
allowed-tools: Read(*), Glob(*), Grep(*), Task(subagent_type:Explore), WebSearch(*), WebFetch(*), mcp__linear__linear_search_issues(*), mcp__linear__linear_update_issue(*), mcp__linear__linear_add_comment(*)
allowed-tools: Read(*), Glob(*), Grep(*), Task(subagent_type:Explore), WebSearch(*), WebFetch(*), mcp__linear__get_issue(*), mcp__linear__list_issues(*), mcp__linear__save_issue(*), mcp__linear__save_comment(*)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 info · correctnesssave_issue/save_comment appear nowhere else in this repo; every other Linear reference uses the create_issue/get_user/list_teams family (e.g. development-pr-workflow/commands/start-linear-task.md:4), which is the shape update_issue/create_comment would follow. If these two names are wrong, Phase 5 fails at write time after the user has approved the refinement — worth confirming against the live mcp.linear.app tool list before merge.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Approved — see full review in the sticky comment ↑

@wkoutre
wkoutre force-pushed the nickkoutrelakos/opus5-uniswap-integrations-v2 branch from 9333327 to cedbc3e Compare August 5, 2026 23:26
argument-hint: <target> [--strategy blue-green|canary|rolling] [--environment dev|staging|prod] [--dry-run]
allowed-tools: Read(*), Write(*), Task(subagent_type:cicd-agent), Task(subagent_type:infrastructure-agent), Task(subagent_type:agent-orchestrator-agent)
# Note: agent-orchestrator-agent is from development-codebase-tools plugin (optional - see fallback below)
allowed-tools: Read(*), Write(*), Task(subagent_type:cicd-agent), Task(subagent_type:infrastructure-agent), Task(subagent_type:development-codebase-tools:agent-orchestrator-agent)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ warning · correctness — This frontmatter isn't enforced: deploy-guide.md is a reference doc linked from SKILL.md:78, not a registered skill or command. The enforced surface is orchestrate-deployment/SKILL.md:3, which still grants only Task(subagent_type:cicd-agent) and Task(subagent_type:infrastructure-agent) — so the qualified dispatch at line 40 stays ungranted and the skill keeps taking the sequential fallback. Adding Task(subagent_type:development-codebase-tools:agent-orchestrator-agent) to SKILL.md:3 is what makes the fix take effect.

…unded reports, no fabricated cost field

Broken references:
- refine-linear-task: linear_search_issues / linear_update_issue /
  linear_add_comment are not tools on mcp.linear.app. Replaced with the
  real surface: get_issue, list_issues, save_issue, save_comment.
- daily-standup: mcp__github__get_pull_request no longer exists on the
  GitHub MCP; it is pull_request_read.
- orchestrate-deployment: agent-orchestrator-agent lives in the
  development-codebase-tools plugin, so it needs the plugin-qualified
  dispatch name.

Invented numbers:
- deploy-guide output shape demanded a monthly cost estimate with no
  measured input. Made the field optional, required a named pricing
  source, and said plainly not to derive one from recall.

Output length (effort settings do not shorten authored deliverables):
- Bounded the datadog-cost-tracker and investigate-incident reports.

Delegation:
- Restated the deploy agent split as a ceiling and added a
  when-NOT-to-delegate rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wkoutre
wkoutre force-pushed the nickkoutrelakos/opus5-uniswap-integrations-v2 branch from cedbc3e to 6f716bb Compare August 5, 2026 23:40
@wkoutre
wkoutre merged commit 0833f0f into next Aug 5, 2026
18 checks passed
@wkoutre
wkoutre deleted the nickkoutrelakos/opus5-uniswap-integrations-v2 branch August 5, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant