feat(uniswap-integrations): add datadog-cost-tracker skill with gotchas - #544
Conversation
Captures two operational gotchas discovered during a Datadog cost-tracker run so future runs avoid the same false-negative / false-positive traps: 1. env:prod filter silently drops Lambda/serverless services (unirpc-v2, notification-service) from estimated_usage.logs.ingested_bytes -- omit env filter and verify tag presence via get_datadog_metric_context. 2. Uniroute APM baseline is invalid before 2026-05-14 (PRs #7996/#7510/#7998 normalised DD_ENV tag, causing a 22x apparent jump) -- mark pre/post comparisons as INCONCLUSIVE, not REGRESSION. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaswXmq8Xotq8La9SNJrSy
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🤖 Claude Code Review
Review SummaryThis PR adds a new Version bump & plumbingThe changes are consistent and complete per the repo's conventions:
Previously-raised issue (resolved)The prior review thread flagged three problems in
The skill also builds in a self-check (step 3 instructs confirming exact metric names in the Metrics Explorer before trusting results), which is the right guard given estimated-usage metric names vary by integration setup. Non-blocking observations
No blocking issues found. 💡 Want a fresh review? Add a comment containing |
📚 Documentation Check ✅Verdict: Passed Plugin version was properly bumped from 2.5.0 to 2.6.0 (minor bump for new skill addition). All relevant CLAUDE.md, README.md, plugin.json, and skill documentation files are updated in the PR. PR #544 Documentation ReviewSummaryPR #544 adds the
The version bump type (minor: Missing Updates
Suggestions (1)
🤖 Generated by Claude Documentation Validator | Mode: |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
📋 Review verdict: COMMENT
👆 The main review comment above is the source of truth for this PR review. It is automatically updated on each review cycle, so always refer to it for the most current feedback.
This formal review submission is for the verdict only. 1 inline comment(s) are attached below.
wkoutre
left a comment
wkoutre
left a comment
There was a problem hiding this comment.
Approving: independent code-reviewer pass confirms correct skills-array registration, correct 2.4.0→2.5.0 minor bump, valid references/gotchas.md path, and safe (read-only + scoped Datadog MCP) tool allowances. One non-blocking nit: SKILL.md cites datadog.estimated_usage.apm_hosts (a host-count metric) alongside ingested_spans (a volume metric) as if interchangeable — worth a follow-up tweak, not a blocker.
wkoutre
left a comment
wkoutre
left a comment
There was a problem hiding this comment.
Re-approving after branch update (prior approval auto-dismissed on push). Independent code-reviewer verdict was READY TO MERGE; only a non-blocking Datadog metric-name nit at SKILL.md:24.
There was a problem hiding this comment.
📋 Review verdict: REQUEST_CHANGES
👆 The main review comment above is the source of truth for this PR review. It is automatically updated on each review cycle, so always refer to it for the most current feedback.
This formal review submission is for the verdict only.
…ol names - APM span ingestion metric: apm_hosts (host count, not service-tagged) removed; ingested_spans corrected to datadog.estimated_usage.apm.ingested_spans - Replace bare unprefixed Datadog tool names with intent-based language via the Datadog MCP server, matching the sibling investigate-incident skill's pattern - Note that estimated-usage metric names must be confirmed against the account's Metrics Explorer (querying a nonexistent name returns empty, not an error) Addresses the automated review's blocking finding on PR #544.
…tracker - Bump plugin 2.5.0 -> 2.6.0 (2.5.0 was taken by use-datadog in #432) - Add datadog-cost-tracker to plugin README skills table, plugin CLAUDE.md skills list, and file-structure tree (the review's noted doc gaps)
|
Addressed the blocking review findings (commits 7e12647, 91423cf): Metric / tool names (blocking)
Docs (non-blocking)
Caveat: the corrected metric names follow the review's suggestion but were not verified against a live Datadog instance. @dylanschmittle-uniswap could you confirm @request-claude-review |
github-actions
Bot
changed the title
Superseded by new review after PR update
There was a problem hiding this comment.
📋 Review verdict: APPROVE
👆 The main review comment above is the source of truth for this PR review. It is automatically updated on each review cycle, so always refer to it for the most current feedback.
This formal review submission is for the verdict only.
Superseded by new review after PR update
There was a problem hiding this comment.
📋 Review verdict: APPROVE
👆 The main review comment above is the source of truth for this PR review. It is automatically updated on each review cycle, so always refer to it for the most current feedback.
This formal review submission is for the verdict only.
2 participants
Summary
datadog-cost-trackerskill to theuniswap-integrationsplugin that guides future cost-analysis runs against Datadog estimated-usage metricsreferences/gotchas.mdfile capturing two pitfalls discovered in a live cost-tracker session that produce silent empty results or misleading anomaly signalsGotchas captured
Gotcha 1 —
env:prodfilter breaks Lambda/serverless log queriesdatadog.estimated_usage.logs.ingested_bytesqueried withenv:prodreturns empty for Lambda/serverless services likeunirpc-v2andnotification-servicebecause they don't emit that tag. Correct pattern: omit the env filter; verify the service tag exists viaget_datadog_metric_contextbefore trusting an empty result.Gotcha 2 — Uniroute APM baseline invalid before 2026-05-14
PRs #7996, #7510, and #7998 (deployed 2026-05-14) changed uniroute to prefer
DD_ENVoverENV/ENVIRONMENT, causingservice:uniroute,env:prodingested spans to jump 22x due to tag normalization — not traffic growth. Any pre-May-14 uniroute APM baseline must be marked INCONCLUSIVE rather than REGRESSION.Test plan
gotchas.mdis linked from SKILL.md Step 2plugin.jsonskills array includes./skills/datadog-cost-trackerand version bumped to 2.5.0Generated with Claude Code
https://claude.ai/code/session_01CaswXmq8Xotq8La9SNJrSy
Generated by Claude Code
AI-Generated Description
What
Adds a new
datadog-cost-trackerskill to theuniswap-integrationsplugin that guides cost-analysis runs against Datadog estimated-usage metrics (datadog.estimated_usage.logs.ingested_bytes,datadog.estimated_usage.apm.ingested_spans), broken down by service, flagging anomalous growth and reduction opportunities.The skill ships a
references/gotchas.mdfile capturing two pitfalls discovered in a live cost-tracker session that produce silent empty results or misleading anomaly signals, plus the standard plugin/marketplace wire-up.Why
Cost-tracker runs against Datadog estimated-usage metrics are error-prone in two specific ways that silently corrupt the analysis. Both were hit during a real session, so they're codified as a
gotchas.mdreference the skill reads before trusting any empty or anomalous result:env:prodfilter breaks Lambda/serverless log queries —datadog.estimated_usage.logs.ingested_bytesqueried withenv:prodreturns empty for serverless services likeunirpc-v2andnotification-servicebecause they don't emit that tag. The filter silently drops their data rather than erroring. Correct pattern: omit the env filter and verify the tag exists on the metric before trusting an empty result.DD_ENVoverENV/ENVIRONMENT, consolidatingservice:uniroute,env:prodingested spans into a ~22× jump from tag normalization, not traffic growth. Any comparison spanning that boundary must be marked INCONCLUSIVE rather than REGRESSION.Changes
packages/plugins/uniswap-integrations/skills/datadog-cost-tracker/SKILL.mddescription,allowed-tools,model: sonnet), a 7-step workflow that reads gotchas before querying, and a structured output format (Cost Summary / Top Ingesters / Anomalies / Inconclusive Comparisons / Recommendations)packages/plugins/uniswap-integrations/skills/datadog-cost-tracker/references/gotchas.mdpackages/plugins/uniswap-integrations/.claude-plugin/plugin.json./skills/datadog-cost-trackerin theskillsarray;version2.5.0→2.6.0packages/plugins/uniswap-integrations/CLAUDE.mdpackages/plugins/uniswap-integrations/README.mddatadog-cost-trackerrow to the skills table (plus minor markdown-lint spacing fixes)CLAUDE.mduniswap-integrationsrow in the Current plugins table:2.5.0→2.6.0Why minor (not patch)
A new user-invocable skill is a backward-compatible feature addition, which matches the minor-bump criteria in the root policy ("new skills, agents, commands, or MCP servers added").
Test plan
SKILL.mdfrontmatter parses (description,allowed-tools,modelpresent)gotchas.mdis linked fromSKILL.md(Step 2 and Notes)plugin.jsonskillsarray includes./skills/datadog-cost-trackerCLAUDE.md+README.mdlist the new skillplugin.jsonversion (2.6.0) matches the Current plugins table row in rootCLAUDE.md/datadog-cost-trackerintent is discoverable after installFollow-up
Per the repo docs policy, the Notion "Uniswap Claude Code Plugin Marketplace" doc should be updated to add this skill and bump the skill count. Happy to do this on request.