Add toolset property to telemetry payloads in addon-mcp#78
Conversation
🦋 Changeset detectedLatest commit: 0dcd3e8 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR 💥 An error occurred when fetching the changed packages and changesets in this PR |
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
commit: |
Bundle ReportChanges will increase total bundle size by 84 bytes (0.22%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: @storybook/addon-mcp-esmAssets Changed:
Files in
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## next #78 +/- ##
=======================================
Coverage 39.10% 39.10%
=======================================
Files 23 23
Lines 693 693
Branches 172 172
=======================================
Hits 271 271
Misses 391 391
Partials 31 31 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Pull Request Overview
This PR adds a toolset property to all telemetry payloads in the addon-mcp package to identify which toolset ('dev' or 'docs') each tool belongs to. This enhancement improves telemetry tracking by making it possible to analyze tool usage by toolset.
Key Changes:
- Added
toolset: 'dev'to telemetry calls in dev tools (get-story-urls, get-ui-building-instructions) - Added
toolset: 'docs'to telemetry calls in docs tools (onListAllComponents, onGetComponentDocumentation handlers) - Updated test expectations to verify the toolset property is included
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| packages/addon-mcp/src/tools/get-ui-building-instructions.ts | Added toolset: 'dev' to telemetry payload |
| packages/addon-mcp/src/tools/get-ui-building-instructions.test.ts | Updated test expectation to include toolset: 'dev' |
| packages/addon-mcp/src/tools/get-story-urls.ts | Added toolset: 'dev' to telemetry payload |
| packages/addon-mcp/src/tools/get-story-urls.test.ts | Updated test expectation to include toolset: 'dev' |
| packages/addon-mcp/src/mcp-handler.ts | Added toolset: 'docs' to both component manifest tool telemetry handlers |
* enter prereelase mode on next branch * Handle HTML in a separate file (#56) * extract human-readable /mcp to maintainable html file * upgrade tsdown, remove json treeshaking workaround * add changeset * fix tsdown types * add changeset release branches to checks * commit releases with gh api. see https://github.com/changesets/action#inputs * Version Packages (next) (#57) Co-authored-by: storybook-app-bot[bot] <175111413+storybook-app-bot[bot]@users.noreply.github.com> * Replace Storybook canary versions with 10.1.0 prereleases (#59) * upgrade to storybook 10.1.0-alpha.2 * changesets * Version Packages (next) (#60) Co-authored-by: storybook-app-bot[bot] <175111413+storybook-app-bot[bot]@users.noreply.github.com> * Rename "examples" to "stories" in component manifest format (#61) * Initial plan * Rename "examples" to "stories" in component manifest format - Updated type definitions in types.ts files to rename Example to Story - Updated format-manifest.ts to use story terminology (story, story_name, story_description, story_code) - Updated all fixture JSON files to use "stories" instead of "examples" - Updated test files and descriptions to use "stories" terminology - Updated test snapshots to reflect the new XML output format - All tests passing, build and typecheck successful --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jeppe Reinhold <jeppe@reinhold.is> * Support name in manifest errors (#55) * add name to manifest errors * add fixtures with errors * add changeset * improve test reports in ci * update fixtures to use stories instead of examples * more example -> story renaming * Improve code quality and development setup (#64) * replace prettier with oxfmt * add oxlint for linting * update actions using npx actions-up * add publint * fix types * add check-everything script * add build-storybook to check all * split GH Workflows * explain lint disables * Version Packages (next) (#63) Co-authored-by: storybook-app-bot[bot] <175111413+storybook-app-bot[bot]@users.noreply.github.com> * Replace oxfmt with Prettier (#68) * replace oxfmt with prettier * typo * Update to the latest SB alpha so the internal storybook version works with the server (#71) Co-authored-by: Jeppe Reinhold <jeppe@reinhold.is> * Revert Embed demo image from storybook.js.org#21 (#75) * Evals (#69) * add initial eval setup * well, a lot happened here... * add clack * Add interactive prompts and styled output to eval CLI (#65) * Initial plan * Add interactive prompts and prettier output to eval CLI Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * Use tasks API for parallel evaluation steps Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * Apply oxfmt formatting to eval.ts --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * improve terminal experience * save environment * improve terminal experience * only allow one eval at a time * add support for custom context * format * add support for eval hooks, add log about how to rerun experiments * prompt to start storybook at the end of the evaluation * add message about getting into the experiment * improve experiment dir name * take screenshots of failed stories too * cleanup * improve reshaped stories, improve test+a11y summary, improve mcp server config arg * support --[no-]storybook flag * collect experiment description and branch name * save result summary to google sheets * improve plain prompt * prompt for google sheets upload * fix google sheets upload * support "Storybook MCP" context, which starts up the docs-only @storybook/mcp server with a given component manifest * Add basic Radix eval (#66) * Add Radix eval * Add Rsuite eval (#67) --------- Co-authored-by: Jeppe Reinhold <jeppe@reinhold.is> * format * fix typechecking * add reshaped component manifest * add conversation-viewer.html with approximate token count * cleanup * add documentation, fixups * format * fix stories not having imports anymore * fix plain and radix experiments * experiments will have unique package names * more eval test fixing * more story fixes * fix typecheck and lint summary * improve conversation viewer * simplify viewer content * simplify viewer content * result visualisations is via storybook * upload to chromatic * update google sheet row order * add Chromatic link to CLI log * add note about public results * remove description arg from evals * Evals: Add Radix UI website prompt (#74) --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> Co-authored-by: Michael Shilman <shilman@users.noreply.github.com> * Review Kasper (#70) * Start review * Fix * More comments * Fix config files and restructure * Resolve conflicts * Fix github actions * Fix coverage * Fix type error * Fix * Fix * Dedupe * Update packages/mcp/src/index.ts Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> * Update .github/workflows/check.yml Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> * Update .github/workflows/check.yml Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> * Improve get/post handling * Dedupe vite * lock file * test perf of check-everything in CI * rename * rename * Add turbo caching * check cache invalidation * refactor * refactor * refactor * refactor * Use node version file * description * refactor * rollback * use turbo for artifacts * install node * optimize * install offline for faster symlinking * optimize * Check ci * Only upload test results on failure * Check github reporter * Fix command * Fix test * Remove check everything * test corepack enable * test corepack enable * test corepack enable * fix * Check if this is faster * Check if this is faster * no cache * rollback * Change nothing * Fix prettier * Modify changeset for MCP server GET responses Updated the changeset to handle GET responses in the MCP server. * Prettier * use docker * debug * use node 24 * Try own caching * Prune it * Don't format pnpm lock * Fix * again * use composite * change * Revert "change" This reverts commit 8031a63. * Revert "use composite" This reverts commit 7f26a54. * Revert "again" This reverts commit 7fdccdf. * Revert "Fix" This reverts commit f4dd004. * Revert "Don't format pnpm lock" This reverts commit c11c4ec. * Revert "Prune it" This reverts commit 1009ad5. * Revert "Try own caching" This reverts commit 82eb804. * Revert "use node 24" This reverts commit c63f9ee. * Revert "debug" This reverts commit d647a91. * Revert "use docker" This reverts commit 766462e. * Address feedback * Initial plan * Update README and Copilot instructions for script changes Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * Address feedback * Make it loose * Watch storybook by default * Fix command * Fix * Add pnpm to ignore * Fix dev command * Cleanup * get CI green --------- Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * Make `get-component-documentation` tool only accept a single component ID instead of multiple (#79) * cleanup * get-component-documentation only accepts a single component id * Fix evals (#81) * cleanup * get-component-documentation only accepts a single component id * fix versions * use vitest cli instead of node for evals * prefix experiment scripts so they are not picked up by turborepo * Add toolset property to telemetry payloads in addon-mcp (#78) * Initial plan * Add toolset property to all telemetry payloads in addon-mcp Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> * add changeset --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> Co-authored-by: Jeppe Reinhold <jeppe@reinhold.is> Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> * remove source API and use the request instead (#54) * remove source API and use the request instead * cleanup * add changesets * add path argument to manifestProvider * cleanup * update changeset * fix serve.ts * cleanup * Fix internal stdio-based MCP server (#85) * allow undefined requests when using custom manifestProvider * changeset * add tests for internal stdio transport * cleanup * Add end-to-end tests and improve unit test quality (#84) * add e2e tests * improve e2e scripting * add tests for mcp index * add preset tests * add telemetry tests * simplify tool test mocks * simplify mcp-handler tests, improve disableTelemetry handling * add tests for manifest availability * exclude evals from coverage * cleanup * changeset * fix preset registering handlers instead of middlewares * update tests to match changes in base branch * cleanup * await sb process kill * globally mock storybook deps * clean lock file * Output in markdown instead of XML (#86) * add e2e tests * improve e2e scripting * add tests for mcp index * add preset tests * add telemetry tests * simplify tool test mocks * simplify mcp-handler tests, improve disableTelemetry handling * add tests for manifest availability * exclude evals from coverage * cleanup * changeset * fix preset registering handlers instead of middlewares * update tests to match changes in base branch * cleanup * await sb process kill * refactor formatter, splitting into markdown and xml, configurable, defaulting to markdown * globally mock storybook deps * clean lock file * fix context arg * fix tests * fix types * "Examples" -> "Stories", simplify tests * simplify tests and types * simplify * use ts-like prop type docs format * add script to clean experiments * add changeset * exit pre mode (#88) * Update reshaped flight booking eval (#87) * Update reshaped flight booking eval * format --------- Co-authored-by: Jeppe Reinhold <jeppe@reinhold.is> Co-authored-by: Jeppe Reinhold <jeppe@chromatic.com> * Version Packages (#80) Co-authored-by: storybook-app-bot[bot] <175111413+storybook-app-bot[bot]@users.noreply.github.com> --------- Co-authored-by: storybook-app-bot[bot] <175111413+storybook-app-bot[bot]@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: Tom Coleman <tom@chromatic.com> Co-authored-by: Michael Shilman <shilman@users.noreply.github.com> Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com> Co-authored-by: Kasper Peulen <kasperpeulen@gmail.com>
Tool call telemetry payloads now include a
toolsetproperty identifying whether the tool belongs to thedevordocstoolset.Changes
Dev tools: Added
toolset: 'dev'to telemetry payloads in:get-story-urlstoolget-ui-building-instructionstoolDocs tools: Added
toolset: 'docs'to telemetry payloads in:onListAllComponentshandleronGetComponentDocumentationhandlerUpdated test expectations to verify the
toolsetproperty is presentExample
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/repos/storybookjs/mcp/actions/runs/19445952513/jobsnode /home/REDACTED/work/mcp/mcp/packages/addon-mcp/node_modules/.bin/../tsdown/dist/run.mjs --config ../../tsdown.config.ts(http block)o26192.ingest.us.sentry.ionode /home/REDACTED/work/mcp/mcp/packages/mcp/node_modules/.bin/../tsdown/dist/run.mjs(dns block)node /home/REDACTED/work/mcp/mcp/packages/addon-mcp/node_modules/.bin/../tsdown/dist/run.mjs --config ../../tsdown.config.ts(dns block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.