Add get-setup-instructions tool backed by tagged docs entries#198
Add get-setup-instructions tool backed by tagged docs entries#198
get-setup-instructions tool backed by tagged docs entries#198Conversation
✅ Deploy Preview for storybook-mcp-self-host-example canceled.
|
🦋 Changeset detectedLatest commit: f3ccf1d The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
get-setup-instructions tool backed by tagged docs entries
JReinhold
left a comment
JReinhold
left a comment
There was a problem hiding this comment.
@copilot
- In the case where multiple docs have the
setup-instructionstag, instead of only returning the first entry found, return all of the entries, isolated in separatetexttool contents - We need an eval task that evaluates the effectiveness of the setup instructions. Look through the existing 9xx eval tasks and see how they are set up, and create a new one that evaluates if the agents reliably know to fetch the setup instructions in appropriate scenarios, and that they actually reacts to the content. That probably means the instructions should have something specific, that we can grade against afterwads.
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
Addressed in b03b6ed. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
commit: |
❌ 2 Tests Failed:
View the full list of 2 ❄️ flaky test(s)
To view more test analytics, go to the Test Analytics Dashboard |
Bundle ReportChanges will decrease total bundle size by 401 bytes (-0.81%) ⬇️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: @storybook/mcp-esmAssets Changed:
Files in
Files in
|
There was a problem hiding this comment.
@copilot it seems that this test file is being picked up by the repo global test config, but it shouldn't. FIx that, it is causing tests to fail.
There was a problem hiding this comment.
Fixed in 5f99616. The grading test is no longer stored as a repo-visible *.test.ts file; it now gets copied into the generated trial project during preGrade, so the repo-level Vitest config won’t pick it up anymore.
|
|
||
| Requirements: | ||
|
|
||
| 1. Implement this directly in `src/main.tsx` |
There was a problem hiding this comment.
@copilot to make the eval trial's Storybook slightly more useful, we should instead prompt the agent to write this to a separate component file that we have static stories for, just so we can see the component in the Storybook.
There was a problem hiding this comment.
Updated in 5f99616. Task 916 now asks the agent to implement src/components/MissionControl.tsx, updates src/main.tsx to render it, and includes a static MissionControl story so the component is visible in the eval trial’s Storybook.
Co-authored-by: JReinhold <5678122+JReinhold@users.noreply.github.com>
…t/add-get-setup-instructions-tool
This adds a dedicated
get-setup-instructionsMCP tool so agents can fetch project/bootstrap guidance directly from Storybook docs instead of inferring setup from general component documentation. The tool is exposed only when the docs manifest contains docs entries taggedsetup-instructions.New docs tool
get-setup-instructionsto@storybook/mcpsetup-instructions, with each entry isolated in its owntexttool contentget-documentationfocused on component APIs and examplesDynamic tool visibility
tmcp's asyncenabledcallback to inspect manifests attools/listtimeenabledcheck is evaluated duringtools/list, not at server initializationManifest typing
tags?: string[]Multi-source support
storybookId, following the same source-selection pattern as other docs toolsIntegration surface
packages/mcp/src/index.tspackages/mcp/bin.ts)@storybook/addon-mcpso addon consumers get the same behaviorEval coverage
916-follow-setup-instructionsget-setup-instructionsand actually apply the returned setup guidance in generated codeMissionControlcomponent while relying on a pre-seeded static story so the result is visible in the eval trial's StorybookpreGradeso it is used for eval validation without being picked up by the repo-level Vitest configDocs and contract updates
Example manifest entries:
{ "docs": { "entry-setup": { "id": "entry-setup", "name": "Entry Setup", "title": "Entry Setup", "path": "docs/entry-setup.mdx", "tags": ["setup-instructions"], "content": "# Entry Setup\n\nImport `@acme/ui/styles.css` exactly once from the app entry file before rendering any Acme UI components." }, "provider-setup": { "id": "provider-setup", "name": "Provider Setup", "title": "Provider Setup", "path": "docs/provider-setup.mdx", "tags": ["setup-instructions"], "content": "# Provider Setup\n\nWrap the rendered app in `<AcmeProvider theme=\"midnight\" density=\"comfortable\">`.\n\nPlace your UI inside an element with `data-acme-app=\"true\"`." } } }> [!WARNING]
>
>
Original prompt
This section details on the original issue you should resolve
<issue_title>Add get-setup-instructions tool powered by docs entry tags</issue_title>
<issue_description>We want a new MCP tool named
get-setup-instructions.The purpose of this tool is to give agents a dedicated way to fetch project or design-system setup instructions from Storybook docs, instead of forcing them to infer setup details from general component documentation.
The contract is:
setup-instructionstag@storybook/mcpdetects that tagged docs entry from the docs manifest@storybook/mcpexposes aget-setup-instructionstool when such an entry existsWhy this matters
Today the MCP package already offers:
list-all-documentationget-documentationget-documentation-for-storyThose tools are useful for component APIs and example usage, but they are not explicit about project-level setup.
For design systems and component libraries, setup instructions are often the first thing an agent needs:
We want agents to have an explicit tool for this instead of hoping they discover the right docs page through broader docs browsing.
Current state / code pointers
Relevant files in this repo:
packages/mcp/src/index.tscreateStorybookMcpHandler()packages/mcp/src/types.tsDocschema used for docs manifest entriestagspackages/mcp/src/tools/get-documentation.tspackages/mcp/src/tools/list-all-documentation.tspackages/mcp/src/index.test.tspackages/mcp/fixtures/small-docs-manifest.fixture.jsonImportant upstream dependency
Storybook core is being changed separately to expose docs-entry
tagsin generated manifests.That work is the real source of truth, but this issue should still proceed now.
For parallelization:
tagsinto MCP fixtures/tests firstTracked upstream in: storybookjs/storybook#34149
Requested behavior
Add a new tool named
get-setup-instructions.Expected behavior:
setup-instructionsTool semantics
The tool should be for setup guidance, not general component reference.
The tool description should make it clear that this is the right tool when the agent needs:
The tool description should also make clear that:
get-documentationremains the main tool for component APIs and examplesget-setup-instructionsis specifically for environment/library setupOpen design point to resolve early
The product requirement is that the tool should be dynamically enabled/disabled based on whether the setup doc exists.
Please resolve this early in the implementation:
tmcpevaluates tool visibility viaenabledThere is already an existing conditional-exposure pattern in the repo using
enabled, but it currently relies on synchronous context flags rather than manifest inspection.Proposed implementation direction
packages/mcp/src/types.tstags?: string[]packages/mcp/src/tools/get-setup-instructions.tspackages/mcp/src/index.ts💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.