Skip to content

Skill: orphan tool azsdk_run_generate_sdk mis-routes to generate-sdk-locally #15950

@helen229

Description

@helen229

Summary

The MCP tool azsdk_run_generate_sdk (defined in SpecWorkFlowTool.cs"Generate SDK from a TypeSpec project using pipeline") is not declared by any skill in .github/skills/. When an agent prompt requires it (e.g. "generate SDK for this release plan"), the skill loader semantically matches the closest-named skill — azsdk-common-generate-sdk-locally — and pulls in an entirely wrong workflow.

Evidence

  1. grep -r azsdk_run_generate_sdk .github/skills/ returns no matches.
  2. Observed in Vally runs of generate-sdk-for-existing-release-plan against the mock MCP: the agent loads azsdk-common-generate-sdk-locally, follows its 11-step local workflow (clone language repo, azsdk_verify_setup, azsdk_package_generate_code, build, customize, test, metadata, commit) — none of which is appropriate for a release-planner-driven pipeline kickoff.
  3. The two tools have very different semantics:
    • azsdk_run_generate_sdk → triggers a remote DevOps pipeline keyed to a release-plan work item.
    • azsdk_package_generate_code (owned by generate-sdk-locally) → runs TypeSpec emitters on disk in a local SDK repo clone.

Impact

  • Wrong skill activation in production. Real users prompting the release-planner agent for "generate SDK" land in the local-build skill, which then asks them to clone repos, run azsdk_verify_setup, etc.
  • Wasted tokens / wall time. Agent spends turns reading local-build references it shouldn't have loaded.
  • Without a skill that owns this tool, Vally graders could not assert correct skill routing for any release-planner scenario that touches generation. (Worked around in PR Port tool-scenario benchmarks from Azure.Sdk.Tools.Cli.Evaluations to Vally (#15124) #15811 by adding disallowed: [azsdk-common-generate-sdk-locally] guards, but the root cause is the missing ownership.)

Proposed fix (skill-owner action)

  1. azsdk-common-prepare-release-plan/SKILL.md — add azsdk_run_generate_sdk to the MCP-Tools table and add a Generate SDK (pipeline) step to the workflow. Update the frontmatter description.INVOKES list accordingly.
  2. azsdk-common-generate-sdk-locally/SKILL.md — add an explicit DO NOT USE FOR clause: "pipeline-driven SDK generation triggered from a release plan (azsdk_run_generate_sdk) — that flow is owned by azsdk-common-prepare-release-plan." Consider trimming WHEN triggers that overlap with the release-planner flow (e.g. bare "generate SDK") so the loader prefers the right skill.

Metadata

Metadata

Assignees

Labels

needs-triageWorkflow: This is a new issue that needs to be triaged to the appropriate team.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions