test(vscuse): add local debug and Python scaffolding cases - #16476
test(vscuse): add local debug and Python scaffolding cases#16476Alive-Fish wants to merge 6 commits into
Conversation
Add semantic target adapters for the three local debug launch profiles and cover them with generated weather agent cases. The local profiles run the local lifecycle in their own preLaunchTask chain, so they require neither provision nor deploy; the Agents Playground profile serves the agent from the local machine and requires no account at all. VCB-64, VCB-65, VCB-66
Adds a pythonEnvironment semantic operation that drives the Python extension virtual environment flow, registers Python and Basic Custom Engine Agent as scaffold answers, and registers the Python remote Teams launch profile title. Weather Agent is registered for TypeScript, JavaScript, and C# only, so Python scaffolding coverage lands on Basic Custom Engine Agent instead, mapped to the three recorded Python plans.
❌ VscUse Test Plan — Tests unknownWhy these tests: Selected 6 impacted test plan(s) changed in this PR (no product code modified). Branch diff: Plans run:
ℹ️ How were these tests selected?GitHub Copilot (Claude Sonnet 4.6, high reasoning) analysed the PR title, description, and the diff between |
E2E Test Selection — AI SelectedWhy these tests: All changed files fall under packages/tests/vscuse/** (rule 8: vscuse UI tests, not e2e), docs/** (rule 9: infra/docs only), and templates/weather-agent (rule 6: maps to teamsAgent/* which has no matching entry in the available test list); the only available test is a VS/.NET test unrelated to these changes. Cases selected (0): Need to run more tests?Comment on this PR:
Then re-run the workflow. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #16476 +/- ##
==========================================
+ Coverage 87.77% 87.83% +0.05%
==========================================
Files 647 647
Lines 34066 34216 +150
Branches 8028 8078 +50
==========================================
+ Hits 29903 30055 +152
+ Misses 2353 2344 -9
- Partials 1810 1817 +7 🚀 New features to boost your workflow:
|
…matrix Authors the weather bundle as the complete Azure OpenAI / OpenAI x TypeScript / JavaScript x remote / local Teams launch matrix, and samples the remaining surfaces once: Copilot remote, Copilot local, and Playground on both languages. Adds the (Preview) Launch Remote in Copilot (Chrome) target adapter, which unlike the declarative-agent preview requires deploy because a custom engine agent answers from a deployed bot. OpenAI cases stop at the launched surface: the run configuration declares no openai.com key and the recordings they replace type a placeholder, so only the OpenAI scaffold branch and lifecycle are honestly covered. Adds AC rows VCB-72, VCB-73, and VCB-74. Generated plans go from eighteen to twenty-five.
❌ VscUse Test Plan — Tests failureWhy these tests: Selected 15 impacted test plan(s) changed in this PR (no product code modified). Branch diff: Results: ✅ 4 passed · ❌ 11 failed (of 15 plans) Plans run:
ℹ️ How were these tests selected?GitHub Copilot (Claude Sonnet 4.6, high reasoning) analysed the PR title, description, and the diff between |
❌ VscUse Test Plan — Tests failureWhy these tests: PR adds new local-debug weather-agent plans (TS local-teams, local-copilot, playground covering VCB-64/65/66) and new Python basic-custom-engine-agent scaffolding plans; selected the 5 newly-added generated plans most directly exercised by the PR's two feature areas. (Also running 25 impacted test plan(s) changed in this PR.) Branch diff: Results: ✅ 7 passed · ❌ 18 failed (of 25 plans) Plans run:
ℹ️ How were these tests selected?GitHub Copilot (Claude Sonnet 4.6, high reasoning) analysed the PR title, description, and the diff between |
1 participant
Extends the generated vscuse case engine with the two coverage gaps left by the initial bundle: local debug launches and Python scaffolding. Six new generated plans, each mapped 1:1 to an existing hand-recorded plan.
Local debug (
weather-agent.yml, 2 → 5 cases)Three new target adapters —
Debug in Teams (Chrome),(Preview) Debug in Copilot (Chrome), andDebug in Microsoft 365 Agents Playground.weather-ts-azure-openai-local-teamsWeather_Agent_ts_Local_Debug.jsonweather-ts-azure-openai-local-copilotWeather_Agent_ts_local_copilot.jsonweather-ts-azure-openai-playgroundWeather_Agent_Azure_OpenAI_ts_playground.jsonNew acceptance criteria:
provisionand nodeploy; the profile's ownpreLaunchTaskchain runs the local lifecycle.loginand emits no browser sign-in; the Playground serves the agent from the local machine.chatcheck under the Playground target emits the Playground message adapter.Python scaffolding (
basic-custom-engine-agent.yml, new)The Weather Agent template has no Python variant. v3
allTemplates.jsonregistersweather-agent-ts,weather-agent-js, andcustom-copilot-weather-agent-csharponly, and the v4 descriptor declares"languages": ["typescript", "javascript"]. Thetemplates/vsc/python/weather-agentfolder exists but is unregistered, Python never appears in the language picker, and the 319-plan legacy corpus contains no Python weather recording. Registering it would be a product change, so Python coverage lands on Basic Custom Engine Agent, the custom engine agent template the toolkit does offer in Python.basic-cea-py-azure-openai-remote-teamsBasic_Custom_Engine_Agent_Azure_OpenAI_py_Remote_Debug.jsonbasic-cea-py-azure-openai-local-teamsBasic_Custom_Engine_Agent_Azure_OpenAI_py_Local_Debug.jsonbasic-cea-py-azure-openai-playgroundBasic_Custom_Engine_Azure_OpenAI_py_playground.jsonNew
pythonEnvironmentsemantic operation. A Python scaffold writessrc/requirements.txtand installs nothing, and every launch profile starts the app from the workspace interpreter, so a Python case that launches without creating its environment starts an app with no dependencies. The operation is composed entirely from existing coordinate-free components:Python: Create Environment...→ filterVenv→ filter the authored interpreter → confirmSelect dependencies to installwith every dependency checked → open the notification center → assertThe following environment is selected:.Also registers the
pythonlanguage answer, thebasic-custom-engine-agentcapability answer, and theLaunch Remote (Chrome)target — the Python templates name the remote Teams launch withoutin Teams.New acceptance criteria:
pythonEnvironmentoperation drives the Venv creation flow.Launch Remote (Chrome)target reuses the Teams add-and-open transition.Verification
pnpm --dir packages/tests run test:vscuse-engine— 107 pass, 0 fail.pnpm --dir packages/tests run generate:vscuse-casesrun twice; the second printsNo generated plan changes.Follow-up
The Python local Copilot profile is titled
Debug in Copilot (Chrome)without the(Preview)prefix the TypeScript templates use. No adapter is registered for it yet, so that case is not included here.