diff --git a/.github/skills/validate-blazor-feature/SKILL.md b/.github/skills/validate-blazor-feature/SKILL.md index ebe34f2b3182..9a88ead6e02e 100644 --- a/.github/skills/validate-blazor-feature/SKILL.md +++ b/.github/skills/validate-blazor-feature/SKILL.md @@ -1,12 +1,12 @@ --- name: validate-blazor-feature description: >- - Validate a Blazor feature or behavior interactively in a browser using the canonical Components samples in this repo, before writing E2E tests. USE FOR exercising a Blazor change in src/Components (a render-mode behavior, an interactive component, enhanced navigation, forms, streaming, prerendering), deciding which sample to use, where to add a test page, how to set the render mode (Server/WebAssembly/Auto/static SSR), how to confirm the app is actually interactive (not just static SSR), and how to inspect the browser console and network for errors. Covers BlazorWebAppGlobal, BlazorWebAppPerPage, and BlazorWebAssemblyStandalone, launching them against the in-tree framework, and driving them with the Playwright MCP browser tools. DO NOT USE FOR writing permanent E2E/Selenium or unit tests, non-Components areas, comparative alternatives research, or isolated native-browser API/HTML/CSS experiments when no Blazor feature implementation is being validated; prefer a minimal browser probe instead. + Validate a Blazor feature or regression in a real browser using canonical Components samples before selecting permanent coverage. USE FOR exercising src/Components changes with Playwright; choosing the sample or render mode; proving interactivity; checking browser errors; turning a failure into a temporary JavaScript diagnostic assertion; assessing determinism; investigating producer or timing behavior; replacing arbitrary Selenium sleeps with observable waits or gates; choosing a faithful unit-versus-browser boundary; and recording the permanent-test handoff. Always use for Components tasks involving deterministic browser evidence, flaky Selenium or Thread.Sleep, timing gates, or a JS-versus-Selenium decision. It may author temporary probes but does not write permanent tests. DO NOT USE FOR non-Components work, comparative alternatives research, or isolated browser API/HTML/CSS experiments without a Blazor implementation; use a minimal browser probe instead. --- # Validate a Blazor feature with the Components samples -Workflow: pick a sample, add a scenario page, set the render mode, build, launch, drive it in a browser, and check for errors. Validate behavior before writing E2E tests; remove the sample code afterward (`git checkout`/`git clean`) once the E2E test covers it. +Workflow: pick a sample, add a scenario page, set the render mode, build, launch, drive it in a browser, and check for errors. When the run reproduces a browser regression or failure that needs permanent coverage, also reduce it to a deterministic diagnostic probe before selecting permanent coverage. Remove any temporary diagnostic and sample code once the selected permanent test covers it. ## 1. Pick the sample and where the page goes @@ -88,12 +88,52 @@ After interacting, inspect the console. **Scope it to the current page**: call ` See [references/error-checks.md](references/error-checks.md) for the catalog of common failures, the symptom each produces, and the fix. -## 6. Finish +## 6. Distill the reproduction and establish determinism (browser regressions only) + +Apply this section only when steps 4-5 reproduced a browser regression or failure that needs permanent coverage. Skip it for new-feature validation with no failure to reduce. + +After Playwright reproduces the failure, reduce the scenario to the smallest JavaScript probe and assertion that observes the same final browser state. Run it in the real page with Playwright evaluation or from temporary scratch code. This probe is diagnostic: do not add it to a Jest or `.test.ts` suite, do not include it in the production change, and remove it after the permanent scenario supersedes it. + +Repeat the unchanged scenario and probe a recorded, bounded number of times. When repository evidence does not suggest another count, 10 runs is a reasonable default confidence sample, not proof that the behavior is deterministic across machines, browsers, or load. Record the count, environment, assertion, and failure signature. Call the reproduction deterministic under the exercised conditions only when every run fails the same assertion for the same reason; never describe the bounded sample as proving determinism. + +If the runs do not agree, stop the permanent-test handoff and investigate the owning producer and timing: + +- Add targeted logging or browser observations that distinguish whether the producer ran, which precondition differed, and where ordering changed. +- Use a faithful unit or lower-boundary test when that module's contract is the final observable and the test exercises its real inputs. +- When managed timing must be controlled for the browser scenario, introduce an explicit test gate such as a `TaskCompletionSource`, endpoint, or test-only release action and wait for observable browser state before and after releasing it. +- Do not make the race appear stable by inserting an arbitrary fixed delay. + +When the browser reproduction is deterministic, record that the permanent real-browser scenario must preserve the same setup and final assertion. For Components Selenium, prefer observable waits such as `Browser.True` and `Browser.Equal` (which poll with `WebDriverWait`) when the final observable is sufficient. For generated-template behavior, use the existing Playwright template suite and its observable waits. Use an explicit test-controlled gate only when deterministic managed or intermediate timing must be controlled. A bounded polling helper may delay between observable checks when it also enforces a timeout; a fixed sleep is not the synchronization mechanism for the regression assertion. + +## 7. Record the permanent regression boundary + +Before handing off to permanent tests, record: + +- **Behavior owner**: the subsystem that owns the behavior. +- **Production producer**: the real mechanism that creates the disputed preconditions. +- **Final observable**: the material result the regression test must assert. +- **Selected permanent surface**: the test suite that exercises that producer and observable. +- **Lower-boundary false-pass risk**: how a lower-level test could pass while the shipped behavior still fails. + +For browser-owned user-visible behavior (real DOM measurement, layout or geometry, scrolling, browser observers, browser scheduling or event ordering, browser-dependent JS interop, navigation, focus or selection, and rendering or rehydration), hand off to the repository's real-browser suite that owns the production path. Use C# Selenium under `src/Components/test/E2ETest` for Components-owned behavior. When generated template content owns the behavior, use the existing Playwright tests under `src/ProjectTemplates/test/Templates.Blazor.Tests` instead of duplicating it in Components E2E. Require the identical assertion to be red without the fix and green with it. Include a nearest-opposite control, meaning the closest scenario that must stay green, and, when meaningful, an adjacent control driven by the same production producer. A browser fix remains blocked while Jest is its only regression proof. + +Do not recommend adding or retaining Jest or `.test.ts` coverage as proof of the same browser scenario, including as supplemental coverage or in an existing test file. When synthetic geometry, mocked observers or events, or direct state or callback injection stand in for that browser scenario's real producer, they are temporary diagnostic probes outside the production change. + +JavaScript or TypeScript unit coverage remains appropriate when the module's deterministic contract is the final observable and its harness faithfully supplies the contract inputs. This includes pure helpers, structural DOM algorithms over ordinary nodes, manager callback or timer-state contracts, and validation-engine contracts. These tests do not prove that real browser scheduling, layout, observers, or a user-visible producer path is reachable; claims about those behaviors remain Selenium scenarios. Managed or service behavior fully owned and observable below the browser remains at that faithful lower boundary. + +If WebDriver cannot perform or observe one exact operation, name that limitation and permit only the smallest existing JavaScript helper or `IJavaScriptExecutor` snippet for that step. Keep the permanent scenario orchestration and final user-visible assertion in C# Selenium. + +Treat `src/Components/AGENTS.md` section "Permanent regression test boundary" as the normative policy when recording this handoff. + +This skill validates the sample interactively and records the handoff. It does not write the permanent Selenium test. + +## 8. Finish - Stop the sample server by its specific PID. - Remove the Playwright artifacts folder (`.playwright-mcp/`) it drops into the working directory. -- Per the Components workflow, once an E2E test covers the behavior, remove the sample scenario code: `git checkout -- src/Components/Samples src/Components/WebAssembly/Samples` and `git clean -df -- src/Components/Samples src/Components/WebAssembly/Samples`. +- Remove any temporary JavaScript diagnostic probe. +- Per the Components workflow, once the selected permanent test covers the behavior, remove the sample scenario code: `git checkout -- src/Components/Samples src/Components/WebAssembly/Samples` and `git clean -df -- src/Components/Samples src/Components/WebAssembly/Samples`. ## Completion criteria -The validation is done when: the page loaded (200, framework JS served), the behavior was exercised by a real interaction, the resulting state change was observed in a snapshot, and the console shows no real errors for the page under test. Only then move on to writing the E2E test. +The validation is done when: the page loaded (200, framework JS served), the behavior was exercised by a real interaction, the resulting state change was observed in a snapshot, and the console shows no real errors for the page under test. For a regression, also record the temporary probe, bounded repeat result, and whether every run failed for the same reason. Only then record the five-field handoff to the selected permanent real-browser surface; inconsistent behavior returns to producer/timing investigation or an explicit gate. diff --git a/eng/skill-evals/validate-blazor-feature/eval.vally.yaml b/eng/skill-evals/validate-blazor-feature/eval.vally.yaml index 68a94678cc22..df6f52af2814 100644 --- a/eng/skill-evals/validate-blazor-feature/eval.vally.yaml +++ b/eng/skill-evals/validate-blazor-feature/eval.vally.yaml @@ -10,8 +10,8 @@ defaults: scoring: weights: - output-matches: 0.3 - prompt: 0.7 + output-matches: 0.2 + prompt: 0.8 threshold: 0.6 stimuli: @@ -105,3 +105,202 @@ stimuli: - "Says to add/verify the @rendermode (per page, or on Routes/HeadOutlet for a global app)" - "Considers that a WebAssembly/Auto component must be in the .Client project, otherwise it won't be interactive" - "Suggests confirming the framework JS (blazor.web.js) is built and served and checking the browser console, rather than only guessing" + + - name: choose-selenium-for-real-browser-geometry + prompt: | + I am fixing a Components regression where a virtualized list sometimes jumps after a resize. The + production path combines real scroll geometry with ResizeObserver and IntersectionObserver callbacks. + I plan to add a Selenium scenario for the visible jump. Can I also keep a fast supplemental test in an + existing .test.ts file by defining scrollHeight/clientHeight with Object.defineProperty, stubbing both + observers, and invoking their callbacks directly? What should the production change include, and what + boundary decision should I record before handing the fix off? + graders: + - type: output-matches + config: + pattern: '(?i)(selenium|E2E)' + - type: prompt + rubric: + - "Selects permanent C# Selenium coverage under src/Components/test/E2ETest because the production behavior is owned by real browser geometry, scrolling, and observers" + - "Rejects the synthetic TypeScript/Jest test as production regression proof, including as primary or supplemental coverage; merely mentioning Jest to reject it is correct, but permitting it anywhere in the production change fails this rubric" + - "Records all five boundary fields with substantive case-specific values rather than placeholders: behavior owner, production producer, final observable, selected permanent surface, and lower-boundary false-pass risk" + - "Explains that mocked geometry and directly invoked observer callbacks can pass without proving that the browser producer reaches the shipped state" + - "Requires strict Selenium red/green with the same assertion, a nearest-opposite control, and, when meaningful, an adjacent control driven by the same production producer" + + - name: reject-fast-jest-only-observer-proof + prompt: | + Please keep this fix small and fast: add only a Jest test for a ResizeObserver-driven component bug. + The test can mock ResizeObserver and getBoundingClientRect, and CI should not pay for Selenium. I need + the production PR ready today. What test plan should I use? + graders: + - type: output-matches + config: + pattern: '(?i)(selenium|E2E)' + - type: output-matches + config: + pattern: '(?i)(block|blocked|not.*Jest|Jest.*not|only regression proof)' + - type: prompt + rubric: + - "Fails closed despite the request for speed and requires a permanent C# Selenium scenario for the browser-owned ResizeObserver behavior" + - "States that the fix remains blocked while mocked Jest coverage is its only regression proof" + - "Does not compromise by proposing the Jest test as primary, supplemental, faster, or more precise production coverage; merely mentioning Jest to reject it is correct" + - "Allows mocked observer or geometry work only as a temporary diagnostic outside the production change and removes it after Selenium supersedes it" + - "Requires the identical Selenium assertion to fail without the fix and pass with it, with a nearest-opposite control and, when meaningful, an adjacent control driven by the same production producer" + + - name: allow-pure-typescript-helper-test + prompt: | + I extracted a TypeScript helper that parses a comma-separated component option string into a plain + object. It accepts only a string, returns only strings and booleans, reads no globals, and does not + touch the DOM, navigation, events, layout, observers, or JS interop. A bug is entirely in its escaping + rules. Should the permanent regression test use Selenium or a focused TypeScript unit test? + graders: + - type: output-matches + config: + pattern: '(?i)(TypeScript|unit test|pure helper)' + - type: output-matches + config: + pattern: '(?i)(plain input|plain output|browser-independent|no.*fidelity|lower boundary)' + - type: prompt + rubric: + - "Permits focused TypeScript unit coverage because both the helper's preconditions and observable are browser-independent plain inputs and outputs" + - "Explains that Selenium adds no fidelity for a parsing and escaping defect wholly owned by the pure helper" + - "Keeps the exception narrow and would disqualify it if the test mocked browser observers, layout APIs, geometry, navigation, events, or browser-dependent interop" + - "Does not require Selenium or block the fix on E2E merely because the helper is written in TypeScript or used by Components" + - "Still records a coherent five-field boundary decision with substantive case-specific values, the pure helper as owner, and the focused unit suite as the selected surface" + + - name: keep-dom-sync-at-faithful-lower-boundary + prompt: | + A regression in synchronizeDomContentCore reuses the wrong keyed node and changes sibling order while + recursively synchronizing source and destination fragments. The production function operates only on + ordinary DOM Node operations. The smallest faithful reproduction creates ordinary nodes with + document.createElement and innerHTML, then asserts object identity and + sibling order without mocking geometry, layout, observers, events, timers, navigation, focus, or + production callbacks. Which permanent regression surface should own this case, and what boundary + decision supports that choice? + graders: + - type: output-matches + config: + pattern: '(?i)(Jest|jsdom|TypeScript|\.test\.ts)' + - type: prompt + rubric: + - "Selects the existing Jest/jsdom TypeScript unit-test surface, such as DomSync.test.ts or an adjacent test, rather than Selenium" + - "Explains that ordinary jsdom nodes are the faithful production-shaped inputs for this deterministic Node-interface algorithm, not mocked browser producers" + - "Treats object identity and sibling order as the final observable and explains why browser E2E would add no fidelity for it" + - "Does not require or block on Selenium merely because the helper participates in rendering" + - "Records all five boundary fields with substantive case-specific values rather than placeholders: behavior owner, production producer, final observable, selected permanent surface, and lower-boundary false-pass risk" + + - name: keep-endpoint-persistence-at-managed-boundary + prompt: | + During EndpointHtmlRenderer prerender persistence, two RegisterOnPersisting callbacks both call + PersistentComponentState.PersistAsJson with the same key. The second call throws ArgumentException; + ComponentStatePersistenceManager catches and logs the callback failure, aborts persistence, and clears + the pending state instead of writing it to the store. There is no DOM, JavaScript, navigation, timing, + or browser dependency. Which permanent regression surface should own this case, and why? + graders: + - type: output-matches + config: + pattern: '(?i)(managed|C#|unit test|integration test)' + - type: prompt + rubric: + - "Selects a managed unit or integration test under src/Components/test rather than browser E2E" + - "Identifies the EndpointHtmlRenderer and ComponentStatePersistenceManager prerender-persistence path as the behavior owner and production producer" + - "Uses the logged callback failure, aborted store write, and cleared pending state as the final observable rather than claiming the duplicate-key exception escapes EndpointHtmlRenderer" + - "Explains that Selenium adds no fidelity because every precondition and observable is fully owned and observable below the browser" + - "Records all five boundary fields with substantive case-specific values rather than placeholders: behavior owner, production producer, final observable, selected permanent surface, and lower-boundary false-pass risk" + + - name: keep-selenium-with-minimal-javascript-step + prompt: | + I am validating focus and selection restoration after a component rerender. C# Selenium can drive the + interaction and assert the user-visible edited value, but WebDriver has no precise API for setting an + input's selection range to exact UTF-16 offsets. May I use JavaScript, and if so how should the permanent + test be divided? + graders: + - type: output-matches + config: + pattern: '(?i)(IJavaScriptExecutor|setSelectionRange|JavaScript)' + - type: output-matches + config: + pattern: '(?i)(?:(?:C#|Selenium).*(?:final|assert|orchestrat)|(?:final|assert|orchestrat).*(?:C#|Selenium))' + - type: prompt + rubric: + - "Keeps the permanent scenario in C# Selenium rather than moving it to a JavaScript or TypeScript unit test" + - "Names the exact limitation: WebDriver cannot precisely set the input selection range to exact UTF-16 offsets" + - "Allows only a minimal existing helper or IJavaScriptExecutor snippet for the setSelectionRange step" + - "Keeps navigation, rerender orchestration, and the final user-visible edited-value assertion in C# Selenium" + - "Does not move scenario orchestration or the final assertion into JavaScript or TypeScript; any answer granting that permission fails even if some C# Selenium coverage remains" + + - name: deterministic-browser-probe-to-selenium + prompt: | + A Blazor Virtualize regression leaves a blank gap after a real browser resize. I reproduced it with + Playwright against the Components sample and can inspect the gap from JavaScript. Before I write the + permanent test, how should I turn this into deterministic evidence and then hand it off? Include what + happens to any diagnostic JavaScript afterward. + graders: + - type: output-matches + config: + pattern: '(?is)(Playwright|real browser).*(JavaScript|JS|probe|assert)' + - type: output-matches + config: + pattern: '(?i)(selenium|E2ETest)' + - type: prompt + rubric: + - "Reduces the Playwright reproduction to the smallest temporary JavaScript diagnostic assertion against the real page, not permanent Jest or .test.ts coverage" + - "Repeats the unchanged scenario a recorded bounded number of times and calls it deterministic only when every run fails the same assertion for the same reason" + - "May suggest 10 runs as a reasonable default confidence sample, but explicitly says the count is not magical and does not prove determinism across environments; wording that the bounded sample proves determinism fails this rubric" + - "Hands the same scenario and final assertion to permanent C# Selenium, using observable waits or an explicit gate rather than a fixed sleep" + - "Removes the diagnostic JavaScript after Selenium supersedes it and does not check it into the production change" + + - name: investigate-nondeterministic-browser-reproduction + prompt: | + A focus restoration bug fails in 4 of 10 Playwright attempts, and two failures show different final + states. A teammate wants to add the Selenium test immediately with Thread.Sleep(1000) after the rerender + so CI is stable. What should I do before choosing the permanent regression test? + graders: + - type: output-matches + config: + pattern: '(?i)(producer|timing|ordering|precondition)' + - type: output-matches + config: + pattern: '(?i)(Browser\.True|Browser\.Equal|WebDriverWait|observable condition|faithful lower boundary|TaskCompletionSource|explicit (?:test )?gate|test-controlled gate)' + - type: prompt + rubric: + - "Does not call the reproduction deterministic because not every run failed the same assertion for the same reason" + - "Rejects Thread.Sleep(1000) as a synchronization mechanism and does not paper over the race" + - "Investigates the real producer, preconditions, and timing with targeted observations or logging before finalizing permanent coverage" + - "Chooses a faithful lower-boundary test only if that boundary owns the final observable; otherwise uses observable browser waits when the final observable is sufficient, or introduces an explicit test-controlled gate only when deterministic managed or intermediate timing must be controlled" + - "Does not claim an immediate Selenium conversion is ready until the scenario is controlled and repeatable" + + - name: replace-arbitrary-selenium-sleep + prompt: | + My Components Selenium regression clicks a button that starts asynchronous managed work, sleeps for two + seconds, and then asserts the updated DOM. It passes locally but flakes in CI. What synchronization + pattern should replace the sleep, and are all Thread.Sleep calls forbidden? + graders: + - type: output-matches + config: + pattern: '(?i)(Browser\.True|Browser\.Equal|WebDriverWait|observable condition)' + - type: prompt + rubric: + - "Rejects the arbitrary two-second sleep as synchronization for the regression assertion" + - "Prefers Browser.True/Browser.Equal or another WebDriverWait-backed observable browser condition" + - "Requires an explicit test gate such as TaskCompletionSource and a release action only when deterministic managed or intermediate timing must be controlled; accepts Browser.True/Browser.Equal or another WebDriverWait-backed wait when the final DOM observable is sufficient" + - "Does not impose a syntactic ban on every Thread.Sleep; allows bounded polling helpers that repeatedly observe a condition and enforce a timeout" + + - name: split-manager-contract-from-browser-reachability + prompt: | + AutoPauseManager has a deterministic bug where a visibility callback plus its controlled timer state + invokes pause twice. Separately, the user-visible pause UI sometimes fails after a real tab visibility + transition and may depend on browser scheduling. Should both regressions move to Selenium, or should the + permanent tests be split? + graders: + - type: output-matches + config: + pattern: '(?i)(Jest|jsdom|TypeScript unit)' + - type: output-matches + config: + pattern: '(?i)(selenium|E2ETest)' + - type: prompt + rubric: + - "Keeps the deterministic AutoPauseManager callback and timer-state contract at its existing Jest/jsdom unit boundary when that module contract is the final observable" + - "Uses C# Selenium for the user-visible pause behavior whose claim depends on the real visibility producer or browser scheduling" + - "Explains that controlled callbacks or fake timers establish manager behavior but do not prove real browser reachability or scheduling" + - "Does not force every JavaScript manager contract into Selenium and does not use the manager unit test as proof of the separate user-visible browser scenario" diff --git a/src/Components/AGENTS.md b/src/Components/AGENTS.md index b203a7821ec3..6747f8c6a871 100644 --- a/src/Components/AGENTS.md +++ b/src/Components/AGENTS.md @@ -23,12 +23,37 @@ For implementation work: - Research the problem area using the microsoft docs, existing code, git history, and logging on the sample project. - Components and template files move. Resolve historical paths with `git log --follow --name-status` or `git ls-tree` before using `git show :`. - Implement the fix or feature in the sample project first. -- Test the fix or feature interactively using Playwright. -- Once the fix or feature is validated in the sample, implement E2E tests for it. - - When you create an E2E test. First execute it interactively with Playwright. +- Exercise a new feature, or reproduce a behavioral failure, interactively in the real browser using Playwright. +- For a browser regression or reproduced browser failure, reduce the reproduction to the smallest temporary JavaScript diagnostic probe and assertion that observes the same failure in that page. Keep the probe outside the production change; it is not permanent Jest or `.test.ts` coverage. +- For that regression, repeat the unchanged browser scenario and probe a recorded, bounded number of times. Ten runs is a reasonable default confidence sample when repository evidence does not suggest another count, not proof across environments. Treat the issue as deterministic only when every run fails the same assertion for the same reason. +- If the result is not deterministic, investigate the owning producer and timing before selecting permanent coverage. Use a faithful lower-boundary test when that boundary owns the final observable, or add an explicit test-controlled gate when managed timing must be controlled. Do not stabilize a regression assertion with an arbitrary fixed sleep. +- Once the fix or feature is validated in the sample, select its permanent test surface using the boundary below. + - For browser-owned behavior, translate the validated browser scenario and final observable into the real-browser suite selected by the boundary below; for a regression, preserve the diagnostic assertion. - If an E2E test is failing, debug it by running the test server manually and navigating to the scenario in a browser. -- Only after the E2E tests are passing, remove the sample code you added in the Samples projects. - - Use `git checkout` and `git clean -fd` to remove the sample code. +- Only after the selected permanent tests are passing, remove any temporary JavaScript probe and the sample code you added in the Samples projects. + - Use `git checkout -- src/Components/Samples src/Components/WebAssembly/Samples` and `git clean -df -- src/Components/Samples src/Components/WebAssembly/Samples` to remove the sample code. + +### Permanent regression test boundary + +Before selecting a permanent test surface, record these five fields: + +- **Behavior owner**: the subsystem that owns the behavior. +- **Production producer**: the real mechanism that creates the disputed preconditions. +- **Final observable**: the material result the regression test must assert. +- **Selected permanent surface**: the test suite that exercises that producer and observable. +- **Lower-boundary false-pass risk**: how a lower-level test could pass while the shipped behavior still fails. + +Browser-owned user-visible behavior requires permanent coverage in the repository's real-browser suite that owns the production path. For Components-owned behavior, use C# Selenium under `src/Components/test/E2ETest`. When generated template content owns the behavior, use the existing Playwright tests under `src/ProjectTemplates/test/Templates.Blazor.Tests` instead of duplicating the scenario in Components E2E. This boundary includes claims that depend on real DOM measurement, layout and geometry, scrolling, browser observers (`ResizeObserver`, `IntersectionObserver`, and `MutationObserver`), browser scheduling or event ordering, browser-dependent JS interop, navigation, focus and selection, and rendering or rehydration. Extend existing test assets and classes when practical. + +Do not add or retain Jest or `.test.ts` coverage in the production change as proof of the same browser scenario, whether described as primary, supplemental, faster, or more precise. When synthetic geometry, mocked observers or events, or direct state mutation or callback invocation stand in for that browser scenario's real producer, they are temporary diagnostic probes outside the production change and must be removed after the selected real-browser test supersedes them. + +Permanent JavaScript or TypeScript unit tests remain appropriate when the JavaScript module's deterministic contract is the final observable and the harness faithfully supplies that contract's inputs. Examples include pure helpers, structural DOM algorithms that operate on ordinary nodes and assert identity or document structure, manager callback or timer-state contracts, and validation-engine contracts. Such tests do not establish that real browser scheduling, layout, observers, or user-visible producer paths are reachable; a claim about those behaviors still requires Selenium. Managed or service behavior that is fully owned and observable below the browser likewise remains at its faithful lower boundary. + +If WebDriver cannot perform or observe one operation, keep the permanent C# Selenium scenario, name the exact WebDriver limitation, and use only the smallest existing JavaScript helper or `IJavaScriptExecutor` snippet for that step. Keep scenario orchestration and the final user-visible assertion in C#. + +For browser fixes, require strict red/green evidence: the identical assertion in the selected real-browser suite must fail for the expected reason without the fix and pass with it. Include a nearest-opposite control, meaning the closest scenario that must remain green, and, when meaningful, an adjacent control driven by the same production producer. The fix remains blocked while Jest is its only regression proof. + +In Components Selenium tests, synchronize assertions with observable conditions through existing waits such as `Browser.True` and `Browser.Equal`, which poll with `WebDriverWait`, or with explicit test-controlled gates such as `TaskCompletionSource` when managed timing must be controlled. Do not use an arbitrary `Thread.Sleep` as the synchronization mechanism for the regression assertion. Existing bounded polling helpers may sleep between checks when they repeatedly observe the condition and enforce a timeout. ### Cross-runtime design checkpoint @@ -95,7 +120,7 @@ Before expanding validation across this full matrix, state the render modes and **Always start by adding your feature scenario to whichever sample matches the render mode you need.** This allows you to: - Quickly iterate on the implementation - Test the feature interactively in a real browser -- Verify the feature works before writing formal E2E tests +- Verify the feature works before writing its selected permanent tests - Debug issues more easily with full logging capabilities 3. **Debug when needed**: @@ -106,9 +131,9 @@ Before expanding validation across this full matrix, state the render modes and 4. **Validate the sample works** - You must have a validated, working sample in the Samples folder before proceeding. Use Playwright to confirm the feature works end-to-end in the browser. -5. **Implement E2E tests** - Only after the sample is validated, implement E2E tests for it. +5. **Implement permanent tests** - Only after the sample is validated, select the permanent test surface using the boundary above. Browser-owned behavior requires the owning real-browser suite identified there. -6. **Clean up sample code** - After your E2E tests are passing, remove the sample code you added to the Samples projects. The sample was only for development and interactive testing; the E2E tests now provide the permanent test coverage. Use `git checkout -- src/Components/Samples` and `git clean -df -- src/Components/Samples` to remove the sample code. +6. **Clean up sample code** - After the selected permanent tests are passing, remove the sample code you added to the Samples projects. The sample was only for development and interactive testing; the selected tests now provide the permanent coverage. Use `git checkout -- src/Components/Samples src/Components/WebAssembly/Samples` and `git clean -df -- src/Components/Samples src/Components/WebAssembly/Samples` to remove the sample code. ## Build Tips