Skip to content

VR tests produce diffs on most PRs #35414

@dmytrokirpa

Description

@dmytrokirpa

Area

React Components (@fluentui/react-components)

Describe the feature that you would like added

Problem

Visual Regression Tests (VR tests) are producing diffs on almost every PR, even when no package source files were changed. This generates noise, hides real regressions, and increases review overhead.

Examples

https://github.com/microsoft/fluentui/pull/35369/files#r2452119834

#35354 (comment)

Observed causes (high-level)

  • Non-deterministic story output
    Some stories rely on runtime-varying APIs (e.g., Date.now(), Math.random(), omitted value/today props), which causes screenshots to differ between runs.
    Example: utils.tsx is missing explicit value/today props, making the calendar render differently depending on the current date.

  • Assets not fully loaded before capture
    Storywright sometimes captures screenshots before images or fonts finish loading, producing intermittent diffs.

  • Pixel-level drift in certain components
    Components such as Callout and Positioning show small, consistent pixel diffs across runs. This may stem from rendering timing, subpixel rounding, platform/browser differences, or animation/position recalculation race conditions.

Goal

Reduce VR noise by ensuring stable, deterministic story output and reliable capture timing so that diffs signal real, actionable regressions.

Proposed action checklist

  • Stabilize story outputs by removing non-deterministic behavior
    • Replace or seed random sources where used
    • Provide explicit props for time-dependent components (e.g., calendar value/today)
    • Mock or inject deterministic data where appropriate
  • Make StoryWright wait for assets/images/fonts to finish loading.
    Add reliable wait-for selectors or network-idle checks in story capture flow
    Consider explicit story-level readiness signals (e.g., a DOM attribute like data-vrt-ready="true") that StoryWright will wait for before capture
  • Investigate Callout and Positioning pixel diffs

Additional context

No response

Have you discussed this feature with our team

@Hotell

Validations

  • Check that there isn't already an issue that requests the same feature to avoid creating a duplicate.

Priority

Medium

Metadata

Metadata

Assignees

Type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions