Skip to content

Add image comparison metrics, tolerance, options pattern, and difference image generation#374

Merged
mattleibow merged 23 commits intomainfrom
copilot/explore-pixel-comparer-libraries
Mar 5, 2026
Merged

Add image comparison metrics, tolerance, options pattern, and difference image generation#374
mattleibow merged 23 commits intomainfrom
copilot/explore-pixel-comparer-libraries

Conversation

Copy link
Contributor

Copilot AI commented Mar 3, 2026

Summary

Adds standard image comparison metrics, tolerance-based comparison, an extensible options pattern (SKPixelComparerOptions), alpha channel comparison, and per-channel difference image generation to SKPixelComparer.

New metrics on SKPixelComparisonResult

  • SumSquaredError (long) — Σ(ΔR² + ΔG² + ΔB²)
  • MeanAbsoluteError — per-channel MAE, range [0, 255]
  • MeanSquaredError — per-channel MSE
  • RootMeanSquaredError — RMSE (the metric MAUI's MagickNetVisualComparer uses)
  • NormalizedRootMeanSquaredError — RMSE / 255, range [0, 1] for threshold checks
  • PeakSignalToNoiseRatio — PSNR in dB
  • ChannelCount — 3 (RGB) or 4 (RGBA), used to normalize MAE/MSE

SKPixelComparerOptions

Extensible options class replacing boolean parameters:

var options = new SKPixelComparerOptions
{
    TolerancePerChannel = true,  // Check each channel independently (default)
    CompareAlpha = true          // Include alpha in comparison (default: false)
};
var result = SKPixelComparer.Compare(expected, actual, options);

Options work with all comparison modes (base, tolerance, mask) and all input types (file paths, SKImage, SKBitmap, SKPixmap).

Tolerance-based comparison

Like ImageMagick's fuzz — a uniform per-pixel threshold without needing a full mask image:

var result = SKPixelComparer.Compare(expected, actual, tolerance: 10);

Per-channel difference image

GenerateDifferenceImage produces actual RGB difference values per pixel (vs. the binary B/W GenerateDifferenceMask):

using var diff = SKPixelComparer.GenerateDifferenceImage(expected, actual);

Bug fixes

  • Premultiplied alpha: GetNormalizedBitmap now uses SKAlphaType.Unpremul so semi-transparent pixels have correct RGB values
  • Platform color type: Diff images are built in BGRA8888 (matching SKColor memory layout) then converted to platform default via canvas draw
  • Resource leak: resultBitmap in GenerateDifferenceMask/GenerateDifferenceImage is now properly disposed
  • Divide-by-zero: Metrics return 0 when TotalPixels is 0

Other changes

  • Nullable annotations enabled (#nullable enable) on all comparer files
  • Compare(SKImage, SKImage) and Compare(SKImage, SKImage, options) delegate to the tolerance overload (no duplicated loops)
  • Full XML documentation on all public APIs
  • Comprehensive user-facing docs at docs/docs/pixel-comparer.md

Backward compatibility

  • Existing SKPixelComparisonResult constructors preserved (3-param and 4-param chain to new 5-param with channelCount: 3)
  • All original Compare method signatures unchanged

Blazor sample

The Pixel Comparer demo page showcases metrics display, tolerance slider, and difference image visualization with responsive layout.

Tests

188 tests across 8 focused test files:

File Tests Coverage
SKPixelComparerCompareTest 19 Base comparison, metrics, null args, file overloads
SKPixelComparerToleranceTest 16 Tolerance, per-channel, boundary, max value
SKPixelComparerMaskTest 9 Mask comparison, bitmap/pixmap overloads, dimension mismatch
SKPixelComparerOptionsTest 9 Options pattern, CompareAlpha, semi-transparent premul
SKPixelComparerDifferenceImageTest 7 Per-channel diff, null args, size validation
SKPixelComparerDifferenceMaskTest 3 Binary mask, null args, size validation
SKColorLayoutTest 4 Platform color type, encode roundtrip verification
BlurHash tests 121 Pre-existing (unchanged)

Copilot AI and others added 2 commits March 3, 2026 23:21
…ifference image generation

- Enhance SKPixelComparisonResult with SumSquaredError, MeanAbsoluteError, MeanSquaredError,
  RootMeanSquaredError, NormalizedRootMeanSquaredError, and PeakSignalToNoiseRatio
- Add tolerance-based Compare overloads (like ImageMagick's fuzz) for all input types
- Add GenerateDifferenceImage method for per-channel difference visualization
- Add 22 new tests covering all new functionality
- Update pixel-comparer documentation

Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
…mpatibility

Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate pixel comparer libraries for feature alignment Add standard image comparison metrics, tolerance-based compare, and difference image generation Mar 3, 2026
@mattleibow
Copy link
Collaborator

@copilot please review these code changes and make sure we are doing it right.

github-actions bot pushed a commit that referenced this pull request Mar 3, 2026
@github-actions
Copy link

github-actions bot commented Mar 3, 2026

📖 Documentation Preview

The documentation for this PR has been deployed and is available at:

🔗 View Staging Documentation

🔗 View Staging Blazor Sample

This preview will be updated automatically when you push new commits to this PR.


This comment is automatically updated by the documentation staging workflow.

Copy link
Contributor Author

Copilot AI commented Mar 3, 2026

@copilot please review these code changes and make sure we are doing it right.

I reviewed the full diff across all 4 changed files. Here's a summary:

Metrics (SKPixelComparisonResult) — All formulas are mathematically correct and follow standard definitions used by ImageMagick and MAUI's MagickNetVisualComparer:

  • MAE/MSE normalize by TotalPixels × 3 (per-channel average)
  • PSNR uses standard 10 × log₁₀(MAX² / MSE) with for identical images
  • SumSquaredError uses long to prevent overflow on large images

Tolerance-based comparison — Semantics match ImageMagick's fuzz: pixels with |ΔR|+|ΔG|+|ΔB| ≤ tolerance are excluded from all metrics. tolerance=0 is verified equivalent to standard comparison via test.

Backward compatibility — Old 3-param SKPixelComparisonResult constructor preserved, chains to new 4-param constructor with sumSquaredError: 0.

GenerateDifferenceImage — Follows same pattern as existing GenerateDifferenceMask.

Tests — 22 new tests covering metric correctness (boundary + formula), tolerance edge cases, and diff image. All 63 tests pass (41 original + 22 new).

No issues found — the implementation looks correct.

@mattleibow
Copy link
Collaborator

@copilot can you make sure to update the sample and ensure all the apis have docs.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands SKPixelComparer in SkiaSharp.Extended to provide more standard image comparison metrics, add a tolerance-based comparison mode, and generate a per-channel (RGB) difference image, along with corresponding docs and tests.

Changes:

  • Added sum-squared error tracking and derived metrics (MAE/MSE/RMSE/NRMSE/PSNR) to SKPixelComparisonResult.
  • Added tolerance-based Compare(...) overloads and GenerateDifferenceImage(...) overloads to SKPixelComparer.
  • Updated Pixel Comparer documentation and extended comparer test coverage for the new behaviors.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
tests/SkiaSharp.Extended.Tests/Comparer/SKPixelComparerTest.cs Adds tests for new metrics, tolerance compare, and difference image generation.
source/SkiaSharp.Extended/Comparer/SKPixelComparisonResult.cs Adds SumSquaredError + computed metric properties and a new ctor overload.
source/SkiaSharp.Extended/Comparer/SKPixelComparer.cs Computes SSE in comparisons; adds tolerance-based comparisons and difference image generation APIs.
docs/docs/pixel-comparer.md Documents new metrics, tolerance compare usage, and difference image generation.

| `NormalizedRootMeanSquaredError` | `double` | RMSE divided by 255 (range: 0–1) |
| `PeakSignalToNoiseRatio` | `double` | PSNR in dB (∞ for identical images) |

The **RMSE** metric is commonly used in visual testing tools (such as .NET MAUI's `VisualTestUtils.MagickNet`) to quantify image differences as a single number. The **NormalizedRootMeanSquaredError** maps this to a 0–1 range, where 0 means identical and 1 means maximum difference—useful for threshold-based pass/fail testing.
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Process: since this PR changes the Pixel Comparer public API and user-visible behavior/outputs, please add updated screenshots of the Blazor sample’s Pixel Comparer page (/pixelcomparer) to the PR conversation so reviewers can visually verify the feature end-to-end.

Copilot uses AI. Check for mistakes.
Comment on lines +361 to +385
public static SKImage GenerateDifferenceImage(string firstFilename, string secondFilename)
{
using var first = SKImage.FromEncodedData(firstFilename);
using var second = SKImage.FromEncodedData(secondFilename);
return GenerateDifferenceImage(first, second);
}

public static SKImage GenerateDifferenceImage(SKBitmap first, SKBitmap second)
{
using var firstPixmap = first.PeekPixels();
using var secondPixmap = second.PeekPixels();
return GenerateDifferenceImage(firstPixmap, secondPixmap);
}

public static SKImage GenerateDifferenceImage(SKPixmap first, SKPixmap second)
{
using var firstWrapper = SKImage.FromPixels(first);
using var secondWrapper = SKImage.FromPixels(second);
return GenerateDifferenceImage(firstWrapper, secondWrapper);
}

public static SKImage GenerateDifferenceImage(SKImage first, SKImage second)
{
Validate(first, second);

Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new GenerateDifferenceImage overloads are public but missing XML documentation comments. Please add

// (and exception docs for the SKImage overload) consistent with GenerateDifferenceMask so the API reference is complete and users know the output pixel semantics (RGB = absolute per-channel diffs).

Copilot uses AI. Check for mistakes.
…trics, tolerance slider, and difference image

Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 4, 2026

@copilot can you make sure to update the sample and ensure all the apis have docs.

Done in 336c0e7.

XML docs added to all new public APIs:

  • SKPixelComparisonResult: 4-param constructor, SumSquaredError, MeanAbsoluteError, MeanSquaredError, RootMeanSquaredError, NormalizedRootMeanSquaredError, PeakSignalToNoiseRatio
  • SKPixelComparer: all 4 tolerance-based Compare overloads, all 4 GenerateDifferenceImage overloads

Blazor sample updated with:

  • New metrics row (MAE, RMSE, NRMSE, PSNR)
  • Interactive tolerance slider
  • Difference Image panel alongside the Difference Mask
  • Links to API and documentation pages

Pixel Comparer sample screenshot

github-actions bot pushed a commit that referenced this pull request Mar 4, 2026
@mattleibow
Copy link
Collaborator

@copilot please address the review comments and also make the sample page more fluid. Maybe we can put the mask and diff images below the comparison images when on a phone?

…n mobile

Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 4, 2026

@copilot please address the review comments and also make the sample page more fluid. Maybe we can put the mask and diff images below the comparison images when on a phone?

Done in 0585459. The review comments about missing XML docs were already addressed in 336c0e7 — all new public APIs have complete XML doc comments.

For responsiveness, the image grid now uses col-6 col-lg-3 so on phones the mask and diff images stack below the comparison images (2-per-row). Metrics also use col-6 col-sm-3 for the same treatment. Added g-3 gutter spacing for cleaner wrapping.

Desktop:
Desktop layout

Mobile (375px):
Mobile layout

@mattleibow
Copy link
Collaborator

@copilot For the results box, can you add the sub note to the first 4 as well to say what they are

github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
- Add SKPixelComparerOptions class with TolerancePerChannel and CompareAlpha properties
- Add ChannelCount to SKPixelComparisonResult (3 for RGB, 4 for RGBA)
- Add options-based Compare overloads for all input types (base, mask, tolerance)
- Implement CompareAlpha support in all three core comparison methods
- Keep existing bool tolerancePerChannel overloads as thin wrappers
- Add 9 new tests for options pattern and alpha comparison
- Update docs: options section, ChannelCount in results table, Learn More links

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
mattleibow and others added 2 commits March 5, 2026 03:00
…Options

The bool overloads were just added in this PR, so removing them is not
a breaking change. Users should use SKPixelComparerOptions instead.

- Remove 8 bool tolerancePerChannel method overloads (4 mask, 4 tolerance)
- Update tests to use SKPixelComparerOptions
- Update docs to reference options pattern instead of bool parameter

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes duplicated loop logic by having the parameterless overload
delegate to Compare(SKImage, SKImage, SKPixelComparerOptions) with
null options, preserving existing RGB-only behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
mattleibow and others added 2 commits March 5, 2026 03:06
Add #nullable enable to SKPixelComparer.cs, SKPixelComparisonResult.cs,
and SKPixelComparerOptions.cs. Mark options parameters as nullable since
they accept null for default behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With tolerance=0 both per-channel and sum paths produce identical
results, so the options-only overload can simply delegate to
Compare(SKImage, SKImage, 0, options) eliminating a duplicate loop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
mattleibow and others added 2 commits March 5, 2026 03:11
…erloads

- Options delegation: Compare(img, img, opts) matches Compare(img, img, 0, opts)
- CompareAlpha with tolerance sum-based mode
- CompareAlpha with mask sum-based mode
- Bitmap and pixmap overloads with options
- Alpha-only diff detected when RGB within tolerance
- Tolerance with bitmap/pixmap + options

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nceImage

Use SKColorType.Bgra8888 explicitly for diff bitmaps, matching the
format used by GetNormalizedBitmap. The default SKImageInfo uses
PlatformColorType which is RGBA on macOS, causing channel swaps
when writing/reading SKColor values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
GetPixelSpan<SKColor>() is a raw memory blit with no color-type
swizzle. SKColor's ARGB uint matches BGRA8888 byte layout on LE,
so raw span writes only produce correct pixels in BGRA8888 buffers.
On RGBA8888 platforms (e.g. macOS), writing SKColor to a platform-
default bitmap via raw span swaps R and B in the encoded output.

Fix: build diff in BGRA8888 for correct span writes, then canvas-
draw to a platform-default bitmap before returning. This gives
callers an image in the expected platform layout that encodes and
displays correctly on all platforms.

Add SKColorLayoutTest with encode-roundtrip tests that verify the
diff image channels survive PNG encode/decode regardless of platform.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
Split the monolithic 775-line test class into focused files:
- SKPixelComparerTestBase: shared constants and helpers
- SKPixelComparerCompareTest: base comparison and metrics (15 tests)
- SKPixelComparerToleranceTest: tolerance-based comparison (14 tests)
- SKPixelComparerMaskTest: mask-based comparison (5 tests)
- SKPixelComparerDifferenceMaskTest: GenerateDifferenceMask (1 test)
- SKPixelComparerDifferenceImageTest: GenerateDifferenceImage (5 tests)
- SKPixelComparerOptionsTest: SKPixelComparerOptions/alpha (7 tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
Bug fixes:
- Dispose resultBitmap in GenerateDifferenceMask/Image with using
- Use SKAlphaType.Unpremul in GetNormalizedBitmap so semi-transparent
  pixels have correct unpremultiplied RGB values for comparison

New tests (15):
- Null argument validation for Compare, mask, and diff methods
- File-path overload matches SKImage overload
- Tolerance boundary (diff == tolerance is not error)
- Mask dimension mismatch throws
- Mask compare with bitmap/pixmap overloads
- Max tolerance excludes all pixels
- CompareAlpha + TolerancePerChannel=false combined
- Semi-transparent premul: CompareAlpha=false ignores alpha correctly
- Semi-transparent premul: CompareAlpha=true detects alpha
- Null/size checks for GenerateDifferenceMask/Image

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
@mattleibow mattleibow changed the title Add standard image comparison metrics, tolerance-based compare, and difference image generation Add image comparison metrics, tolerance, options pattern, and difference image generation Mar 5, 2026
- Fix docs to mention 'BGRA8888 unpremultiplied format'
- Add SSE and ChannelCount metrics to Blazor sample
- Reorganize metrics grid into 3 rows for better layout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mattleibow
Copy link
Collaborator

mattleibow commented Mar 5, 2026

Blazor Sample Screenshots — Verified ✅

All Blazor sample pages load and render correctly with no console errors. The Pixel Comparer page was updated with:

  • SSE (Sum of Squared Errors) metric added to the metrics grid
  • ChannelCount metric added showing 3 (RGB) or 4 (RGBA)
  • Metrics reorganized into a clean 3-row grid layout

Pages Verified

Page Status Notes
Home ✅ Pass Renders correctly
Shapes ✅ Pass Renders correctly
Path Interpolation ✅ Pass Renders correctly
BlurHash ✅ Pass Renders correctly
Pixel Comparer ✅ Pass Updated — new SSE/ChannelCount metrics, 3-row grid

Screenshots captured with Playwright (headless Chromium + WebGL) against dotnet run --project samples/SkiaSharpDemo.Blazor. No console errors on any page.

github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
- Add checkboxes for SKPixelComparerOptions in the controls bar
- Dynamic max tolerance (255 per-channel, 765/1020 summed)
- Options affect both Compare metrics and regenerate diffs on change

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
- Fix tolerance docs: default is per-channel (not summed)
- Rename 'Per-Channel Tolerance Mode' to 'Summed Tolerance Mode' (per-channel is default)
- Add max tolerance values for per-channel vs summed modes
- Document ArgumentOutOfRangeException for negative tolerance
- Document mask dimension mismatch InvalidOperationException
- Add RMSE/PSNR to Quick Start example
- Clarify mask per-channel behavior with concrete example

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
@mattleibow mattleibow merged commit 4f16bb8 into main Mar 5, 2026
4 checks passed
@mattleibow mattleibow deleted the copilot/explore-pixel-comparer-libraries branch March 5, 2026 20:56
github-actions bot pushed a commit that referenced this pull request Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants