Add image comparison metrics, tolerance, options pattern, and difference image generation#374
Conversation
…ifference image generation - Enhance SKPixelComparisonResult with SumSquaredError, MeanAbsoluteError, MeanSquaredError, RootMeanSquaredError, NormalizedRootMeanSquaredError, and PeakSignalToNoiseRatio - Add tolerance-based Compare overloads (like ImageMagick's fuzz) for all input types - Add GenerateDifferenceImage method for per-channel difference visualization - Add 22 new tests covering all new functionality - Update pixel-comparer documentation Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
…mpatibility Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
|
@copilot please review these code changes and make sure we are doing it right. |
|
📖 Documentation Preview The documentation for this PR has been deployed and is available at: This preview will be updated automatically when you push new commits to this PR. This comment is automatically updated by the documentation staging workflow. |
I reviewed the full diff across all 4 changed files. Here's a summary: Metrics (SKPixelComparisonResult) — All formulas are mathematically correct and follow standard definitions used by ImageMagick and MAUI's
Tolerance-based comparison — Semantics match ImageMagick's fuzz: pixels with Backward compatibility — Old 3-param
Tests — 22 new tests covering metric correctness (boundary + formula), tolerance edge cases, and diff image. All 63 tests pass (41 original + 22 new). No issues found — the implementation looks correct. |
|
@copilot can you make sure to update the sample and ensure all the apis have docs. |
There was a problem hiding this comment.
Pull request overview
This PR expands SKPixelComparer in SkiaSharp.Extended to provide more standard image comparison metrics, add a tolerance-based comparison mode, and generate a per-channel (RGB) difference image, along with corresponding docs and tests.
Changes:
- Added sum-squared error tracking and derived metrics (MAE/MSE/RMSE/NRMSE/PSNR) to
SKPixelComparisonResult. - Added tolerance-based
Compare(...)overloads andGenerateDifferenceImage(...)overloads toSKPixelComparer. - Updated Pixel Comparer documentation and extended comparer test coverage for the new behaviors.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| tests/SkiaSharp.Extended.Tests/Comparer/SKPixelComparerTest.cs | Adds tests for new metrics, tolerance compare, and difference image generation. |
| source/SkiaSharp.Extended/Comparer/SKPixelComparisonResult.cs | Adds SumSquaredError + computed metric properties and a new ctor overload. |
| source/SkiaSharp.Extended/Comparer/SKPixelComparer.cs | Computes SSE in comparisons; adds tolerance-based comparisons and difference image generation APIs. |
| docs/docs/pixel-comparer.md | Documents new metrics, tolerance compare usage, and difference image generation. |
| | `NormalizedRootMeanSquaredError` | `double` | RMSE divided by 255 (range: 0–1) | | ||
| | `PeakSignalToNoiseRatio` | `double` | PSNR in dB (∞ for identical images) | | ||
|
|
||
| The **RMSE** metric is commonly used in visual testing tools (such as .NET MAUI's `VisualTestUtils.MagickNet`) to quantify image differences as a single number. The **NormalizedRootMeanSquaredError** maps this to a 0–1 range, where 0 means identical and 1 means maximum difference—useful for threshold-based pass/fail testing. |
There was a problem hiding this comment.
Process: since this PR changes the Pixel Comparer public API and user-visible behavior/outputs, please add updated screenshots of the Blazor sample’s Pixel Comparer page (/pixelcomparer) to the PR conversation so reviewers can visually verify the feature end-to-end.
| public static SKImage GenerateDifferenceImage(string firstFilename, string secondFilename) | ||
| { | ||
| using var first = SKImage.FromEncodedData(firstFilename); | ||
| using var second = SKImage.FromEncodedData(secondFilename); | ||
| return GenerateDifferenceImage(first, second); | ||
| } | ||
|
|
||
| public static SKImage GenerateDifferenceImage(SKBitmap first, SKBitmap second) | ||
| { | ||
| using var firstPixmap = first.PeekPixels(); | ||
| using var secondPixmap = second.PeekPixels(); | ||
| return GenerateDifferenceImage(firstPixmap, secondPixmap); | ||
| } | ||
|
|
||
| public static SKImage GenerateDifferenceImage(SKPixmap first, SKPixmap second) | ||
| { | ||
| using var firstWrapper = SKImage.FromPixels(first); | ||
| using var secondWrapper = SKImage.FromPixels(second); | ||
| return GenerateDifferenceImage(firstWrapper, secondWrapper); | ||
| } | ||
|
|
||
| public static SKImage GenerateDifferenceImage(SKImage first, SKImage second) | ||
| { | ||
| Validate(first, second); | ||
|
|
There was a problem hiding this comment.
The new GenerateDifferenceImage overloads are public but missing XML documentation comments. Please add
// (and exception docs for the SKImage overload) consistent with GenerateDifferenceMask so the API reference is complete and users know the output pixel semantics (RGB = absolute per-channel diffs).
…trics, tolerance slider, and difference image Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
Done in 336c0e7. XML docs added to all new public APIs:
Blazor sample updated with:
|
|
@copilot please address the review comments and also make the sample page more fluid. Maybe we can put the mask and diff images below the comparison images when on a phone? |
…n mobile Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com>
Done in 0585459. The review comments about missing XML docs were already addressed in 336c0e7 — all new public APIs have complete XML doc comments. For responsiveness, the image grid now uses |
|
@copilot For the results box, can you add the sub note to the first 4 as well to say what they are |
- Add SKPixelComparerOptions class with TolerancePerChannel and CompareAlpha properties - Add ChannelCount to SKPixelComparisonResult (3 for RGB, 4 for RGBA) - Add options-based Compare overloads for all input types (base, mask, tolerance) - Implement CompareAlpha support in all three core comparison methods - Keep existing bool tolerancePerChannel overloads as thin wrappers - Add 9 new tests for options pattern and alpha comparison - Update docs: options section, ChannelCount in results table, Learn More links Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…Options The bool overloads were just added in this PR, so removing them is not a breaking change. Users should use SKPixelComparerOptions instead. - Remove 8 bool tolerancePerChannel method overloads (4 mask, 4 tolerance) - Update tests to use SKPixelComparerOptions - Update docs to reference options pattern instead of bool parameter Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes duplicated loop logic by having the parameterless overload delegate to Compare(SKImage, SKImage, SKPixelComparerOptions) with null options, preserving existing RGB-only behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add #nullable enable to SKPixelComparer.cs, SKPixelComparisonResult.cs, and SKPixelComparerOptions.cs. Mark options parameters as nullable since they accept null for default behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With tolerance=0 both per-channel and sum paths produce identical results, so the options-only overload can simply delegate to Compare(SKImage, SKImage, 0, options) eliminating a duplicate loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…erloads - Options delegation: Compare(img, img, opts) matches Compare(img, img, 0, opts) - CompareAlpha with tolerance sum-based mode - CompareAlpha with mask sum-based mode - Bitmap and pixmap overloads with options - Alpha-only diff detected when RGB within tolerance - Tolerance with bitmap/pixmap + options Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nceImage Use SKColorType.Bgra8888 explicitly for diff bitmaps, matching the format used by GetNormalizedBitmap. The default SKImageInfo uses PlatformColorType which is RGBA on macOS, causing channel swaps when writing/reading SKColor values. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetPixelSpan<SKColor>() is a raw memory blit with no color-type swizzle. SKColor's ARGB uint matches BGRA8888 byte layout on LE, so raw span writes only produce correct pixels in BGRA8888 buffers. On RGBA8888 platforms (e.g. macOS), writing SKColor to a platform- default bitmap via raw span swaps R and B in the encoded output. Fix: build diff in BGRA8888 for correct span writes, then canvas- draw to a platform-default bitmap before returning. This gives callers an image in the expected platform layout that encodes and displays correctly on all platforms. Add SKColorLayoutTest with encode-roundtrip tests that verify the diff image channels survive PNG encode/decode regardless of platform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Split the monolithic 775-line test class into focused files: - SKPixelComparerTestBase: shared constants and helpers - SKPixelComparerCompareTest: base comparison and metrics (15 tests) - SKPixelComparerToleranceTest: tolerance-based comparison (14 tests) - SKPixelComparerMaskTest: mask-based comparison (5 tests) - SKPixelComparerDifferenceMaskTest: GenerateDifferenceMask (1 test) - SKPixelComparerDifferenceImageTest: GenerateDifferenceImage (5 tests) - SKPixelComparerOptionsTest: SKPixelComparerOptions/alpha (7 tests) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bug fixes: - Dispose resultBitmap in GenerateDifferenceMask/Image with using - Use SKAlphaType.Unpremul in GetNormalizedBitmap so semi-transparent pixels have correct unpremultiplied RGB values for comparison New tests (15): - Null argument validation for Compare, mask, and diff methods - File-path overload matches SKImage overload - Tolerance boundary (diff == tolerance is not error) - Mask dimension mismatch throws - Mask compare with bitmap/pixmap overloads - Max tolerance excludes all pixels - CompareAlpha + TolerancePerChannel=false combined - Semi-transparent premul: CompareAlpha=false ignores alpha correctly - Semi-transparent premul: CompareAlpha=true detects alpha - Null/size checks for GenerateDifferenceMask/Image Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix docs to mention 'BGRA8888 unpremultiplied format' - Add SSE and ChannelCount metrics to Blazor sample - Reorganize metrics grid into 3 rows for better layout Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Blazor Sample Screenshots — Verified ✅All Blazor sample pages load and render correctly with no console errors. The Pixel Comparer page was updated with:
Pages Verified
|
- Add checkboxes for SKPixelComparerOptions in the controls bar - Dynamic max tolerance (255 per-channel, 765/1020 summed) - Options affect both Compare metrics and regenerate diffs on change Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix tolerance docs: default is per-channel (not summed) - Rename 'Per-Channel Tolerance Mode' to 'Summed Tolerance Mode' (per-channel is default) - Add max tolerance values for per-channel vs summed modes - Document ArgumentOutOfRangeException for negative tolerance - Document mask dimension mismatch InvalidOperationException - Add RMSE/PSNR to Quick Start example - Clarify mask per-channel behavior with concrete example Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>



Summary
Adds standard image comparison metrics, tolerance-based comparison, an extensible options pattern (
SKPixelComparerOptions), alpha channel comparison, and per-channel difference image generation toSKPixelComparer.New metrics on
SKPixelComparisonResultSumSquaredError(long) — Σ(ΔR² + ΔG² + ΔB²)MeanAbsoluteError— per-channel MAE, range [0, 255]MeanSquaredError— per-channel MSERootMeanSquaredError— RMSE (the metric MAUI'sMagickNetVisualCompareruses)NormalizedRootMeanSquaredError— RMSE / 255, range [0, 1] for threshold checksPeakSignalToNoiseRatio— PSNR in dBChannelCount— 3 (RGB) or 4 (RGBA), used to normalize MAE/MSESKPixelComparerOptions
Extensible options class replacing boolean parameters:
Options work with all comparison modes (base, tolerance, mask) and all input types (file paths,
SKImage,SKBitmap,SKPixmap).Tolerance-based comparison
Like ImageMagick's
fuzz— a uniform per-pixel threshold without needing a full mask image:Per-channel difference image
GenerateDifferenceImageproduces actual RGB difference values per pixel (vs. the binary B/WGenerateDifferenceMask):Bug fixes
GetNormalizedBitmapnow usesSKAlphaType.Unpremulso semi-transparent pixels have correct RGB valuesSKColormemory layout) then converted to platform default via canvas drawresultBitmapinGenerateDifferenceMask/GenerateDifferenceImageis now properly disposedTotalPixelsis 0Other changes
#nullable enable) on all comparer filesCompare(SKImage, SKImage)andCompare(SKImage, SKImage, options)delegate to the tolerance overload (no duplicated loops)docs/docs/pixel-comparer.mdBackward compatibility
SKPixelComparisonResultconstructors preserved (3-param and 4-param chain to new 5-param withchannelCount: 3)Comparemethod signatures unchangedBlazor sample
The Pixel Comparer demo page showcases metrics display, tolerance slider, and difference image visualization with responsive layout.
Tests
188 tests across 8 focused test files: