GS: Split and round sprites to emulate UV rounding error.

[TJnotJT]

Description of Changes

Some games draws point-sampled sprites that should sample exactly at texel boundaries. For such cases, it seems that the GS rounds texels coordinates to the nearest texel. However, in many cases there appears to interpolation error in UVs so that the GS rounds down to the nearest texel instead (details on hardware tests below). This PR attempts to detect such cases, and split and round sprites to account for this.

Credits: This is built on ideas and work by refraction (see #6553).

Rationale behind Changes

Fixes accuracy issues in games that rely on accurate rounding. The initial intention was to fix the Beyond Good and Evil water reflections (see #1986).

Appears to fix #14002.

Caveats:

• There are some cases where text appears to be slightly blurrier by 1 pixel in the PR. However, I've checked a few examples by hand and they seem to appear on the PS2 also (examples below).
• Future work: This PR is conservative in choosing which sprites to modify, so it might not fix all cases. For instance, only sprites whose X/Y dimensions are pixel multiples and such that the gradient dU/dX or dV/dY is an integer are modified.
• Future work: This could probably be extended to axis-aligned triangles also, but for now it handles sprites only.
• Future work: Upscaling is not considered in this PR, but it may require modifications to work correctly.
• Performance: For any draws where sprites are split, the number of polys will be quadrupled (though number of pixels should be the same). This may have a performance impact, though I suspect it will be small since it only applies to point-sampled sprites.

Comparisons

bge.gs.xz (note, red strip that looks like a bug on left occurs on both PS2 and PR)

PS2

Master SW

PR SW

---

Gallop Racer Inbreed_SLPS-25701.gs.xz (UI elements)

PS2

Master SW

PR SW

---

Dragon Ball Z - Budokai Tenkaichi 3_SLUS-21678_20221208223633.gs.xz (character outlines, some might still be a bit off in PR)

PS2

Master SW

PR SW

---

Ace Combat - Squadron Leader_SCES-52424_20250401050853.gs.xz (example where PS2 and PR look slightly worse, "Edge" in HUD looks a bit squished compared to master)

PS2

Master SW

PR SW

---

Wild_ARMs_3_SCUS-97203.gs.xz (see borders of UI elements)

PS2

Master SW

PR SW

---

roguegalaxytext.gs.xz (black line at top in both PS2 and PR, though alignment of circle UI elements is a bit more centered compared to master)

PS2

Master SW

PR SW

---

Dragon Quest VIII - Journey of the Cursed King_SLES-53974_20230321134214.gs.xz ("HP" and "MP" in PS2 and PR are a bit more stretched and may look slightly worse than master)

PS2

Master SW

PR SW

Suggested Testing Steps

Use any renderer, as both SW or HW are affected. Many games may not have not visual differences (although a large number of pixel might have tiny differences), or have subtle changes (UI alignment changes or single black line at edge of frame changed, etc.).

Hardware Tests/Details

Based on hardware tests the following rounding rules seems to apply when UVs are should ("should" means if the interpolation was mathematically exact) be on texel boundaries for sprites (similar conditions may apply to triangles and lines):

• The top-most and/or left-most pixels never seems to have rounding error (since the GS likely rasterizes top-left to bottom-right). They are always rounded up.
• When the width is not power of 2, the interpolated Us other than the left-most pixels seem to round down.
• When the height is not power of 2, the interpolated Vs other than the top-most pixels seem to round down.
• If the width and/or height is a power of 2, the interpolated UVs seem to round up.
• If the directions of U and X (or Y and V) is opposite, the error seems to cause the UVs to round up instead of down (meaning that all UVs are rounded as if there is no error).
• The above rules apply to the horizontal and vertical axes separately (e.g. we can have U round down and V round up).
• There is an exception to these rules for heights or widths > 512 pixels (see below), which was not emulated here.
  Note: The default behavior on the PCSX2 SW and HW renderers appears to be to round up at texel boundaries, hence the inaccuracies.

Additional details:

• Shifting the sprite by any amount (including fractional) does not seem to affect the rounding rules above (as long as pixel centers map to texel boundaries).
• The same rules seem to apply even if dU / dX (or dY / dV) is not exactly 1 (however this PR only modifies sprites where the ratio is an integer >= 1).
• When the sprite dimensions are a power of 2, there appears to be no interpolation error. This may be caused by accuracy of the gradients during interpolation (I think this was suggested to me by TellowKrinkle).
• For heights or widths > 512 pixels, there is sporadic round up/down behavior that I couldn't find a simple pattern for. Again this may have to do with the precision for computing gradients or reciprocals. Seemingly by chance, sprites with 640 width or height round UVs down, which is partly the cause of the Beyond Good and Evil water issue.

The hardware tests were straightforward: draw some sprites and lines with different dimensions and choose UVs in such a way that the sampling values should fall on texel boundaries. No tests were done for triangles, though I'd assume the results would be the same for axis-aligned triangles.

Did you use AI to help find, test, or implement this issue or feature?

I asked AI on advice for types of hardware tests to use to reverse GS internal precision. However, this did not end up being necessary to fix the rounding issues.

[mrrguest]

The 2 Tenchu games are mentioned here: #1562

Tenchu - Wrath of Heaven
Master SW:

PR SW:

Tenchu - Fatal Shadows
Master SW:

PR SW:

Tenchu - Wrath of Heaven_SLUS-20397_20260217201346.zip
Tenchu - Fatal Shadows_SLUS-21129_20260217201605.zip

[TJnotJT]

It seems that underflow (https://en.wikipedia.org/wiki/Arithmetic_underflow) has a different technical meaning than I thought, so changing it to rounding error everywhere.

[TJnotJT]

The 2 Tenchu games are mentioned here: #1562

Thanks for finding this, I mentioned it in the new PR.

Closing this in favor of #14078.