GS: Accurate UV rounding for axis-aligned prims.

[TJnotJT]

Status: Everything needs to be retested and SW has to be ported to ARM.

Description of Changes

Do UV rounding at the pixel level (pixel shader for HW and scanline renderer for SW). Currently only applies to native resolution and point-sampled draws.

This replaces my previous attempt at vertex level adjustment (#14029), as doing it per-pixel is probably more accurate.

As before, this is built on work by refraction (#6553).

Rationale behind Changes

Intended to fix issues in some games that rely on accurate UV rounding such as:

• Beyond Good & Evil water reflections: Beyond good and evil : broken graphics #1986
• Dynasty Warriors 5 lines: [BUG]: Dynasty Warriors 5 seams/lines on ground #5218
• Tenchu 2 (mentioned by mrrguest in GS: Split and round sprites to emulate UV rounding error. #14029, and in Meta: GS SW issue #1562).
• Yu-Gi-Oh font: [BUG]: Yu-Gi-Oh Duelist of the roses (SLUS-20515) font not displaying correctly #13659
• Gallop Racer 6 UI: [BUG]: Gallop Racer 6 -Revolution- & Gallop Racer Lucky7 Texture bug in SW. #6375
• Dark Cloud 2 UI: [BUG]: Dark Cloud 2 (SCUS-97213): HUD visual glitch (texture sampling?) #5261
• Dot Hack text: .hack//INFECTION: Minor Font-Rendering and Texture Glitch #2136
• Samurai Showdown lines: [BUG]: Samurai Shodown Anthology graphical issues in HW renderer #14002
• Klonoa 2 lines: [BUG] Klonoa 2: horizontal line artifacts in cutscenes on software renderer #6724
• Battle Gear lines: [BUG]: Battle Gear 2 - Double-rendered textures #14263

Others are mentioned in #6553, though I haven't checked all.

Further Details

Implementation

• Applies to sprites and triangles that form axis-aligned quads.
• Do a preprocessing step on vertices to determine whether UV should be rounded up or down for each primitive.
• ST handling: if Q is flat for very quad, convert ST to UV and do the same steps.
• Send the per-primitive information to HW shaders and SW scanline renderer as a 32 bit attribute (use Q since it's not otherwised used for UV draws).
• Do UV rounding in the pixel shader/scanline renderer before texture sampling.

Limitations

• Applies only to native resolution currently (upscaling is not handled).
• Only handles primitives that have nice coordinates:
  • X, Y, U, V, must be half-pixel aligned.
  • The ratios dX/dU, dY/dV must be a fraction a/b in lowest terms where b < 32 (if the denominator is too large, it could lead to incorrect rounding).
• May reproduce some behavior on PS2 that doesn't always look better. Example below.

Performance

• I expect a small performance impact. However, since this only applies to sprites and axis-aligned triangles, I expect it to be mostly unnoticeable .

Hardware Details

The following rounding rules for axis-aligned prims were determined using hardware tests, though it's possible they are nuances that are not fully understood.

Assume the corners of the quad are (X0, U0, Y0, V0), (X1, U1, Y1, V1). Let dX = X1 - X0 and similarly for dU, dY, dV. Sprites are specified by these two vertices. For axis-aligned triangles that form quads, assume these are the two right angle corners.

Sprites

U and V are rounded independently, so we only discuss X and U below:

• If dU > 0, then U is rounded down.
• If dU < 0, then U is rounded up.
• Exception: If we're at X0 exactly (i.e. X0 is on a pixel center), then round U up.
• Exception: If dX is a power of two, then round U up.

Hypothesis:

• The GS's default rounding behavior is to round up on a texel boundary (if there's no error).
• The GS computes dU/dX in fixed point and truncates it towards 0 (no error if dX is a power of two).
• The rasterizer steps sprites in the direction of dX, regardless of whether dX < 0 (right to left) or dX > 0 (left to right). This would explain round down when dU > 0.
• The X0 exception applies because no error has propagated at X0. This will only apply if X0 is on the left (i.e. X0 < X1) since the GS does not draw right edges.

Axis-aligned Triangles

• If dU/dX, dV/dY > 0, then round U, V down.
• If dU/dX, dV/dY < 0, then round U, V up.
• Exception: If we're on a left edge (i.e. min(X0, X1) is on a pixel center), then round U up. If we're on a top edge (i.e. min(Y0, Y1) is on a pixel center), then round V up.
• Exception: If the triangle has a left edge that goes from bottom to top (for an axis-aligned triangle this can only happen if the right angle corner is the bottom-right), then flip the sign of dY/dV above.
• Exception (overrides previous exceptions): If both dX and dY are powers of two, then round both U, V up.

Hypotheses:

• The GS's default rounding behavior if exactly on a texel boundary (without error) is to round up.
• The GS computes dU/dX, dV/dY in fixed point and truncates them towards 0.
• The reason for both dX and dY needing to be powers of two in the rule could be because triangle setup uses the area, which is a power of two iff both dX and dY are powers of two.
• The GS steps the left edge of the triangle from left-to-right (if not vertical) or top-to-bottom (if vertical), and steps each scanline from left-to-right. This would explain why the sign of dU/dX, dV/dY matters (rather than just dU or dV as with sprites), and why the rounding of V is flipped if the left edge goes from bottom to top.
• The left/top edge exception applies because no error has propagated to cause round down. The GS does not draw right/bottom edges, so there is no analogous exception for those.

Additional Details

• The rules seems to be the same when ST coordinates are used and Q is flat (in which case we can pre-divide vertices by Q and convert to UV). This is based on only a few tests with Q = 1.0 and without checking edges cases like S, T, Q < 0, etc.
• The same rules seems to apply to bilinear draws. Additionally, it seems that the GS truncates coordinates down to the nearest 1/16 texel before sampling. This means that round down errors result in bilinear sampling off by 1/16 texel, but round up errors give exact sampling. (Note: The SW renderer already truncates to 1/16 before sampling so no changes were required. I didn't implemented this in HW, since it probably won't look so good if this method is extended to upscaling.)

Examples

Armored_Core_-_Nexus_Disc_2_SLPS-25339_20221021180346.gs.xz (reproducing buggy PS2 behavior, see right side of lock-on box):

PS2

Master SW

PR SW

---

Gallop Racer Inbreed_SLPS-25701.gs.xz (bottom-right triangles being rounded differently, zoom in on the gray circular things in the top-left UI)

PS2

Master SW

PR SW

---

World Heroes Anthology_SLES-55233_20250314193501.gs.xz (round down error + bilinear causing slight blurriness compared to master)

PS2

Master SW

PR SW

Suggested Testing Steps

Use any renderer, HW or SW. The setting is on by default, though it can be disabled with the INI setting:

[EmuCore/GS]
AccurateUVRounding = 0

Did you use AI to help find, test, or implement this issue or feature?

Yes, as a reference to help porting things to the scanline JIT, Metal, and probably other places.

[AmandaRoseChaqueta]

Fixes lines showing up on some menus for armored core 2: #2394
Left is PR, Right is Master

Software:

Vulkan:

Also helps in-game hud artifacting

[mrrguest]

Fixes Mercenaries
#1562 (comment)

[bigol83]

Dark Cloud 2 ( i have PAL version, Dark Chronicle) UI glitches are fixed in software mode
Vulkan renderer doesn't fix those issues (at least it doesn't at 3x resolution) and when using Native resolution it is completely broken

here is a gsdump
Dark Chronicle_SCES-51190_20260301213806.gs.zip

i checked to be sure and with AccurateUVRounding set to 0 (disabled) it doesn't have this issue

[AmandaRoseChaqueta]

Dark Cloud 2 ( i have PAL version, Dark Chronicle) UI glitches are fixed in software mode Vulkan renderer doesn't fix those issues (at least it doesn't at 3x resolution) and when using Native resolution it is completely broken

The pr is only for native res, for now. But also I cannot reproduce this on linux...except for OpenGL:

OpenGL:

Vulkan:

Software:

Vulkan is fixed too:

[TJnotJT]

@bigol83 @AmandaRoseChaqueta Are there any shader compilation/linking errors in the log when it breaks? I'm not able to reproduce for some reason.

[AmandaRoseChaqueta]

@bigol83 @AmandaRoseChaqueta Are there any shader compilation/linking errors in the log when it breaks? I'm not able to reproduce for some reason.

Let me check...
oh wow. there are some bad shader errors
Emulog:
emulog.txt

bad shader:
pcsx2_bad_shader_1.txt
pcsx2_bad_shader_2.txt
pcsx2_bad_shader_3.txt
pcsx2_bad_shader_4.txt
pcsx2_bad_shader_5.txt
pcsx2_bad_shader_6.txt

[bigol83]

i also have lots of bad shaders in my log, put them all here in a zip file

pcsx2_bad_shaders.zip

[TJnotJT]

Thanks both. My drivers might be a bit more lenient with GLSL. Hopefully the last push resolves it.

[bigol83]

it is better with Vulkan at native but not completely fixed

[mrrguest]

Shadow Hearts - Covenant
Master SW

PR SW

Shadow Hearts - From the New World (top-right UI) Still some gaps.
Master SW

PR SW

Shadow Hearts - From the New World_SLUS-21326_20260301143235.zip

[AmandaRoseChaqueta]

it is better with Vulkan at native but not completely fixed

I suggest resetting your settings completely, mine doesn't look like that at all

Vulkan:

openGL is broken like this: (But not longer shows bad shaders)

[Shifroval]

Tested your latest PR in dot hack (Mirror SW renderer UV rounding setup in HW renderer SW draw), for native it remains fixed, but for upscaling it's not. The behaviour this time is consistent no matter what multiplier I choose. With all scaling modes on screen sprites are shifted to the side by the same amount. It happened before, but now I checked with other elements on the screen like a map, it's very noticable there. If you look closely, the whole image is shifted a bit to the left side (better use a picture viewer, so you can scroll between two images). Software mode behaves the same as native hw. Scaled lookes more correct if you look at map elements, especially the portal symbol. It has a black bar to the left which shouldn't be there, because now it's shifted. Subtitles and character menu have distinct black background under them, now when it's shifted it's more visible.

Screens and dumps, first is native, second is 2x.

.hack Infection Part 1_SLUS-20267_20260311055117.gs.zip

.hack Infection Part 1_SLUS-20267_20260311055128.gs.zip

Before that I checked older builds from gs-accurate-uv branch as you pushed them and they each worked differently (some where just broken), so I never reported anything.

Edit.
It may be just my imagination about the whole image shift (since it's 3d and your pr shouldn't affect it), native image has very low quality edges, but 2x adds sharpness to it, visibly changing the whole image.

[TJnotJT]

@Shifroval Thanks again for the info. Just to clarify, this PR only affects native resolution and doesn’t change upscaling. I’m using the dump you provided to test some upscaling fixes on a separate branch, which I’ll PR separately if/when it’s ready.

[Shifroval]

@Shifroval Thanks again for the info. Just to clarify, this PR only affects native resolution and doesn’t change upscaling. I’m using the dump you provided to test some upscaling fixes on a separate branch, which I’ll PR separately if/when it’s ready.

I understand that, but I'm testing upscaling anyway, since it may come handy in the future. It produces different results now, so that may give you some better understanding what to expect later.

[James-F2]

Dragonball Z - Budokai Tenkaichi 3, Fixed.
Looks nice upscaled too.
*Before this fix the black outline around the character looked segmented, now it looks like a complete and smooth line as it does on the hardware.

[Mrlinkwii]

fixes battle gear 2 before

after:

battle gear 2.zip