DJUI color interpolation / combiner editing / transforms

[Cooliokid956]

Three sets of features that people of all ages will enjoy!

Color interpolation

• djui_hud_set_color_interpolated: Takes twice as many arguments as djui_hud_set_color

u8 prevR, u8 prevG, u8 prevB, u8 prevA, u8 r, u8 g, u8 b, u8 a

Combiner editing

• djui_hud_set_combiner

Sets the current DJUI HUD combiner.
Each part uses the following equation: `P = (A - B) * C + D`.
Cycle 2 may be used to extend the equation, with the result of the previous cycle accessible through CS_COMBINED

void djui_hud_set_combiner(u8 cycle, bool alpha, OPTIONAL enum CombinerSource a, OPTIONAL enum CombinerSource b, OPTIONAL enum CombinerSource c, OPTIONAL enum CombinerSource d);

• djui_hud_reset_combiner: Resets the above, reverting combine modes back to the defaults for each element
• djui_hud_set_combiner_cycles: Sets the amount of cycles used in rendering

Transforms

• djui_hud_transform: Applies a transformation (translate, rotate, scale) to subsequent DJUI HUD elements until closed
• djui_hud_transform_interpolated: The interpolated version of the previous function! Takes twice as many arguments
• djui_hud_close_transform: Closes the previous transform
• djui_hud_reset_transform: Closes all current transforms, restoring a fullscreen state

[Cooliokid956]

Hi! I've had this PR in the works for a while but the third and final feature has had me stumped for a while... it all seems to work but somewhere along the way crashing has become prominent even when using regular DJUI functions... so I'd like to ask for guidance on the matter, and maybe also a review on the two already completed features. (@coderabbitai)

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

Hi @Cooliokid956! Happy to help. Let me pull up the PR code to review the completed features and investigate potential crash causes in the transforms implementation.

---

🏁 Script executed:

#!/bin/bash
# Get list of files changed in the PR
gh pr view 1216 --repo coop-deluxe/sm64coopdx --json files --jq '.files[].path'

Length of output: 518

---

🏁 Script executed:

#!/bin/bash
# Get the full diff of the PR
gh pr diff 1216 --repo coop-deluxe/sm64coopdx

Length of output: 89346

---

@Cooliokid956 Thanks for sharing — happy to dig in! Here's a review of what's in the PR, along with analysis of the crash.

---

🐛 Root Cause of the Crashes

1. djui_hud_close_transform never records an interp entry (most likely crash source)

This is the central issue. In the real-time display list, djui_hud_close_transform correctly calls gSPPopMatrix. But patch_djui_hud (the interpolation replay pass) never sees a pop for the transform, because no interp entry is ever created for djui_hud_close_transform.

djui_hud_transform_interpolated creates an interp entry with noMtx = true, which tells the replay pass not to pop at the end of that entry's replay. This is intentional to keep the transform matrix alive for subsequent render elements. But the matching close is simply never recorded:

// djui_hud_close_transform — real-time pass only, nothing goes into sInterpHuds
void djui_hud_close_transform(void) {
    if (sTransformDepth) {
        gSPPopMatrix(gDisplayListHead++, G_MTX_MODELVIEW);
        sTransformDepth--;
    }
}

Every frame that patch_djui_hud runs, the transform matrix is pushed but never popped. This silently overflows the matrix stack after a few frames, corrupting GPU state and crashing.

Fix: Add a new INTERP_HUD_CLOSE_TRANSFORM type (or a simpler sentinel), create an interp entry in djui_hud_close_transform, and handle it in patch_djui_hud with a plain gSPPopMatrix.

// In djui_hud_close_transform:
void djui_hud_close_transform(void) {
    if (sTransformDepth) {
        gSPPopMatrix(gDisplayListHead++, G_MTX_MODELVIEW);
        sTransformDepth--;

        // Record pop for the interpolation pass
        struct InterpHud *interp = djui_hud_create_interp();
        if (interp) {
            interp->noMtx = true;
            djui_hud_create_interp_gfx(interp, INTERP_HUD_CLOSE_TRANSFORM);
        }
    }
}

// In patch_djui_hud switch:
case INTERP_HUD_CLOSE_TRANSFORM:
    gSPPopMatrix(gDisplayListHead++, G_MTX_MODELVIEW);
    break;

---

2. djui_hud_transform_interpolated mixes prev-position with current-scale

// Current code — note prevX/prevY paired with current scaleX/scaleY
djui_hud_transform_internal(prevX, prevY, scaleX, scaleY, interp);

This passes the previous frame's position but the current frame's scale to the real-time display list, which is inconsistent. It should pass either all-current values (so the real-time display list shows the canonical "this frame" state before interpolation overrides it), or all-prev. Typically you'd want all-current:

djui_hud_transform_internal(x, y, scaleX, scaleY, interp);

---

3. sDjuiHudMtxPushed is a shared static — fragile across nested contexts

djui_hud_setup_matrix() resets this flag at the top of every render call. If djui_hud_transform_internal sets sDjuiHudMtxPushed = true and then any render function calls djui_hud_setup_matrix(), the flag is silently cleared. The sTransformDepth counter compensates for this in the real-time pass, but the interp pass has no equivalent overflow guard — the check if (sTransformDepth == MAX_MATRIX_STACK_SIZE - 3) only exists in the real-time path.

---

✅ Feature 1: Color Interpolation — Generally Good

The implementation is clean. The union reuse in struct InterpHud (sharing posX/posY/scaleX/scaleY with r/g/b/a) is a reasonable space optimization, but I'd suggest a comment to make the aliasing intent clear for future maintainers.

One thing worth confirming: djui_hud_set_color_interpolated calls djui_hud_set_color(r, g, b, a) at the end, which correctly sets both sHudUtilsState.color and emits the gDPSetEnvColor. The interp entry replays the interpolated value correctly via INTERP_HUD_COLOR. 👍

---

⚠️ Feature 2: Combiner Editing — A Few Issues

Optional ABCD params default to CS_0, not CS_KEEP

In the Lua autogen binding, unspecified optional parameters default to (int) 0:

int a = (int) 0;  // CS_0, not CS_KEEP (-1)
if (top >= 3) { a = smlua_to_integer(L, 3); ... }

Since djui_hud_set_combiner only skips applying a value if > CS_KEEP, passing 0 silently sets that slot to CS_0. Calling djui_hud_set_combiner(1, false, CS_TEXTURE) would set a = CS_TEXTURE but also force b, c, d to CS_0, even if you wanted to preserve them. The default should be -1 (i.e., CS_KEEP):

int a = (int) CS_KEEP;  // -1

sCombinerCycleType not reset in djui_hud_reset_combiner

void djui_hud_reset_combiner() {
    gCombinerState.cycles = 1;
    gCombinerOverride = false;
    // sCombinerCycleType is NOT reset here
}

If the combiner was used in 2-cycle mode, sCombinerCycleType remains G_CYC_2CYCLE. After djui_hud_reset_combiner, the next call to djui_gfx_update_combine_mode for a plain rect/texture will compute cycleType = G_CYC_1CYCLE but compare it against the stale G_CYC_2CYCLE, so it will emit a corrective gDPSetCycleType — this is actually fine since the check does handle it. But it's cleaner to reset it:

void djui_hud_reset_combiner() {
    gCombinerState.cycles = 1;
    gCombinerOverride = false;
    sCombinerCycleType = G_CYC_1CYCLE;  // or ~0 to force re-emit
}

Initial combiner state uninitialized

gCombinerState.cycle is zero-initialized by default (global), which means all a/b/c/d slots start as CS_0. This is fine, but it means a user calling djui_hud_set_combiner(1, false, CS_KEEP, CS_KEEP, CS_KEEP, CS_KEEP) expecting to "use defaults" would be using all-zero, not the default combine mode. Worth documenting.

---

Summary

Issue	Severity
djui_hud_close_transform missing interp entry → matrix stack overflow	🔴 Critical / crash
djui_hud_transform_interpolated passes prevX/prevY + current scale	🟠 Bug
Combiner optional params default to CS_0 not CS_KEEP	🟡 Correctness
sCombinerCycleType not reset in djui_hud_reset_combiner	🟡 Minor
Color interpolation feature	✅ Looks good

The fix for the crash is focused: djui_hud_close_transform needs to enqueue an interp entry so the interpolation pass can balance every push with a pop. Let me know if you'd like me to help sketch out more of the solution!

🐇 (ó﹏ò｡)

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 425b5c5f-427b-4977-b1e6-2331ab0b6610

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}