Unexpected Y-axis translation when using `ortho()` and `resetMatrix()`

[SableRaf]

Note

This issue is a repost of processing/processing#6175 by @usuallyannoyed, updated and edited for tone and clarity.

Description

The ortho() function does not behave as expected.

Expected Behavior

ortho(0, width, 0, height) with a cleared modelview matrix (resetMatrix) should allow users to draw within the specified bounds (e.g., 0 to width on the x-axis and 0 to height on the y-axis).

Current Behavior

Instead, the drawing area behaves incorrectly, requiring adjustments (e.g., translating -height on the y-axis) to achieve the expected results. This behavior makes it unintuitive for users who expect the specified bounds to align with the visible viewport.

Steps to Reproduce

void setup() {
  size(500, 500, P3D);
}

void draw() {
  ortho(0, width, 0, height);
  resetMatrix(); // clear modelview matrix
  fill(255);
  
  // Uncomment the line below to 'fix' the behavior
  // translate(0, -height); 
  
  beginShape(TRIANGLES);
  vertex(640, 640);
  vertex(0, 640);
  vertex(0, 0);
  endShape();
}

Observed Result

The triangle only becomes visible when compensating for the unexpected offset by translating the y-axis.

Expected Result

The triangle should appear within the specified ortho bounds without additional transformations.

Environment

• Operating System: macOS 13.4
• Processing Version: 4.3.1

Possible Causes / Solutions

The issue appears to stem from the following block of code in the Processing source:

processing4/core/src/processing/opengl/PGraphicsOpenGL.java

Lines 4482 to 4486 in 937f528

	// The minus sign is needed to invert the Y axis.
	projection.set(x, 0, 0, tx,
	0, -y, 0, ty,
	0, 0, z, tz,
	0, 0, 0, 1);

Despite the comment saying that The minus sign is needed to invert the Y axis., this implementation does not fully invert the y-axis as intended. Instead, it flips the y-axis around zero, resulting in all the y-coordinates being offset into negative space.

Suggested Fix

To invert the y-axis properly in normalized device coordinates (NDC), pre-multiply the projection matrix with a -1 scale for the y-axis. This can be achieved by modifying the projection matrix calculation as follows:

// Add a pre-multiplied -1 y scale to flip the y axis in NDC.
    projection.set(x,  0, 0, tx,
                   0, -y, 0, -ty,
                   0,  0, z, tz,
                   0,  0, 0,  1);

This approach ensures the y-axis behaves as expected without requiring additional transformations.

Additional comments

This behavior stems from Processing's choice to have the Y-axis increase downward, consistent with traditional computer graphics. While reasonable, the implementation relies on ad-hoc adjustments throughout the code. Centralizing it at a lower level (NDC or viewport) and exposing it as an optional flag would provide more clarity and flexibility.