[rcore] Wayland: GLFW framebuffer scale is handled incorrectly

[gfaster]

Please, before submitting a new issue verify and check:

• I tested it on latest raylib version from master branch
• I checked there is no similar issue already reported
• I checked the documentation on the wiki
• My code has no errors or misuse of raylib

Issue description

This is effectively a followup to #4815.

GLFW 4.3 now handles fractional scaling correctly by default, namely changing GLFW_COCOA_RETINA_FRAMEBUFFER to GLFW_SCALE_FRAMEBUFFER and enabling it by default. much of the handling the Raylib to handle when screen size doesn't equal render size is redundant and breaks rendering.

I'm including a workaround that goes back to pre-4.3 behavior. I also wrote a more complete fix, but I don't have the means to test it, so I figured going back to old behavior would be preferable for now.

Environment

OS: NixOS 25.05pre785698.b024ced1aac2 (Warbler)
Kernel: Linux 6.13.11
Display (LG HDR 4K): 3840x2160 @ 60 Hz (as 2259x1271) in 27"
Display (XB271HU): 2560x1440 @ 144 Hz (as 2229x1253) in 27"
WM: KWin (Wayland)
CPU: Intel(R) Core(TM) i7-7700 (8) @ 4.20 GHz
GPU: NVIDIA GeForce RTX 2080 Rev. A

INFO: GL: Supported extensions count: 395
INFO: GL: OpenGL device information:
INFO:     > Vendor:   NVIDIA Corporation
INFO:     > Renderer: NVIDIA GeForce RTX 2080/PCIe/SSE2
INFO:     > Version:  3.3.0 NVIDIA 570.133.07
INFO:     > GLSL:     3.30 NVIDIA via Cg compile

Cmake command:

cmake .. -DUSE_EXTERNAL_GLFW=ON -DBUILD_EXAMPLES=OFF -DCUSTOMIZE_BUILD=ON -DBUILD_SHARED_LIBS=ON

Issue Screenshot

Current behavior is identical to the screenshot in #4815

Code Example

#include "raylib.h"
#include <GLFW/glfw3.h>
#include <stdio.h>

#define RECT_SIZE 100.0f

void
print_info(GLFWwindow *handle)
{
	int glfw_winx, glfw_winy;
	glfwGetWindowSize(handle, &glfw_winx, &glfw_winy);
	printf("GLFW window size:      %d, %d\n", glfw_winx, glfw_winy);

	int glfw_fbx, glfw_fby;
	glfwGetFramebufferSize(handle, &glfw_fbx, &glfw_fby);
	printf("GLFW framebuffer size: %d, %d\n", glfw_fbx, glfw_fby);

	int rl_winx, rl_winy;
	rl_winx = GetScreenWidth();
	rl_winy = GetScreenHeight();
	printf("RL window size:        %d, %d\n", rl_winx, rl_winy);

	int rl_fbx, rl_fby;
	rl_fbx = GetRenderWidth();
	rl_fby = GetRenderHeight();
	printf("RL render size:        %d, %d\n", rl_fbx, rl_fby);

	puts("");
}

void
draw(GLFWwindow *handle)
{
	// Draw a square at each corner of the screen, turning blue if the
	// mouse is over the top left square
	ClearBackground(BLACK);
	Vector2 pos = GetMousePosition();
	Color rcolor = (pos.x < RECT_SIZE && pos.y < RECT_SIZE) ? BLUE : RED;
	Vector2 bot = (Vector2) { (float)GetScreenWidth() - RECT_SIZE, (float)GetScreenHeight() - RECT_SIZE };
	DrawRectangleRec((Rectangle){0.0f, 0.0f, RECT_SIZE, RECT_SIZE}, rcolor);
	DrawRectangleRec((Rectangle){bot.x, 0.0f, RECT_SIZE, RECT_SIZE}, rcolor);
	DrawRectangleRec((Rectangle){bot.x, bot.y, RECT_SIZE, RECT_SIZE}, rcolor);
	DrawRectangleRec((Rectangle){0.0f, bot.y, RECT_SIZE, RECT_SIZE}, rcolor);

	(void)handle;
	// print_info(handle);
}

int
main(void)
{
	glfwInitHint(GLFW_PLATFORM, GLFW_PLATFORM_WAYLAND);

	// SetConfigFlags(FLAG_WINDOW_RESIZABLE | FLAG_MSAA_4X_HINT);
	SetConfigFlags(FLAG_WINDOW_RESIZABLE | FLAG_WINDOW_HIGHDPI | FLAG_MSAA_4X_HINT);
	InitWindow(640, 480, "Issue on Wayland");
	GLFWwindow *handle = glfwGetCurrentContext();
	print_info(handle);
	SetTargetFPS(60);
	while(!WindowShouldClose()) {
		BeginDrawing();
		draw(handle);
		EndDrawing();
	}
}

[raysan5]

much of the handling the Raylib to handle when screen size doesn't equal render size is redundant and breaks rendering.

Definitely, that should be reviewed.

[gfaster]

Via a friend's Mac, it seems like FLAG_WINDOW_HIGHDPI is broken in a similar way on MacOS (seems like not fixed by #4909 ). git-bisect tells us it was broken by 4bed374 -- this should be fixed when GLFW 4.3 behavior is matched