[rcore] `ToggleBorderlessWindowed()` can't enable window decoration on Linux Mint (X11)

[and3md]

Hi, I tried to add full screen/window mode to my project by using ToggleBorderlessWindowed(). I found that switching to full screen window mode without decoration works great but unfortunately calling ToggleBorderlessWindowed() again does not return to window mode with decorations. I also tried raylib current master and it does not work.

Environment

I use Linux Mint 22.1 with Cinnamon on X11, using NVIDIA GeForce GTX 1060 6GB (driver 550.120), raylib master

Issue workaround

After a little investigation I came across the code in issue #4149 (comment) that works great. Maybe there should be ifdef for linux in code.

Implementation that work for me (based on #4149 (comment) code from master commented) :

// Toggle borderless windowed mode
void ToggleBorderlessWindowed(void)
{
    // Leave fullscreen before attempting to set borderless windowed mode
    bool wasOnFullscreen = false;
    if (CORE.Window.fullscreen)
    {
        // fullscreen already saves the previous position so it does not need to be set here again
        ToggleFullscreen();
        wasOnFullscreen = true;
    }

    const int monitor = GetCurrentMonitor();
    int monitorCount;
    GLFWmonitor **monitors = glfwGetMonitors(&monitorCount);

    if ((monitor >= 0) && (monitor < monitorCount))
    {
        const GLFWvidmode *mode = glfwGetVideoMode(monitors[monitor]);

        if (mode)
        {
            if (!IsWindowState(FLAG_BORDERLESS_WINDOWED_MODE))
            {
                // Store screen position and size
                // NOTE: If it was on fullscreen, screen position was already stored, so skip setting it here
                if (!wasOnFullscreen) CORE.Window.previousPosition = CORE.Window.position;
                CORE.Window.previousScreen = CORE.Window.screen;

                // Set undecorated flag
                //glfwSetWindowAttrib(platform.handle, GLFW_DECORATED, GLFW_FALSE);
                glfwSetWindowMonitor(platform.handle, monitors[monitor], 0, 0, mode->width, mode->height, mode->refreshRate);
                CORE.Window.flags |= FLAG_WINDOW_UNDECORATED;

                // Get monitor position and size
                int monitorPosX = 0;
                int monitorPosY = 0;
                glfwGetMonitorPos(monitors[monitor], &monitorPosX, &monitorPosY);
                const int monitorWidth = mode->width;
                const int monitorHeight = mode->height;

                // Set screen position and size
                glfwSetWindowPos(platform.handle, monitorPosX, monitorPosY);
                glfwSetWindowSize(platform.handle, monitorWidth, monitorHeight);

                // Refocus window
                glfwFocusWindow(platform.handle);

                CORE.Window.flags |= FLAG_BORDERLESS_WINDOWED_MODE;
            }
            else
            {
                // Remove undecorated flag
                //glfwSetWindowAttrib(platform.handle, GLFW_DECORATED, GLFW_TRUE);
                int prevPosX = CORE.Window.previousPosition.x ;
                int prevPosY = CORE.Window.previousPosition.y ;
                int prevWidth = CORE.Window.previousScreen.width ;
                int prevHeight = CORE.Window.previousScreen.height ;
                glfwSetWindowMonitor(platform.handle, NULL, prevPosX , prevPosY, prevWidth, prevHeight, GLFW_DONT_CARE);
                CORE.Window.flags &= ~FLAG_WINDOW_UNDECORATED;

                // Return previous screen size and position
                // NOTE: The order matters here, it must set size first, then set position, otherwise the screen will be positioned incorrectly
                glfwSetWindowSize(platform.handle,  CORE.Window.previousScreen.width, CORE.Window.previousScreen.height);
                glfwSetWindowPos(platform.handle, CORE.Window.previousPosition.x, CORE.Window.previousPosition.y);

                // Refocus window
                glfwFocusWindow(platform.handle);

                CORE.Window.flags &= ~FLAG_BORDERLESS_WINDOWED_MODE;

                CORE.Window.position.x = CORE.Window.previousPosition.x;
                CORE.Window.position.y = CORE.Window.previousPosition.y;
            }
        }
        else TRACELOG(LOG_WARNING, "GLFW: Failed to find video mode for selected monitor");
    }
    else TRACELOG(LOG_WARNING, "GLFW: Failed to find selected monitor");
}

[and3md]

I tested the behavior of ToggleBorderlessWindowed() function also on a computer with integrated Intel graphics (i7 1255U, Mesa 24.2.8-1ubuntu1~24.04.1, Linux Mint 22.1) and in this case also the ToggleBorderlessWindowed() function from master does not work. Everything works when I make the correction I mentioned in above issue details.