Fix DWM frame glitches during fullscreen

[MarceloAlejandroJorquera]

Summary

Fixes fullscreen transition composition artifacts by temporarily cloaking the
main window while MPC-BE performs its existing fullscreen style/layout
transition, then waiting for the compositor to reach the completed presentation
state before revealing it again.

Problem

On the affected Windows 11 system, switching between windowed and fullscreen
mode can expose intermediate window states.

Depending on the playback state, this can appear as:

• corrupted/non-final window frame contents;
• previous windowed contents appearing temporarily in the upper-left portion
  of the fullscreen window.

The issue is reproducible not only during video playback, but also with
audio-only media and with no media loaded.

Investigation

Several smaller approaches were tested while isolating the problem.

Simply performing the style and geometry changes atomically was insufficient.

Temporarily hiding the main HWND prevented the original frame corruption, but
introduced another issue. Instrumentation showed that during fullscreen entry
the main window could already be at its final fullscreen dimensions while the
child view still retained its previous windowed dimensions until the subsequent
layout/WM_SIZE processing completed.

This produced the transient upper-left content.

Using SW_HIDE is therefore unsuitable because it changes the logical
visibility of the window and interferes with the normal child/layout path.

Fix

On Windows 8 and later:

1. Cloak the main HWND before MPC-BE's existing fullscreen transition.
2. Leave the existing fullscreen style, window geometry, and child-layout logic
   unchanged.
3. Allow the normal MoveVideoWindow() / layout processing to complete.
4. Call DwmFlush() while the HWND is still cloaked so pending compositor
   updates reach their presentation point.
5. Remove the cloak.

This keeps the window logically visible throughout the transition, allowing
normal MFC and child-window layout processing to run, while preventing DWM from
presenting the intermediate state.

Windows 7 continues through MPC-BE's original path unchanged.

Scope

The patch intentionally does not introduce:

• a temporary covering window;
• WS_POPUP;
• SW_HIDE / SW_SHOW;
• changes to MPC-BE's fullscreen style model;
• forced RedrawWindow() calls;
• DWM transition-policy changes;
• renderer-specific resize handling;
• tray restore changes.

Only the fullscreen transition composition is affected.

Testing

Tested on Windows 11 x64 in all of these states:

• video playback;
• audio-only playback;
• no media loaded;
• repeated fullscreen/windowed toggling.

The corrupted frame and upper-left intermediate-content artifacts are no longer
visible in these cases, and window activation/focus remains functional.

Before

2026-08-09.10-37-25.mp4

After

2026-08-09.02-57-30.mp4

[Aleksoid1978]

Какой-то "плохой" подход с созданием временного окна для перекрытия основного чтобы не видеть "артефакты".

[Aleksoid1978]

Тоже самое касается задания стиля WS_POPUP для полноэкранного окна - зачем ?

[Aleksoid1978]

DWM-cloak только с 8 винды поддерживается.

[Aleksoid1978]

Я проверил - подтверждаю что артефакты ушли и стало "хорошо". Но надо еще как-то протестировать начиная с Win 7 и заканчивая Win 10.

[MarceloAlejandroJorquera]

I've substantially reduced the patch based on your feedback.

The temporary cover HWND, WS_POPUP conversion, DWM transition changes, style preservation machinery, tray changes and other unrelated modifications have all been removed.

While isolating the issue, I instrumented the fullscreen transition. One failed approach using SW_HIDE showed why simply hiding the window isn't viable: the main HWND could become visible at 2560×1440 while the child view was still at its previous windowed size, producing the upper-left stale content until the following WM_SIZE.

The revised patch now uses only DWMWA_CLOAK on Windows 8+ around MPC-BE's existing fullscreen transition. The normal style/layout implementation is otherwise unchanged, and Windows 7 follows the original path.

This eliminated the original artifact in my testing without introducing the intermediate layout problem.

[MarceloAlejandroJorquera]

Updated again after testing the reduced implementation across video, audio-only,
and empty-player states.

The earlier cloak-only revision fixed the video case, but further testing showed
that audio-only and empty-player transitions could still expose an intermediate
presentation state.

While isolating this, I also tested forced synchronous repainting, but that was
incorrect: it could repaint the child while it still held its pre-WM_SIZE
geometry and reintroduced the upper-left artifact.

The final revision therefore remains minimal:

• cloak the HWND on Windows 8+;
• run MPC-BE's existing fullscreen transition unchanged;
• run the existing final MoveVideoWindow();
• call DwmFlush() before removing the cloak.

No cover window, WS_POPUP, hide/show transition, forced redraw, renderer
workaround, or tray-related changes remain.

Retested successfully with video, audio-only media, and no media loaded.

[Aleksoid1978]

Comfirm - all is ok now.

[Aleksoid1978]

It's final code ?

[MarceloAlejandroJorquera]

до

2026-08-09.10-37-25.mp4

после

2026-08-09.02-57-30.mp4

Это исправило перерисовку рамки и верхней левой четверти в полноэкранном режиме для всех трёх сценариев: пустой плеер, загруженное видео, аудио. Изначально исправление затрагивало только видео, поэтому его пришлось доработать. Теперь я займусь проблемой с белым содержимым при восстановлении окна из системного трея.