Fix ntdll!RtlFreeHeap freeze: correct WM_INITDIALOG lParam extraction for property sheet pages - #93
Merged
katahiromz merged 2 commits intoJul 2, 2026
Conversation
…Param for property sheet pages When a dialog proc is used as pfnDlgProc for a property sheet page, the lParam in WM_INITDIALOG is a PROPSHEETPAGEW* (not the dialog object pointer). The old code cast lParam directly to the dialog class pointer (XG_SettingsDialog*, XG_RulePresetDialog*, XG_HiddenDialog*), causing member-variable writes to corrupt the comctl32-allocated PROPSHEETPAGEW heap block. This resulted in ntdll!RtlFreeHeap freezing in MSYS2. Fix: add three thin wrapper dialog procs (XgSettingsPageProc, XgRulePresetPageProc, XgHiddenPageProc) that intercept WM_INITDIALOG, replace lParam with PROPSHEETPAGEW::lParam (the actual dialog object pointer), and forward to the original DialogProc. Use these wrappers as pfnDlgProc in XgGeneralSettings instead of the class DialogProc directly. The existing DialogProc/DoModal paths are unchanged.
katahiromz
marked this pull request as ready for review
July 2, 2026 01:58
Copilot
AI
changed the title
[WIP] Fix freeze in ntdll!RtlFreeHeap during DestroyPropertySheetPage
Fix ntdll!RtlFreeHeap freeze: correct WM_INITDIALOG lParam extraction for property sheet pages
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In MSYS2 builds, opening General Settings froze in
ntdll!RtlFreeHeapbecause three dialog procs used as property sheetpfnDlgProcmisinterpreted theWM_INITDIALOGlParam.Root cause
When a dialog proc is used as a property sheet page's
pfnDlgProc, Windows passes aPROPSHEETPAGEW*aslParaminWM_INITDIALOG— not the value stored inpsp.lParam. The three affected procs castlParamdirectly to their dialog class pointer:Subsequent member-variable writes (e.g.
m_lfCellFont = xg_lfCellLogFont,m_bUpdating = TRUE) then stomped thecomctl32-allocatedPROPSHEETPAGEWheap block — overwritingdwSize,dwFlags,hInstance. WhenPropertySheetWtore down the pages, the corrupted block causedRtlFreeHeapto hang.Fix
Added three thin wrapper procs in
GUI.cpp(just beforeXgGeneralSettings) that interceptWM_INITDIALOG, replacelParamwith the correct value fromPROPSHEETPAGEW::lParam, then forward to the existingDialogProc:XgGeneralSettingsnow setspsp.pfnDlgProcto these wrappers instead of the classDialogProcdirectly. TheDoModalpaths and the originalDialogProcimplementations are untouched.