Terminal Preview Hanging On Startup

[amyb-asu]

Windows Terminal version

1.23.10353.0

Windows build number

22631.4890

Other Software

No response

Steps to reproduce

I don't know what is causing it. For me it happens right now any time I try to open the terminal in any way at all.

Expected Behavior

The terminal opens and does not peg a CPU core.

Actual Behavior

As of today, my Windows Terminal Preview build has started hanging on every startup, spinning the CPU. I also noticed similar behavior with Terminal Canary starting several months ago, so I had to switch to Preview which was working at the time... But now Preview also has the same behavior...

I am now debugging it with windbg, and it seems this thread has the highest usage:

If we let it run and randomly break now and then we see a similar stack trace every time with a call to TerminalApp!Pane::LayoutSizeNode::operator=

That leaves me to believe this is some kind of endless loop somewhere within this call trace.

I have firstWindowPreference": "persistedWindowLayout" enabled in my config. Here is redacted version of my state.json file:

{
	"dismissedMessages" : 
	[
		"deprecated",
		"closeOnExitInfo"
	],
	"generatedProfiles" : 
	[
		"{REDACTED}",
		"{REDACTED}",
		"{REDACTED}",
		"{REDACTED}"
	],
	"persistedWindowLayouts" : 
	[
		{
			"initialPosition" : "0,-14",
			"initialSize" : 
			{
				"height" : 880.4444580078125,
				"width" : 1536
			},
			"launchMode" : "maximized",
			"tabLayout" : 
			[
				{
					"action" : "newTab",
					"commandline" : "REDACTED",
					"profile" : "REDACTED",
					"sessionId" : "{REDACTED}",
					"startingDirectory" : "REDACTED",
					"suppressApplicationTitle" : false,
					"tabTitle" : "REDACTED"
				},
				{
					"action" : "splitPane",
					"commandline" : "REDACTED",
					"profile" : "REDACTED",
					"sessionId" : "{REDACTED}",
					"size" : 0.5,
					"split" : "right",
					"splitMode" : "manual",
					"startingDirectory" : "REDACTED",
					"suppressApplicationTitle" : false,
					"tabTitle" : "REDACTED"
				},
				{
					"action" : "splitPane",
					"commandline" : "REDACTED",
					"profile" : "REDACTED",
					"sessionId" : "{REDACTED}",
					"size" : 0.5,
					"split" : "down",
					"splitMode" : "manual",
					"startingDirectory" : "REDACTED",
					"suppressApplicationTitle" : false,
					"tabTitle" : "REDACTED"
				},
				{
					"action" : "focusPane",
					"id" : 0
				},
				"togglePaneZoom",
				{
					"action" : "newTab",
					"commandline" : "REDACTED",
					"profile" : "REDACTED",
					"sessionId" : "{REDACTED}",
					"startingDirectory" : "REDACTED",
					"suppressApplicationTitle" : false,
					"tabTitle" : "REDACTED"
				},
				{
					"action" : "switchToTab",
					"index" : 0
				}
			]
		}
	],
	"recentCommands" : 
	[
		"ls"
	],
	"settingsHash" : "REDACTED"
}

[amyb-asu]

Some more debugging in windbag:

This loop seems to not terminate (in pane.cpp)

while (sizeTree.size < fullSize)
    {
        lastSizeTree = sizeTree;
        _AdvanceSnappedDimension(widthOrHeight, sizeTree);

        if (sizeTree.size == fullSize)
        {
            // If we just hit exactly the requested value, then just return the
            // current state of children.
            return { { sizeTree.firstChild->size, sizeTree.secondChild->size },
                     { sizeTree.firstChild->size, sizeTree.secondChild->size } };
        }
    }

At the while conditional, sizeTree.size = 54.0 and fullSize = 460.0, however when I step through the loop, passing the _AdvanceSnappedDimension, both these values remain unchanged (in the xmm0 and xmm6 registers respectively)

0:000> bp 00007ff8`267cf15f
breakpoint 0 redefined
0:000> r xmm0
xmm0=           0            0            0           54
0:000> r xmm6
xmm6=           0            0            0          460
0:000> g
Breakpoint 0 hit
TerminalApp!Pane::_CalcSnappedChildrenSizes+0x8f:
00007ff8`267cf15f 0f2ff0          comiss  xmm6,xmm0
0:000> r xmm0
xmm0=           0            0            0           54
0:000> r xmm6
xmm6=           0            0            0          460

So this causes the loop to never terminate...

Now to figure out why _AdvanceSnappedDimension does not cause sizeTree.size to change

[amyb-asu]

Tracing through _AdvancedSnappedDimension, it appears that it ends up not changing the sizeTree at all in this degenerate case, thus it noops, and the loop continues forever.

Here is my walk through of the trace that I see once the inf loop gets going. I'm not sure what the inital state is that leads to this fixed point state...

https://github.com/microsoft/terminal/blob/v1.23.10353.0/src/cascadia/TerminalApp/Pane.cpp#L2701

void Pane::_AdvanceSnappedDimension(
    const bool widthOrHeight, // true
    LayoutSizeNode& sizeNode /*&{
        size: 54.0,
        isMinimumSize: false,

        firstChild: &{ size: 42.0, isMinimumSize: true, NULL, NULL, NULL, NULL },                // 0x000001a86ca31660
        // 0x000001a86ca313f0
        secondChild: &{ size: 12.0, isMinimumSize: true,

            firstChild: &{ size: 12.0, isMinimumSize: true, NULL, NULL, NULL, NULL },            // 0x000001a8674b6260
            secondChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL },     // 0x000001a8674b5e40
            nextFirstChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL },  // 0x000001a8674b5c00
            nextSecondChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL }, // 0x000001a8674b5d50
        },
        nextFirstChild: &{ size: 50.0, isMinimumSize: false, NULL, NULL, NULL, NULL },           // 0x000001a86ca31330
        // 0x000001a86ca316f0
        nextSecondChild: &{ size: 12.0, isMinimumSize: false, 
            firstChild: &{ size: 12.0, isMinimumSize: true, NULL, NULL, NULL, NULL },            // 0x000001a86ca31540
            secondChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL },     // 0x000001a86ca31570
            nextFirstChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL },  // 0x000001a86ca31390
            nextSecondChild: &{ size: -1.#IND00, isMinimumSize: false, NULL, NULL, NULL, NULL }, // 0x000001a86ca31300
        },
    }*/
) const {
    if (_IsLeaf()) { ... } // TAKES FALSE BRANCH
    else {
        if (sizeNode.nextFirstChild == nullptr) { ... } // TAKES FALSE BRANCH
        if (sizeNode.nextSecondChild == nullptr) { ... } // TAKES FALSE BRANCH

        const auto nextFirstSize = sizeNode.nextFirstChild->size; // 50.0
        const auto nextSecondSize = sizeNode.nextSecondChild->size; // 12.0

        bool advanceFirstOrSecond;
        if (_splitState == (widthOrHeight ? SplitState::Horizontal : SplitState::Vertical)) { ... } // FALSE BRANCH
        else {

            const auto firstSize = sizeNode.firstChild->size; // 42.0
            const auto secondSize = sizeNode.secondChild->size; // 12.0

            // 50.0 - (50.0 + 12.0) * 0.5 = 50.0 - 31.0 = 19.0
            const auto deviation1 = nextFirstSize - (nextFirstSize + secondSize) * _desiredSplitPosition;
            // -1 * (42.0 - (42.0 + 12.0) * 0.5) = -15.0
            const auto deviation2 = -1 * (firstSize - (firstSize + nextSecondSize) * _desiredSplitPosition);
            advanceFirstOrSecond = deviation1 <= deviation2; // false
        }

        if (advanceFirstOrSecond) { ... } // TAKES FALSE BRANCH
        else {
            *sizeNode.secondChild = *sizeNode.nextSecondChild; // No change here as both are already identical
            _secondChild->_AdvanceSnappedDimension(widthOrHeight, *sizeNode.nextSecondChild); // nextSecondChild remains unchanged after this recursive call
        }

        if (_splitState == (widthOrHeight ? SplitState::Horizontal : SplitState::Vertical)) { ... } // FALSE BRANCH
        else {
            // 42.0 + 12.0 = 54.0
            sizeNode.size = sizeNode.firstChild->size + sizeNode.secondChild->size; 
        }
    }

    // AS A RESULT OF THIS FUNCTION, sizeNode.size remains unchanged, thus this function does nothing...

    sizeNode.isMinimumSize = false;
}

[lhecker]

There's a good chance that this was fixed by #18627, because it fixes an issue where I create panes faster than WinUI assigns them a layout size. This seems eerily similar to what you found where the sizing of panes results in an infinite loop.
Even if this issue was fixed by #18627 though, I think that this is another issue worth tracking seperately, because no part of Windows Terminal should just enter an infinite loop, no matter the situation.