RpcClient.InvokeCoreAsync doesn't honor CancellationToken

[grokys]

Version Used: 4.12, but seems to be present in main too

Steps to Reproduce:

1. Have a solution which takes a long time to load
2. Pass a cancellation token to MSBuildWorkspace.OpenSolutionAsync
3. Try to cancel the load

The issue is that RpcClient.InvokeCoreAsync does not respect the passed cancellation token when doing the final await.

Expected Behavior:

The operation is cancelled.

Actual Behavior:

The operation is not cancelled.

Machine-generated report contains more information

# Bug Report: RpcClient.InvokeCoreAsync doesn't honor CancellationToken when waiting for response

Summary

The RpcClient.InvokeCoreAsync method in Microsoft.CodeAnalysis.MSBuild doesn't honor the provided CancellationToken when waiting for the server response, causing operations to hang indefinitely even when cancelled.

Location

• File: src/Workspaces/Core/MSBuild/Rpc/RpcClient.cs
• Method: private async Task<object?> InvokeCoreAsync(...)
• Line: ~150 (final return await statement)

Problem Description

The InvokeCoreAsync method correctly handles cancellation during the sending phase of an RPC request (lines ~139-147), but completely ignores the cancellationToken when waiting for the response (line ~150):

return await requestCompletionSource.Task.ConfigureAwait(continueOnCapturedContext: false);

This causes the following issues:

1. If the server is slow or unresponsive, the operation hangs indefinitely
2. When a caller cancels the operation (e.g., via timeout), the cancellation is ignored
3. The requestCompletionSource.Task will never complete unless the server responds or disconnects

Reproduction Steps

1. Call MSBuildWorkspace.OpenSolutionAsync() with a CancellationToken
2. Have the MSBuild server process be slow or hang during solution loading
3. Cancel the CancellationToken (e.g., via CancellationTokenSource.CancelAfter())
4. Observe that the operation does not cancel and continues waiting

Expected Behavior

When the cancellationToken is cancelled, the InvokeCoreAsync method should:

1. Cancel the pending request
2. Remove the request from _outstandingRequests
3. Throw OperationCanceledException

Actual Behavior

The operation continues waiting indefinitely for the server response, ignoring the cancellation request.

Proposed Fix

Option 1: Use Task.WaitAsync (Recommended for .NET 6+)

return await requestCompletionSource.Task.WaitAsync(cancellationToken)
    .ConfigureAwait(continueOnCapturedContext: false);

Option 2: Register cancellation callback

using (cancellationToken.Register(() =>
{
    _outstandingRequests.TryRemove(requestId, out _);
    requestCompletionSource.TrySetCanceled(cancellationToken);
}))
{
    return await requestCompletionSource.Task.ConfigureAwait(continueOnCapturedContext: false);
}

Impact

This bug affects any code that:

• Uses MSBuildWorkspace.OpenSolutionAsync() or OpenProjectAsync() with timeouts
• Needs to cancel long-running MSBuild operations
• Relies on cancellation tokens for resource cleanup or timeout enforcement

Workaround

Callers can work around this by using Task.WhenAny to enforce their own timeout:

var openTask = workspace.OpenSolutionAsync(solutionPath, cancellationToken);
var timeoutTask = Task.Delay(timeout, CancellationToken.None);

if (await Task.WhenAny(openTask, timeoutTask) == timeoutTask)
{
    workspace.Dispose();
    throw new TimeoutException("Operation timed out");
}

await openTask; // Propagate any exceptions

Additional Context

• Found while debugging integration tests that use MSBuildWorkspace with 60-second timeouts
• The cancellation token shows IsCancellationRequested = true in the debugger, but the await continues indefinitely
• This is a correctness issue with async/await patterns where cancellation tokens should be honored throughout the entire async operation

Related Code Paths

All public methods that call InvokeCoreAsync are affected:

• InvokeAsync(int targetObject, ...)
• InvokeNullableAsync<T>(...)
• InvokeAsync<T>(...)

All of these ultimately call InvokeCoreAsync which has the bug.

[jasonmalinowski]

This is somewhat by design:

roslyn/src/Workspaces/MSBuild/Core/Rpc/RpcClient.cs

Lines 165 to 166 in 9bd232f

	// The only cancellation we support is cancelling before we are able to write the request to the stream; once it's been written
	// the other side will execute it to completion. Thus cancellationToken is checked here, but nowhere else.

Even if we aborted the wait on the client side, the BuildHost will still be off doing things. The proposed fixes aren't right -- if we're going to do this properly, then we'd send the cancellation over and let the other side observe the cancellation so that way the work is actually aborted as otherwise things are then in a fairly uncertain state. That's not terribly hard to do, it's just not been something that somebody actually needed. 😄 And that said, if the concern is the solution load is too slow...maybe the time is better spent making that faster.

[grokys]

I didn't get to the bottom of what was happening - it was either an extremely slow solution load or the load was hanging. Either way, aborting after 60 seconds and retrying seems to have fixed it. I should be able to put together a repro if that would help.

[jasonmalinowski]

If you want us to diagnose I'd say try a few things:

1. You can pass a BinaryLogger to the MSBuildWorkspace and that'll produce a binlog which could give you an idea of the actual project loads themselves are slow. You'd see plenty of CPU activity in that case too.
2. If the processes seem to have gone idle then take memory dumps of both and we can take a look. For uploading the best thing to do is open a ticket on https://developercommunity.visualstudio.com/ which gives you a place to upload the files. If you do create a feedback item there link to it here so know to go looking for it (and it doesn't get mistriaged to a different team which happens sometimes.)

[jasonmalinowski]

Also how big is the solution in terms of number of projects?

[grokys]

Thanks @jasonmalinowski - I'll try those. The solution is this one: https://github.com/AvaloniaUI/Avalonia so reasonably big at around 100 projects.

The use-case is a set of integration tests in which a number of repositories are cloned and their solutions opened. To speed things up, they run concurrently, which might be where the issue comes from as I've not seen the problem if I open each solution sequentially (but it takes 5x longer if I do that).

Anyway, I think we've strayed far enough from the original bug report - quite happy to close this if it's not actually an issue (or I'm holding it wrong).

[jasonmalinowski]

I'm happy to leave the issue open, prioritization is a bit funky though. Ensuring we really handle cancellation well and correctly requires a bit of work and testing. If you just wanted us to ignore the pending request that's something you can do on your side today (just do the workaround in the body of the issue.)

My question of "is the project really slow or do we have another issue" was just a question of biggest benefit: if I can spend two hours of my time fixing this, or two hours making things faster, I should just do the latter.