Skip to content

Fix race in realtime tool-result unit test - #42

Merged
GarrettBeatty merged 1 commit into
devfrom
gcbeatty/fix-realtime-toolresult-test-race
Aug 10, 2026
Merged

Fix race in realtime tool-result unit test#42
GarrettBeatty merged 1 commit into
devfrom
gcbeatty/fix-realtime-toolresult-test-race

Conversation

@GarrettBeatty

Copy link
Copy Markdown
Contributor

Problem

The Tests-arm64 action in the aws-dotnet-ai pipeline (Validation stage) failed with a single test failure:

Failed AWS.Bedrock.MEAI.BedrockRealtimeSessionTests.SendAsync_ConversationItemWithToolResult_SendsToolResult
  Assert.NotNull() Failure: Value is null
  at BedrockRealtimeSessionTests.cs:line 607
Total tests: 235   Passed: 234   Failed: 1

Root cause

This is a flaky test, not a product bug. BedrockNovaRealtimeSession.SendToolResultAsync writes three events in order to an unbounded channel — contentStart (TOOL) → toolResultcontentEnd — which a background single-reader task drains asynchronously into the test's capturedEvents list.

The test blocked only on the toolResult event, then immediately asserted that a contentEnd event also existed. Because contentEnd is written after toolResult and captured asynchronously, there is a window where toolResult is present but contentEnd is still in flight. On the arm64 runner that race was lost and the assertion saw null.

Fix

Wait for the trailing contentEnd event in addition to toolResult before running the assertions, closing the race. The session update in this test supplies no Instructions, so no earlier system-prompt contentEnd exists to match spuriously.

Testing

dotnet test for the affected test passes locally on net8.0 (the target that runs in the pipeline).

The test waited only for the toolResult event before asserting that the
trailing contentEnd event had been captured. Since events are drained
asynchronously by the background writer, contentEnd could still be in
flight, causing an intermittent Assert.NotNull failure (observed on the
arm64 pipeline runner). Wait for contentEnd as well to close the race.
@GarrettBeatty
GarrettBeatty requested review from a team as code owners August 7, 2026 18:06
@GarrettBeatty
GarrettBeatty requested a review from normj August 7, 2026 18:06
@GarrettBeatty GarrettBeatty added the Release Not Needed Add this label if a PR does not need to be released. label Aug 7, 2026
@GarrettBeatty
GarrettBeatty merged commit 1886523 into dev Aug 10, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Release Not Needed Add this label if a PR does not need to be released.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants