Potential race condition in `CommonRuntimeKit.initialize()`

[jbelkins]

Describe the bug

A customer recently reported this crash on AWS SDK for Swift, which occurred when they ran tests on Ubuntu / Swift 6.2 which utilize SDK components:
awslabs/aws-sdk-swift#1984

When investigating with the customer, it was discovered that the cause of the crash reported by the customer was this error originating in aws-crt-swift:

AwsCommonRuntimeKit.CRTError(code: 1173, message: "Default TLS trust store not found on this system. 
Trusted CA certificates must be installed, or \"override default trust store\" must be used while creating the TLS 
context.", name: "AWS_IO_TLS_ERROR_DEFAULT_TRUST_STORE_NOT_FOUND")

This error is thrown when the SDK attempts to create a TLS context here:
https://github.com/smithy-lang/smithy-swift/blob/main/Sources/ClientRuntime/Networking/Http/CRT/SDKDefaultIO.swift#L74

We were able to eliminate the crash by calling CommonRuntimeKit.initialize() early in the test setup for the test that was crashing.

However, we already call CommonRuntimeKit.initialize() immediately above where the crash happens:
https://github.com/smithy-lang/smithy-swift/blob/main/Sources/ClientRuntime/Networking/Http/CRT/SDKDefaultIO.swift#L27

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

I would expect that once CommonRuntimeKit.initialize() has been called and has returned to the caller, that CRT and its components would be completely set up & ready for immediate use without any further delay or asynchronous completion.

Current Behavior

The fact that calling CommonRuntimeKit.initialize() long before this crash eliminates it, but calling the same method immediately before it allows the crash, suggests that there is a race condition or some other problem with CRT initialization that results in returning to the caller before CRT setup is actually completed.

Reproduction Steps

See awslabs/aws-sdk-swift#1984 for reproduction of the customer's original issue

Possible Solution

Not known

Additional Information/Context

No response

aws-crt-swift version used

0.54.0

Compiler and Version used

swift:6.2-jammy official Swift docker container

Operating System and version

Ubuntu Jammy

[azkrishpy]

@jbelkins closing this issue as per our discussion since it was not reproduced. Feel free to open it if the issue persists!