[Feature Request] Reliable runtime detection of usable Execution Providers (e.g. CUDA)

[MaizeShark]

Describe the feature request

ONNX Runtime exposes onnxruntime.get_available_providers() to query available execution providers. However, this API reports compile-time–enabled providers, not providers that are actually usable at runtime.

As a result, CUDAExecutionProvider (and TensorrtExecutionProvider) may be reported as available even when required runtime dependencies (e.g. CUDA, cuDNN, or specific shared libraries like libcublasLt.so) are missing or incompatible. Attempting to use these providers then fails during session creation or silently falls back to CPU.

This behavior makes it difficult for applications, especially consumer-facing tools, to reliably select the best execution provider or present accurate diagnostics to users.

Proposed feature

Add a new API (or extend existing APIs) that allows applications to reliably detect whether an execution provider is usable at runtime, not just compiled in.

Examples of possible APIs:

onnxruntime.is_provider_usable("CUDAExecutionProvider")

or

onnxruntime.get_loadable_providers()

Such an API would:

• Attempt to load the provider’s shared library
• Validate required runtime dependencies (e.g. CUDA / cuDNN)
• Return a deterministic result without requiring model loading or exception-based control flow

Describe scenario use case

This feature would benefit applications that need to automatically select the optimal execution provider at startup, including:

• Consumer-facing ML inference applications
• GUI tools that want to display accurate GPU availability status
• Cross-platform software distributed to users with varying CUDA setups
• Applications that want to avoid silent CPU fallbacks or confusing runtime errors

Currently, developers must attempt to create an InferenceSession with CUDAExecutionProvider and catch exceptions to determine usability. This approach:

• Requires a valid model
• Relies on exceptions for normal control flow
• Produces poor UX when CUDA appears “available” but fails at runtime

A first-class, runtime-accurate provider detection API would simplify application logic, reduce user confusion, and improve robustness.