`onnxruntime` and `onnxruntime-gpu` can not co-exist and support CUDAExecutionProvider

[fzehracetin]

Describe the issue

Hi,
I am using onnxruntime to support ONNX model inference runs in my project. My project can be installed as a library; I aim to support environments with both CPU and GPU capabilities. When I install both onnxruntime and onnxruntime-gpu packages, I can not use the CUDAExecutionProvider.

Specified provider ‘CUDAExecutionProvider’ is not in available provider names. Available providers: ‘AzureExecutionProvider, CPUExecutionProvider’

It would be great if these packages could coexist without falling back to CPUExecutionProvider.

To reproduce

Install both onnxruntime and onnxruntime-gpu packages on a GPU device. I am using a Tesla T4 GPU.

import onnxruntime

model_path = "your_onnx_model_name.onnx"
providers = ["CUDAExecutionProvider"]
self.session = onnxruntime.InferenceSession(
            model_path,
            providers=providers,
        )

This should raise the error I described above.

Urgency

Not urgent.

Platform

Linux

OS Version

Linux-6.6.105+-x86_64-with-glibc2.39

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.23.2

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 13.0

[fs-eire]

This problem is caused by violation of an assumption that at most 1 onnxruntime packages should be installed. A simple workaround is to uninstall both package and re-install the onnxruntime-gpu package.

We probably want to migrate to the "extras" in future, so that user can do this:

# CPU
pip install onnxruntime

# GPU/CUDA
pip install onnxruntime[gpu]

[istupakov]

@fs-eire Perhaps this can be fixed by adding Provides-Dist metadata as suggested in #26807?

[microsoft-github-policy-service]

Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.