fix: use correct NVIDIA CUDA base images for debian distro#5590
Merged
frostming merged 2 commits intobentoml:mainfrom Apr 2, 2026
Merged
fix: use correct NVIDIA CUDA base images for debian distro#5590frostming merged 2 commits intobentoml:mainfrom
frostming merged 2 commits intobentoml:mainfrom
Conversation
The debian distro's CUDA image template hardcoded ubuntu24.04 with the newer -cudnn- naming, but none of the previously supported CUDA versions (≤12.1.1) have images published for ubuntu24.04, and the -cudnn- naming only started with CUDA 12.6+. This made cuda_version completely non-functional for the default debian distro. - Add CUDA 12.6.x and 12.8.x to supported versions - Introduce get_cuda_base_image() to select ubuntu24.04/-cudnn- for ≥12.6.0 and ubuntu22.04/-cudnn8- for ≤12.1.1 - Update DEFAULT_CUDA_VERSION to 12.8.1 - Add unit tests for CUDA image resolution and version mapping Fixes bentoml#5589 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
For more information, see https://pre-commit.ci
frostming
approved these changes
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR address?
The
cuda_versionoption inbentofile.yamlis completely broken for the defaultdebiandistro. The hardcoded image templatenvidia/cuda:{version}-cudnn-runtime-ubuntu24.04generates tags that don't exist onDocker Hub for any of the previously supported CUDA versions (≤12.1.1), because:
-cudnn-naming (without a version number) only started with CUDA 12.6+; olderimages use
-cudnn8-Changes:
SUPPORTED_CUDA_VERSIONSandALLOWED_CUDA_VERSION_ARGS— these are the versions that actually exist forubuntu24.04
get_cuda_base_image()to select the correct base image per version:ubuntu24.04 with
-cudnn-for ≥12.6.0, ubuntu22.04 with-cudnn8-for ≤12.1.1DEFAULT_CUDA_VERSIONfrom11.6.2to12.8.1get_cuda_base_image()ingenerate.pyinstead of formatting the statictemplate, so ubi8 and other distros continue to work unchanged
Fixes #5589
Before submitting:
pre-commit run -ascript has passed (instructions)?those accordingly?