Skip to content

chore(huggingface-vllm): re-enable PR workflow; add transformers_version to release spec; rename framework #6287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sallyseok merged 6 commits into from
Jun 25, 2026
Merged

chore(huggingface-vllm): re-enable PR workflow; add transformers_version to release spec; rename framework #6287

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
41e7791
Add transformers_version to release spec; rename huggingface-vllm
sallyseok Jun 25, 2026
7abe3bf
re-enable PR workflow
sallyseok Jun 25, 2026
b958a1e
fix base image URI
sallyseok Jun 25, 2026
079c64f
fix
sallyseok Jun 25, 2026
9d99885
address cves
sallyseok Jun 25, 2026
84faf58
force build sglang
sallyseok Jun 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/actions/generate-release-spec/generate_release_spec.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ else
CUDA_VERSION=""
fi

TRANSFORMERS_VERSION=$(yq '.build.transformers_version // ""' "$CONFIG_FILE")

# Release flags
FORCE_RELEASE=$(yq '.release.force_release // ""' "$CONFIG_FILE")
PUBLIC_REGISTRY=$(yq '.release.public_registry // ""' "$CONFIG_FILE")
Expand All @@ -80,6 +82,7 @@ SPEC+="version: \"${VERSION}\""$'\n'
[[ -n "$OS_VERSION" ]] && SPEC+="os_version: \"${OS_VERSION}\""$'\n'
[[ -n "$CUSTOMER_TYPE" ]] && SPEC+="customer_type: \"${CUSTOMER_TYPE}\""$'\n'
[[ -n "$CUDA_VERSION" ]] && SPEC+="cuda_version: \"${CUDA_VERSION}\""$'\n'
[[ -n "$TRANSFORMERS_VERSION" ]] && SPEC+="transformers_version: \"${TRANSFORMERS_VERSION}\""$'\n'
[[ -n "$PLATFORM" ]] && SPEC+="platform: \"${PLATFORM}\""$'\n'
[[ -n "$FORCE_RELEASE" ]] && SPEC+="force_release: ${FORCE_RELEASE}"$'\n'
[[ -n "$PUBLIC_REGISTRY" ]] && SPEC+="public_registry: ${PUBLIC_REGISTRY}"$'\n'
Expand Down
6 changes: 3 additions & 3 deletions .github/config/image/huggingface-vllm/sagemaker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ image:
description: "Hugging Face vLLM for SageMaker (layered on the AWS vLLM DLC)"

metadata:
framework: "huggingface_vllm"
framework: "huggingface-vllm"

@sirutBuasai sirutBuasai Jun 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the convention is to use _ instead of - for frameworks now. Any reason why we want - for huggingface?

@sallyseok sallyseok Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed the framework name from huggingface_vllm with underscore to huggingface-vllm with hyphen in order to match the ECR repository name and eliminate unnecessary discrepancies. The other framework name variables that use underscores (like vllm_server) do not have corresponding ECR repository names associated with it (repository is just vllm).

framework_version: "0.22.1"
os_version: "ubuntu22.04"
customer_type: "sagemaker"
Expand All @@ -17,7 +17,7 @@ metadata:
build:
dockerfile: "docker/huggingface/vllm/Dockerfile"
target: "huggingface-vllm-sagemaker"
base_image: "public.ecr.aws/deep-learning-containers/vllm:0.22.1-gpu-py312-cu130-ubuntu22.04-sagemaker-v1"
base_image: "public.ecr.aws/deep-learning-containers/vllm:0.22.1-gpu-py312-cu130-ubuntu22.04-sagemaker-v1.0"
python_version: "3.12"
cuda_version: "13.0.2"
transformers_version: "5.10.2"
Expand All @@ -33,4 +33,4 @@ release:
public_registry: false
private_registry: true
enable_soci: true
environment: "production"
environment: "gamma"
34 changes: 16 additions & 18 deletions .github/workflows/huggingface-vllm.pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,21 @@
name: "PR - Hugging Face vLLM"

on:
# TEMP: Enable after all frameworks are refactored
workflow_dispatch:
# pull_request:
# branches: [main]
# types: [opened, reopened, synchronize]
# paths:
# - ".github/config/image/huggingface-vllm/*.yml"
# - ".github/workflows/huggingface-vllm.pipeline.yml"
# - ".github/workflows/huggingface-vllm.pr.yml"
# - "docker/huggingface/vllm/**"
# - "scripts/docker/huggingface/vllm/**"
# - "scripts/docker/telemetry/**"
# - "test/sanity/**"
# - "test/security/data/ecr_scan_allowlist/huggingface_vllm/**"
# - "test/telemetry/**"
# - "test/vllm/sagemaker/**"
# - "!docs/**"
pull_request:
branches: [main]
types: [opened, reopened, synchronize]
paths:
- ".github/config/image/huggingface-vllm/*.yml"
- ".github/workflows/huggingface-vllm.pipeline.yml"
- ".github/workflows/huggingface-vllm.pr.yml"
- "docker/huggingface/vllm/**"
- "scripts/docker/huggingface/vllm/**"
- "scripts/docker/telemetry/**"
- "test/sanity/**"
- "test/security/data/ecr_scan_allowlist/huggingface-vllm/**"
- "test/telemetry/**"
- "test/vllm/sagemaker/**"
- "!docs/**"

permissions:
contents: read
Expand Down Expand Up @@ -51,7 +49,7 @@ jobs:
- ".github/config/image/huggingface-vllm/*.yml"
- "docker/huggingface/vllm/**"
- "scripts/docker/huggingface/vllm/**"
- "test/security/data/ecr_scan_allowlist/huggingface_vllm/**"
- "test/security/data/ecr_scan_allowlist/huggingface-vllm/**"
sanity-test-change:
- "test/sanity/**"
sagemaker-test-change:
Expand Down
4 changes: 2 additions & 2 deletions docker/huggingface/vllm/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
ARG BASE_IMAGE=public.ecr.aws/deep-learning-containers/vllm:0.22.1-gpu-py312-cu130-ubuntu22.04-sagemaker-v1
ARG BASE_IMAGE=public.ecr.aws/deep-learning-containers/vllm:0.22.1-gpu-py312-cu130-ubuntu22.04-sagemaker-v1.0

FROM ${BASE_IMAGE} AS base

ARG FRAMEWORK=huggingface_vllm
ARG FRAMEWORK=huggingface-vllm
ARG FRAMEWORK_VERSION=0.22.1
ARG CONTAINER_TYPE=general
ARG DLC_MAJOR_VERSION=1
Expand Down
2 changes: 1 addition & 1 deletion docker/sglang/Dockerfile.amzn2023
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ RUN cd sglang \
&& FLASHINFER_CUBIN_DOWNLOAD_THREADS=${BUILD_PARALLEL} FLASHINFER_LOGGING_LEVEL=warning \
python3 -m flashinfer --download-cubin

# Persist nvidia pip lib paths for subsequent build steps that import torch
# Persist nvidia pip lib paths for subsequent build steps that import torch.
RUN echo $(python3 -c "import glob; print(':'.join(glob.glob('/usr/local/lib*/python${PYTHON_VERSION}/site-packages/nvidia/*/lib')))") > /tmp/nvidia_lib_paths

# CUDA 13 moved CCCL headers from include/cuda/std/ to include/cccl/cuda/std/.
Expand Down
2 changes: 1 addition & 1 deletion scripts/docker/telemetry/deep_learning_container.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ def parse_args():
"lambda",
"ray",
"vllm_omni",
"huggingface_vllm",
"huggingface-vllm",
],
help="framework of container image.",
required=True,
Expand Down
10 changes: 10 additions & 0 deletions ...huggingface_vllm/framework_allowlist.json → ...huggingface-vllm/framework_allowlist.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,5 +78,15 @@
"vulnerability_id": "CVE-2026-42504",
"reason": "go/stdlib 1.24.11 statically linked inside mooncake libetcd_wrapper.so, cannot be patched independently of mooncake-transfer-engine rebuild",
"review_by": "2026-08-30"
},
{
"vulnerability_id": "RUSTSEC-2026-0185",
"reason": "quinn-proto 0.11.14 statically linked inside the uv binary (/usr/local/bin/uv); no uv release through 0.11.24 has bumped quinn-proto to the fixed 0.11.15, so it cannot be patched independently. DoS only and uv is a build-time installer, not network-facing at runtime",
"review_by": "2026-09-25"
},
{
"vulnerability_id": "CVE-2026-53923",
"reason": "vllm 0.22.1 GGUF dequantize kernel integer truncation; fixed only in 0.23.1rc0 (release candidate), no stable fixed version available to pin to; vllm version cannot be changed",
"review_by": "2026-09-25"
}
]
Loading