Skip to content

[ROCm] Bump AITER to v0.1.16.post4#48683

Open
Fangzhou-Ai wants to merge 1 commit into
vllm-project:mainfrom
Fangzhou-Ai:afz/bump-aiter-v0.1.16.post4
Open

[ROCm] Bump AITER to v0.1.16.post4#48683
Fangzhou-Ai wants to merge 1 commit into
vllm-project:mainfrom
Fangzhou-Ai:afz/bump-aiter-v0.1.16.post4

Conversation

@Fangzhou-Ai

@Fangzhou-Ai Fangzhou-Ai commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Purpose

Advance the ROCm image build from AITER v0.1.16.post3 to
v0.1.16.post4. This brings the gfx950 MXFP4 MoE path, top-k and
MLA metadata/reduction optimizations, and AITER's FlyDSL 0.2.2 dependency.
FlyDSL does not need a separate vLLM pin because the AITER build stage installs
AITER's requirements.txt before building its wheel.

The tested AITER source commit was
72eedddd921d0fb8f274e7b9291dd04306d089ef.

Duplicate-work check

The required open-PR searches were run for AITER 0.1.16.post4,
AITER FlyDSL, bump aiter, and ROCm AITER; no PR already performs this
dependency uplift. #48581 touches the same Dockerfile for a materially
different gfx942 MQA integration and commit-pinning change; it does not bump
the image to this post4 release.

AI assistance was used to prepare this draft, run the duplicate checks, and
organize the existing benchmark evidence. The human submitter is responsible
for reviewing the one-line diff and validating the evidence before marking the
PR ready.

Test Plan

  • Verify the candidate ref and exact commit on ROCm/aiter.
  • Run pre-commit on docker/Dockerfile.rocm_base.
  • Validate the source-built candidate with Kimi-K2.5-MXFP4 on 4x MI355X:
    TP4/EP1, FP4 weights, FP8 KV cache, ISL/OSL 1024, concurrency 4 and 128.
  • Run GSM8K 5-shot through the OpenAI-compatible serving endpoint.

Test Result

  • pre-commit run --files docker/Dockerfile.rocm_base: passed.
  • Candidate provenance: amd-aiter==0.1.16.post4.dev11+g72eedddd9,
    AITER source 72eedddd921d0fb8f274e7b9291dd04306d089ef,
    flydsl==0.2.2.
  • Kimi-K2.5-MXFP4 serving results:
Concurrency Requests Total TPUT TPUT/GPU Median TTFT Median TPOT
4 40 942.39 tok/s 235.60 tok/s 111.61 ms 8.144 ms
128 1280 6950.36 tok/s 1737.59 tok/s 211.28 ms 36.545 ms
  • GSM8K 5-shot: 1319 samples, strict-match 0.9704, flexible-extract
    0.9704.
  • The exact candidate was built from source and exercised by these serving
    runs. A full Docker image build was not repeated locally; CI will validate
    the Docker build path.

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR is explained.
  • The test plan includes the commands and workloads used.
  • Test and model-evaluation results are included.
  • No documentation update is needed for a dependency-only uplift.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing

@mergify mergify Bot added ci/build rocm Related to AMD ROCm labels Jul 15, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Jul 15, 2026
@Fangzhou-Ai

Copy link
Copy Markdown
Contributor Author

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
@Fangzhou-Ai Fangzhou-Ai force-pushed the afz/bump-aiter-v0.1.16.post4 branch from 2c3ceea to 1cc95a2 Compare July 15, 2026 02:49
@Fangzhou-Ai Fangzhou-Ai marked this pull request as ready for review July 15, 2026 02:49

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build rocm Related to AMD ROCm

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant