Skip to content

[Model][ROCm] Enable DeepSeek V4 Vision - #55107

Open
AndreasKaratzas wants to merge 3 commits into
akaratza_noncompiled_cudagraph_fallbackfrom
akaratza_deepseek_v4_vision_rocm
Open

[Model][ROCm] Enable DeepSeek V4 Vision#55107
AndreasKaratzas wants to merge 3 commits into
akaratza_noncompiled_cudagraph_fallbackfrom
akaratza_deepseek_v4_vision_rocm

Conversation

@AndreasKaratzas

@AndreasKaratzas AndreasKaratzas commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Member
  • Enable DeepseekV4ForConditionalGeneration on ROCm by moving the platform-neutral wrapper to common/, retaining the NVIDIA compatibility shim and unsupported XPU stub, and enabling ROCm registry, dummy-init, and tensor-schema paths.
  • Build checkpoint mappings from the active text backend, preserving NVIDIA self-finalization while deferring and idempotently applying ROCm finalization after generic per-layer quantization.
  • Route image sentinels through bias_vl in hash and regular ROCm MoE layers, expose local logits, and extend sparse-prefill visibility across complete image spans without changing text-only row width.
  • This extends PR #54566, merged as 1356635d837c4ef002ec98c1a0296e7ff60be3c1, which introduced the checkpoint with a NVIDIA-only wrapper and non-CUDA guards.

@AndreasKaratzas
[Model][ROCm] Enable DeepSeek V4 Vision
9a5cec8 
Enable the shared DeepSeek V4 vision wrapper on ROCm, preserve platform-specific text behavior, and add focused multimodal and attention coverage.

Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
@mergify mergify Bot added deepseek Related to DeepSeek models multi-modality Related to multi-modality (#4194) DSv4 rocm Related to AMD ROCm labels Sep 3, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Sep 3, 2026
@AndreasKaratzas
AndreasKaratzas marked this pull request as ready for review September 3, 2026 23:42
claude[bot]
claude Bot reviewed Sep 3, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: a91f78c5-f20d-4636-bb0b-ce18cd058c81

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

okorzh-amd
okorzh-amd reviewed Sep 4, 2026
View reviewed changes
Comment thread vllm/models/deepseek_v4/amd/rocm.py Outdated
okorzh-amd
okorzh-amd reviewed Sep 4, 2026
View reviewed changes
Comment thread vllm/models/deepseek_v4/amd/rocm.py Outdated
okorzh-amd
okorzh-amd reviewed Sep 4, 2026
View reviewed changes
Comment thread tests/models/test_deepseek_v4_vl_rocm.py Outdated
okorzh-amd
okorzh-amd reviewed Sep 4, 2026
View reviewed changes
Comment thread tests/models/test_initialization.py Outdated
okorzh-amd
okorzh-amd reviewed Sep 4, 2026
View reviewed changes
Comment thread vllm/models/deepseek_v4/amd/model.py
Isotr0py
Isotr0py reviewed Sep 4, 2026
View reviewed changes

@Isotr0py Isotr0py left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look reasonable. But would like to let @tjtanaa to check the rocm implementation.

Comment thread tests/models/multimodal/processing/test_tensor_schema.py Outdated
Comment thread tests/models/test_registry.py Outdated
@AndreasKaratzas
Address DeepSeek V4 ROCm review feedback
dba58f4 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@mergify mergify Bot added the ci/build label Sep 4, 2026
@AndreasKaratzas

AndreasKaratzas commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #87212 for commit dba58f44dc04.

@AndreasKaratzas @codex
Merge updated base into DeepSeek V4 ROCm
141ab24 
Bring in the latest stacked-base changes, including the AMD LoRA allocator configuration fix.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas

AndreasKaratzas commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #87318 for commit 141ab249ecf1.

@AndreasKaratzas AndreasKaratzas mentioned this pull request Sep 4, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py left review comments
@claude claude[bot] claude[bot] left review comments
@tjtanaa tjtanaa Awaiting requested review from tjtanaa tjtanaa is a code owner
@dllehr-amd dllehr-amd Awaiting requested review from dllehr-amd dllehr-amd is a code owner
@hongxiayang hongxiayang Awaiting requested review from hongxiayang hongxiayang is a code owner
@zyongye zyongye Awaiting requested review from zyongye zyongye is a code owner
@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner
@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner
@shen-shanshan shen-shanshan Awaiting requested review from shen-shanshan shen-shanshan is a code owner
@Harry-Chen Harry-Chen Awaiting requested review from Harry-Chen Harry-Chen is a code owner
@khluu khluu Awaiting requested review from khluu khluu is a code owner
+1 more reviewer
@okorzh-amd okorzh-amd okorzh-amd left review comments
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

ci/build deepseek Related to DeepSeek models DSv4 multi-modality Related to multi-modality (#4194) rocm Related to AMD ROCm

Projects

AMD
Status: Todo

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AndreasKaratzas @Isotr0py @okorzh-amd