Fix/amd wheel jit kernel support#22661
Open
akao-amd wants to merge 3 commits intosgl-project:mainfrom
Open
Conversation
This commit enables JIT (Just-In-Time) kernel compilation for users installing the amd-sglang wheel, addressing issue sgl-project#19917. Changes: 1. Add apache-tvm-ffi dependency to AMD wheel runtime_common - JIT kernel compilation requires apache-tvm-ffi for FFI integration - This dependency was present in the main pyproject.toml but missing from the AMD wheel's runtime_common 2. Document JIT kernel support in AMD wheel README - Add section explaining JIT kernel requirements - Include verification steps for users - Note system requirements (compiler toolchain) The JIT kernel source files were already being packaged via the `jit_kernel/**/*` pattern in package-data. The _resolve_kernel_path() function in jit_kernel/utils.py already handles both editable and wheel installs correctly since source files are included in package data. The main issue was the missing apache-tvm-ffi dependency. Fixes sgl-project#19917 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synchronize 3rdparty/amd/wheel/sglang/pyproject.toml with recent changes from python/pyproject_other.toml to ensure AMD wheel users have access to the same features and bug fixes as other platforms. Key updates: 1. Dependencies updated: - transformers: 4.57.1 → 5.3.0 - Added mistral_common>=1.9.0 (required for transformers 5.x) - sglang-kernel: 0.4.0 → 0.4.1 (both rocm700 and rocm720) - diffusers: upgraded to 0.37.0 - cache-dit: 1.1.8 → 1.3.0 - runai_model_streamer: 0.15.5 → 0.15.7 - peft: unversioned → >=0.18.0 - Added av (video processing) 2. Removed AMD-specific extras that were in main: - hf_transfer, huggingface_hub, decord2 (not in main's runtime_common) 3. Refactored diffusion dependencies: - Extracted diffusion_common to reduce duplication - Both diffusion_hip and diffusion_musa now reference diffusion_common - Follows the same pattern as python/pyproject_other.toml 4. Test dependencies updated: - Added: polars, pytest-cov, diff-cover, bitsandbytes, parameterized - Updated peft to >=0.18.0 - Removed gguf (not in main's test deps) 5. Added killall_sglang script entry point 6. Added tracing to all_hip extras These changes ensure AMD wheel installations have feature parity with other platforms and benefit from recent dependency updates and bug fixes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These platform-specific sections are not relevant for AMD wheels.
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fix #19917 .
Modifications
apache-tvm-ffi.Accuracy Tests
Speed Tests and Profiling
N/A.
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ci