Configure release CI - #12
Open
Isotr0py wants to merge 4 commits into
Open
Conversation
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Isotr0py
marked this pull request as ready for review
May 12, 2026 17:06
3 tasks
Isotr0py
pushed a commit
that referenced
this pull request
May 13, 2026
* Migrate _C_gguf to PyTorch stable C++ ABI + Py_LIMITED_API Builds the CUDA extension against torch::stable::* + STABLE_TORCH_LIBRARY so a single abi3 wheel is binary-compatible across libtorch >= 2.9 and CPython >= 3.10 without per-version rebuilds. - torch_bindings.cpp: TORCH_LIBRARY -> STABLE_TORCH_LIBRARY + TORCH_BOX(...). PYBIND11_MODULE replaced by a minimal PyInit__C_gguf so the module loads under Py_LIMITED_API (no pybind11). - gguf/gguf_kernel.cu: wrappers take torch::stable::Tensor. at::cuda::OptionalCUDAGuard -> torch::stable::accelerator::DeviceGuard, at::cuda::getCurrentCUDAStream() -> aoti_torch_get_current_cuda_stream, torch::empty/zeros(opts) -> torch::stable::new_empty/new_zeros(self, sizes, dtype). Inner CUDA kernels (mmq.cuh, mmvq.cuh, moe.cuh, ...) are unchanged. - dispatch_utils.h: AT_DISPATCH_* (depends on ATen internals) -> THO_DISPATCH_V2 from torch/headeronly (header-only, stable-ABI safe). - setup.py: py_limited_api=True, bdist_wheel.py_limited_api="cp310", nvcc -DUSE_CUDA so the AOTI CUDA-stream shim decl is visible. - pyproject.toml: torch>=2.9 (torch::stable::Tensor minimum). Verified by building the wheel against torch 2.12.0+cu130 and loading it under torch 2.11.0+cu130: ggml_dequantize (CUDA) and ggml_moe_get_block_size (composite) both run end-to-end. Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude * Add release workflow with sdist + manylinux wheel Extends #12's sdist publish flow with a build-wheel job so PyPI gets a prebuilt binary alongside the source distribution. The stable-ABI move in the previous commit is what makes a single wheel viable across libtorch / CPython versions. - scripts/build_release_wheel.sh: derives TORCH_CUDA_ARCH_LIST from `nvcc --version`, mirroring vllm-project/vllm CMakeLists.txt CUDA_SUPPORTED_ARCHS. 10.1 is dropped from the 12.8 list because PyTorch's torch.utils.cpp_extension._get_cuda_arch_flags validator does not list it. - .github/workflows/release.yml: * build-sdist: replicates #12 verbatim. * build-wheel: nvidia/cuda:12.8.1-devel-ubuntu22.04 container (glibc 2.35 floor). Installs torch+cu128, runs the build script, retags the wheel `linux_x86_64` -> `manylinux_2_35_x86_64` via `python -m wheel tags`, twine-checks it. * publish-pypi / publish-testpypi: download both artifacts and publish them together. - MANIFEST.in: pulls vllm_gguf_plugin/csrc/**/*.{h,cpp,cu,cuh} into the sdist so source builds work from PyPI tarballs. - setup.py: gains the `_should_build_extension()` guard from #12 so sdist / egg_info / dist_info commands don't import torch. Local sanity check on CUDA 13.0: `pip install -e .` (no env var): 23s, 4.8 MB .so, local arch only. `./scripts/build_release_wheel.sh`: 101s, 6.0 MB wheel, sm_75..sm_120. `python -m build --sdist`: <2s, 77 KB tar.gz. `twine check` passes on both the sdist and the retagged wheel. Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude --------- Signed-off-by: mgoin <mgoin64@gmail.com>
Feijia1231
pushed a commit
to Feijia1231/vllm-gguf-plugin
that referenced
this pull request
Aug 18, 2026
…oject#14) * Migrate _C_gguf to PyTorch stable C++ ABI + Py_LIMITED_API Builds the CUDA extension against torch::stable::* + STABLE_TORCH_LIBRARY so a single abi3 wheel is binary-compatible across libtorch >= 2.9 and CPython >= 3.10 without per-version rebuilds. - torch_bindings.cpp: TORCH_LIBRARY -> STABLE_TORCH_LIBRARY + TORCH_BOX(...). PYBIND11_MODULE replaced by a minimal PyInit__C_gguf so the module loads under Py_LIMITED_API (no pybind11). - gguf/gguf_kernel.cu: wrappers take torch::stable::Tensor. at::cuda::OptionalCUDAGuard -> torch::stable::accelerator::DeviceGuard, at::cuda::getCurrentCUDAStream() -> aoti_torch_get_current_cuda_stream, torch::empty/zeros(opts) -> torch::stable::new_empty/new_zeros(self, sizes, dtype). Inner CUDA kernels (mmq.cuh, mmvq.cuh, moe.cuh, ...) are unchanged. - dispatch_utils.h: AT_DISPATCH_* (depends on ATen internals) -> THO_DISPATCH_V2 from torch/headeronly (header-only, stable-ABI safe). - setup.py: py_limited_api=True, bdist_wheel.py_limited_api="cp310", nvcc -DUSE_CUDA so the AOTI CUDA-stream shim decl is visible. - pyproject.toml: torch>=2.9 (torch::stable::Tensor minimum). Verified by building the wheel against torch 2.12.0+cu130 and loading it under torch 2.11.0+cu130: ggml_dequantize (CUDA) and ggml_moe_get_block_size (composite) both run end-to-end. Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude * Add release workflow with sdist + manylinux wheel Extends vllm-project#12's sdist publish flow with a build-wheel job so PyPI gets a prebuilt binary alongside the source distribution. The stable-ABI move in the previous commit is what makes a single wheel viable across libtorch / CPython versions. - scripts/build_release_wheel.sh: derives TORCH_CUDA_ARCH_LIST from `nvcc --version`, mirroring vllm-project/vllm CMakeLists.txt CUDA_SUPPORTED_ARCHS. 10.1 is dropped from the 12.8 list because PyTorch's torch.utils.cpp_extension._get_cuda_arch_flags validator does not list it. - .github/workflows/release.yml: * build-sdist: replicates vllm-project#12 verbatim. * build-wheel: nvidia/cuda:12.8.1-devel-ubuntu22.04 container (glibc 2.35 floor). Installs torch+cu128, runs the build script, retags the wheel `linux_x86_64` -> `manylinux_2_35_x86_64` via `python -m wheel tags`, twine-checks it. * publish-pypi / publish-testpypi: download both artifacts and publish them together. - MANIFEST.in: pulls vllm_gguf_plugin/csrc/**/*.{h,cpp,cu,cuh} into the sdist so source builds work from PyPI tarballs. - setup.py: gains the `_should_build_extension()` guard from vllm-project#12 so sdist / egg_info / dist_info commands don't import torch. Local sanity check on CUDA 13.0: `pip install -e .` (no env var): 23s, 4.8 MB .so, local arch only. `./scripts/build_release_wheel.sh`: 101s, 6.0 MB wheel, sm_75..sm_120. `python -m build --sdist`: <2s, 77 KB tar.gz. `twine check` passes on both the sdist and the retagged wheel. Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude --------- Signed-off-by: mgoin <mgoin64@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.