Skip to content

[XPU] [Quant] rename mxfp8_e4m3_quantize and add xpu backend implementation#38682

Open
zufangzhu wants to merge 6 commits intovllm-project:mainfrom
zufangzhu:zufang/refactor_mxfp8_quant
Open

[XPU] [Quant] rename mxfp8_e4m3_quantize and add xpu backend implementation#38682
zufangzhu wants to merge 6 commits intovllm-project:mainfrom
zufangzhu:zufang/refactor_mxfp8_quant

Conversation

@zufangzhu
Copy link
Copy Markdown
Contributor

  1. Rename mxfp8_e4m3_quantize → flashinfer_mxfp8_e4m3_quantize to make it explicit that this path is implemented via FlashInfer (and avoid ambiguity with other backends).
  2. Add xpu_mxfp8_quantize as a new custom op for XPU, providing an XPU MXFP8 quantization path.

Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Apr 1, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zufangzhu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Apr 1, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request renames existing MXFP8 quantization functions to be FlashInfer-specific and introduces a new XPU-specific MXFP8 quantization implementation, including a custom operator registration. The review feedback identifies a critical type mismatch in the XPU fake implementation that would break torch.compile, as well as a non-idiomatic null check. Furthermore, the reviewer pointed out that the model executor still hardcodes FlashInfer calls, which will lead to runtime failures on XPU platforms, necessitating a platform-aware dispatch mechanism.

@zufangzhu zufangzhu force-pushed the zufang/refactor_mxfp8_quant branch from 29218f9 to 8f38386 Compare April 1, 2026 03:12
@mergify mergify bot removed the needs-rebase label Apr 1, 2026
@mergify
Copy link
Copy Markdown

mergify bot commented Apr 1, 2026

Hi @zufangzhu, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Apr 1, 2026

Hi @zufangzhu, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Apr 1, 2026

Hi @zufangzhu, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Apr 1, 2026

Hi @zufangzhu, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

intel-gpu Related to Intel GPU

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant