Restore the TPC causal_conv1d update path, switchable by env var - #1714
Open
rsenaikh wants to merge 1 commit into
Open
Restore the TPC causal_conv1d update path, switchable by env var#1714rsenaikh wants to merge 1 commit into
rsenaikh wants to merge 1 commit into
Conversation
The TPC implementation of the hybrid-model causal_conv1d decode update was removed after a gsm8k accuracy difference was observed against the PyTorch reference on a large MoE model. A controlled repeat does not reproduce it. Running the benchmark repeatedly from a single build, with the conv1d implementation as the only variable, leaves the two implementations closer together than the benchmark's own run-to-run spread -- and that spread is larger on the PyTorch reference than on the TPC kernel, so it does not originate in the kernel. No test separates the two arms. This was measured twice in two independent configurations -- differing in PyTorch version and kernel library build -- with the same outcome. Of the questions whose outcome ever changes, about as many favour the TPC kernel as favour the reference, which is what a last-bit numerical difference looks like rather than a defect. Almost all of the difference between the two arms comes from a small set of questions that each implementation answers reproducibly but oppositely; which one is right there is arbitrary, and the split alone varies by as much as the whole measured difference. An element-wise comparison of the two implementations on identical inputs agrees to one or two bf16 ulp and writes bit-identical state. This restores the removed branch and adds VLLM_HPU_CONV1D_DISABLE_TPC so the two implementations can be compared from one build without rebuilding or swapping commits. The TPC kernel stays the default: unset, 0 or false selects it, 1 or true selects the PyTorch reference. The variable is read per call rather than cached at import so it reaches every worker process however they are spawned. Signed-off-by: Raed Al-Senaikh <raedx.senayh@intel.com>
rsenaikh
requested review from
PatrykWo,
adobrzyn,
afierka-intel,
iboiko-habana,
jbyczkow,
mgawarkiewicz-intel,
michalkuligowski and
xuechendi
as code owners
August 8, 2026 04:38
rsenaikh
had a problem deploying
to
pre-merge-approval
August 8, 2026 04:38 — with
GitHub Actions
Error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restores the TPC implementation of the hybrid-model
causal_conv1ddecode update inhpu_causal_conv1d_update, and addsVLLM_HPU_CONV1D_DISABLE_TPCso that the TPC kernel and thePyTorch reference can be selected from a single build. The TPC kernel is the default; the PyTorch
reference has to be asked for explicitly.
The path was removed in #1585 (cherry-picked to
mainin #1600) because a gsm8k accuracydifference was observed against the PyTorch reference on a large hybrid MoE model.
Why restore it
The difference does not reproduce under controlled conditions.
The benchmark was run repeatedly from one build, on one machine, against one copy of the weights,
with the conv1d implementation as the only variable, six repeats per implementation. What that
shows:
Re-running the identical build already moves more questions than the difference under
discussion.
does not originate in the kernel.
permutation test over every possible relabelling of the runs, nor McNemar on the pooled
per-question outcomes.
defect would concentrate the losses on one side; a reshuffle of borderline questions is what a
last-bit numerical difference looks like.
that each implementation answers reproducibly but oppositely. Which answer is correct there is
arbitrary — the perturbation carries no information about it — and the split of that set alone
varies by as much as the whole measured difference.
This was measured twice, in two independent configurations that differ in PyTorch version and in
kernel library build, with the same outcome both times.
Separately, an element-wise comparison of the two implementations on identical inputs — across
batch sizes, kernel widths, activations, both bias settings and repeated decode steps — agrees to
one or two bf16 ulp and writes bit-identical conv state, with no accumulation across steps.
The environment variable
0,false1,trueThe variable is read per call rather than cached at import, so it applies in every worker process
regardless of how workers are spawned, and it can be flipped between runs of one build. This is
what makes the two implementations comparable without rebuilding or swapping commits — the reason
the original report was hard to act on.
Test plan
tests/unit_tests/ops/test_causal_conv1d_update_dispatch.pyasserts which implementationeach setting of the variable selects, including the default and the case where the TPC op is
absent from the build, and that the fallback to the PyTorch reference still produces correct
results.
pytest tests/unit_tests/ops/test_causal_conv1d_update_dispatch.py—all cases pass.