[GDN] Write spec conv-state from registers instead of epilogue roll - #551
Open
CySpiegel wants to merge 1 commit into
Open
[GDN] Write spec conv-state from registers instead of epilogue roll#551CySpiegel wants to merge 1 commit into
CySpiegel wants to merge 1 commit into
Conversation
The vllm-project#544 epilogue rebuilt the state line in place: an aliased same-pointer shift, a runtime-bounded loop with a per-iteration branch, and K redundant global re-reads of mixed_qkvz. The rolled state's history rows equal the priming window's register slots [1, Width-1), and each draft input passes through a register at load time — write both from registers and drop the epilogue. State content is bit-identical by construction. tests/gdn_attn: 1303/0/128 (unchanged, incl. vllm-project#544 boundary tests and the bitwise mixed-batch differential). benchmark_causal_conv1d spec workloads: -1..-4% real shapes, to -11% synthetic; end-to-end neutral (~64-72us/call kernel). Assisted-by: Claude Code (Claude Fable 5) Signed-off-by: CySpiegel <56271906+CySpiegel@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016G7BGjSYFhwRoAFxY7N76a
CySpiegel
force-pushed
the
gdn-conv-spec-register-writeback
branch
from
August 27, 2026 03:42
1069ebe to
4461418
Compare
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
#544 introduced the sliding-window conv-state convention for the spec-decode causal_conv1d kernel. Its epilogue rebuilds the state line in place after the main loop: an aliased same-pointer shift (which blocks vectorization), a runtime-bounded loop with a per-iteration branch, and K redundant global re-reads of
mixed_qkvz.This PR removes the epilogue entirely. The rolled state's history rows equal the priming window's register slots [1, Width-1), so they are written back immediately after priming (compile-time unrolled, write-only, no aliasing); each draft input is stored to its rolled-state row from the register at load time. State content is bit-identical by construction: history rows are the same values read at priming, and draft rows use the identical
mixed_qkvzindexing expression the epilogue read. Edge cases preserved: Width==1 skip, pad-slot skip, num_spec_tokens==1, num_accepted clamp.Test Plan
pytest tests/gdn_attnon BMG hardware (2x Arc B70, oneAPI 2026.0), full scope.benchmark/benchmark_causal_conv1d.pyspec workloads, before/after.Test Result
Related work
Open PR #476 also optimizes this file (load/store vectorization, batch-id binary search) but targets the pre-#544 version of this function and does not address the epilogue roll. The changes are complementary; happy to coordinate rebase order.
AI-assisted change (Claude Code); reviewed, tested, and submitted by a human.
🤖 Generated with Claude Code
https://claude.ai/code/session_016G7BGjSYFhwRoAFxY7N76a