Skip to content

Commit

Permalink
typo: update decode_maybe_q_rope_offset (#856)
Browse files Browse the repository at this point in the history
Following up on #855 and #847, this PR fixes a typo of inline RoPE in
decode kernel.
  • Loading branch information
MasterJH5574 authored Feb 16, 2025
1 parent ea1d0cb commit 5168308
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion include/flashinfer/attention/decode.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@ __global__ void BatchDecodeWithPagedKVCacheKernel(const __grid_constant__ Params
if constexpr (POS_ENCODING_MODE == PosEncodingMode::kRoPELlama) {
const IdType* q_rope_offset = nullptr;
if constexpr (has_decode_maybe_q_rope_offset_v<Params>) {
q_rope_offset = params.maybe_q_rope_offset;
q_rope_offset = params.decode_maybe_q_rope_offset;
}
int32_t q_rope_offset_val = q_rope_offset == nullptr ? (kv_len - 1) : q_rope_offset[batch_idx];
const float rope_rcp_scale = params.rope_rcp_scale;
Expand Down

0 comments on commit 5168308

Please sign in to comment.