Search results

Possible Bug in chain_speculative_sampling: output_emitted_token_num Exceeds Expected Limit

Status: Open.

#879 In flashinfer-ai/flashinfer;

· JaeminK opened on Feb 19, 2025

The current _append_paged_kv_cache_kernel does not support k_head_dim being different from v_head_dim

Status: Open.

#877 In flashinfer-ai/flashinfer;

· qiyuxinlin opened on Feb 19, 2025

Prebuilt kernels not found, using JIT backend

Status: Open.

#876 In flashinfer-ai/flashinfer;

· rangehow opened on Feb 19, 2025

BatchPrefillWithRaggedKVCacheWrapper custom_mask doesn't work for small batch sizes when head_dim_qk != head_dim_vo

Status: Open.

#860 In flashinfer-ai/flashinfer;

· pankajroark opened on Feb 17, 2025

[Feature] Can BatchDecodeWithPagedKVCacheWrapper return attention scores to all tokens, not just logsumexp?

Status: Open.

#838 In flashinfer-ai/flashinfer;

· yawnzh opened on Feb 14, 2025

PrefillPlan tries to allocate more memory than float_workspace_size_in_bytes passed in.

Status: Open.

#809 In flashinfer-ai/flashinfer;

· rchardx opened on Feb 12, 2025

[BUG] attention/prefill.cuh(138): error: expression must have a constant value

Status: Open.

#806 In flashinfer-ai/flashinfer;

· haohaibo opened on Feb 11, 2025

Numerical stability issue in recent commits since 0.2.0

bugSomething isn't workingSomething isn't working

priority: high

Status: Open.

#805 In flashinfer-ai/flashinfer;

· rchardx opened on Feb 11, 2025

[Docs] TypeError: single_prefill_with_kv_cache() got an unexpected keyword argument 'use_fp16_qk_reduction'

Status: Open.

#802 In flashinfer-ai/flashinfer;

· sleepwalker2017 opened on Feb 10, 2025

[Tracing Issue] Multi-head Latent Attention

Status: Open.

#792 In flashinfer-ai/flashinfer;

· yzh119 opened on Feb 6, 2025

[Feature] Reuse JIT code path for building AOT wheel

enhancementNew feature or requestNew feature or request

Status: Open.

#791 In flashinfer-ai/flashinfer;

· abcdabcd987 opened on Feb 6, 2025

Question about cascade inference

Status: Open.

#789 In flashinfer-ai/flashinfer;

· sleepwalker2017 opened on Feb 5, 2025