Skip to content

Implementation of Dynamic Programming Approach for Minimum Threshold Prediction (Section 2.3) #19

@OrangeEarth15

Description

@OrangeEarth15

While studying the XAttention paper, I noticed that the dynamic programming approach for minimum threshold prediction described in Section 2.3 does not appear to be implemented in the current codebase. Instead, the code provides pre-tuned thresholds for LLaMA3(llama_threshold.py) and a method to compute thresholds by mapping the full attention 90% softmax to XAttention thresholds (implemented in the file threshold_utils.py).
I would like to kindly ask:
Are there any plans to implement and release the dynamic programming approach described in Section 2.3 in future versions? Are there technical challenges or practical considerations that led to using the current heuristic threshold computation method? From the ablation studies in the paper, the dynamic programming algorithm seems to have a significant impact on performance, so I am very interested in understanding the reasoning behind this choice.

Thank you very much for your time and insights!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions