Open
Description
Is there any plan to support varlen interface? It seems useful for batch generation and some kv cache prefetch scheme would enhance the performance.
Metadata
Metadata
Assignees
Labels
No labels
{{ message }}
No one assigned
No labels
No projects
No milestone
None yet
No branches or pull requests