Skip to content

silu_and_mul: qwen2.5-vl VIT intermediate_size=3420, but vec_size in flashinfer is allways 16 #2526

@CrimsonDump

Description

@CrimsonDump

reproduce by replace dim with 3420

@pytest.mark.parametrize("dim", [128, 256, 512, 2048, 4096, 11008, 16384])

Also you can refer to:
https://github.com/alibaba/rtp-llm/blob/main/3rdparty/flashinfer/0010-silu-mul-vec-size.patch

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions