Benchmark on A100, comparing fa3 and flashinfer

[Adoni]

Hi,

I'm using the following command to benchmarking the speed of --attention-backend fa3 and --attention-backend flashinfer on A100 single card:

python3 -m sglang.launch_server --model /data/Qwen/Qwen3-8B --host 0.0.0.0 --dtype bfloat16 --attention-backend fa3
python3 -m sglang.launch_server --model /data/Qwen/Qwen3-8B --host 0.0.0.0 --dtype bfloat16 --attention-backend flashinfer

then run:

python3 -m sglang.bench_serving --backend sglang --dataset-name random --num-prompts 500 --random-input 4096 --random-output 2048

The result is:

	fa3	flashinfer
Input token throughput (tok/s)	1815.95	2155.02
Output token throughput (tok/s)	896.82	1064.28

My questions are:

• Flashinfer is 18% faster than fa3, is it correct?
• Is sglang depends on flash-attention package? Or sglang implements fa3 itself?
• Is the fa3 compatible with the A100 GPU?

[b8zhong]

1. It depends on the device.
2. sgl uses a fork of the tri dao FA repository, see https://github.com/sgl-project/sgl-flash-attn/tree/sgl-kernel
3. FA3 is meant to be used with Hopper+