add gdn2 chunk_fwd operator

[Ghd-2077]

PR Category

Type of Change

Description

Issue

Progress

• Change is properly reviewed (1 reviewer required, 2 recommended).
• Change is responded to an issue.
• Change is fully covered by a UT.

Performance

Performance Results

B	T	H	D	dtype	FLA latency (ms)	TLE optimized latency (ms)	Speedup
2	512	8	64	fp16	0.530	0.214	2.47x
4	1024	8	64	fp16	0.518	0.254	2.04x
1	2048	8	64	fp16	0.518	0.208	2.49x
1	4096	16	64	fp16	0.535	0.293	1.83x
1	8192	96	128	fp16	4.706	2.001	2.35x
2	2048	16	512	fp16	1.242	0.619	2.01x
2	16384	16	128	fp16	3.292	1.901	1.73x
4	1024	8	512	fp16	1.408	0.328	4.29x
4	2048	16	128	fp16	0.870	0.390	2.23x
4	4096	64	128	fp16	6.131	2.400	2.55x
8	1024	8	64	fp16	0.668	0.244	2.74x
8	2048	32	256	fp16	6.699	2.891	2.32x
2	512	8	64	bf16	0.558	0.306	1.82x
4	1024	8	64	bf16	0.541	0.216	2.51x
1	2048	8	64	bf16	0.545	0.223	2.45x
1	4096	16	64	bf16	0.550	0.299	1.84x
1	8192	96	128	bf16	4.724	2.009	2.35x
2	2048	16	512	bf16	1.376	0.620	2.22x
2	16384	16	128	bf16	3.291	1.888	1.74x
4	1024	8	512	bf16	1.389	0.328	4.24x
4	2048	16	128	bf16	0.873	0.390	2.24x
4	4096	64	128	bf16	6.119	2.409	2.54x
8	1024	8	64	bf16	0.530	0.213	2.49x
8	2048	32	256	bf16	6.714	2.926	2.29x

Accuracy Results

TLE branch accuracy tests passed for all 12 shapes and both dtypes:

24 passed, 24 deselected

B	T	H	D	dtype	Accuracy result
2	512	8	64	fp16	pass
2	512	8	64	bf16	pass
4	1024	8	64	fp16	pass
4	1024	8	64	bf16	pass
1	2048	8	64	fp16	pass
1	2048	8	64	bf16	pass
1	4096	16	64	fp16	pass
1	4096	16	64	bf16	pass
1	8192	96	128	fp16	pass
1	8192	96	128	bf16	pass
2	2048	16	512	fp16	pass
2	2048	16	512	bf16	pass
2	16384	16	128	fp16	pass
2	16384	16	128	bf16	pass
4	1024	8	512	fp16	pass
4	1024	8	512	bf16	pass
4	2048	16	128	fp16	pass
4	2048	16	128	bf16	pass
4	4096	64	128	fp16	pass
4	4096	64	128	bf16	pass
8	1024	8	64	fp16	pass
8	1024	8	64	bf16	pass
8	2048	32	256	fp16	pass
8	2048	32	256	bf16	pass

Commands

Query configured GDN2 benchmark shapes:

pytest -q -s benchmark/test_FLA/test_chunk_gdn2.py -m chunk_gdn2 --query

Run fp16 TLE performance test:

pytest -q -s benchmark/test_FLA/test_chunk_gdn2.py -m chunk_gdn2 --mode operator --dtypes float16 --warmup 5 --iter 10

Run bf16 TLE performance test:

pytest -q -s benchmark/test_FLA/test_chunk_gdn2.py -m chunk_gdn2 --mode operator --dtypes bfloat16 --warmup 5 --iter 10

Run TLE accuracy test for fp16 and bf16:

pytest -q -s tests/test_FLA/test_chunk_gdn2.py -m chunk_gdn2 -k tle

Run native fallback accuracy test for fp16 and bf16:

pytest -q -s tests/test_FLA/test_chunk_gdn2.py -m chunk_gdn2 -k native

[huangyiqun]

Plz fix code-style by pre-commit run --all-files.