optimal inference with only 16 inf2 cores and batch size 8 (>80% MFU)#2707
Open
yahavb wants to merge 2 commits intohuggingface:mainfrom
Open
optimal inference with only 16 inf2 cores and batch size 8 (>80% MFU)#2707yahavb wants to merge 2 commits intohuggingface:mainfrom
yahavb wants to merge 2 commits intohuggingface:mainfrom
