feat(genai): Sample/batch prediction (#4181) #1253

Job	Run time
region tags	12s
(legacy) Finding affected tests	19s
lint	5m 3s
(legacy) test	0s
	5m 34s

Provide feedback