[FEATURE] Stateful inference support #274
Description
In QA tasks, the model is better to be stateful. Triton example: triton-inference-server/server#1172
{{ message }}
In QA tasks, the model is better to be stateful. Triton example: triton-inference-server/server#1172