Benchmark proceeds even when JSON parsing fails and dashboard does not record them as errors

[Maze999]

PFA snip for one instance which shows the JSON parsing errors for the use case of ISL 7K and OSL 1K
model being used is Llam3.1-70B with 2 replicas deployed( fp8-tp2-pp1-latency NIM profile)

1. Is there way to specify that the benchmark be terminated instead of continuing when JSON parsing errors are encountered?
2. Why are these errors not accounted and being shown under errors in the dashboard?
3. please let me know if there are any inputs to avoid getting into these JSON parsing errors

aiperf --version
0.3.0

[ajcasagrande]

@Maze999 I have investigated this a little bit. The only way this happens is if a stream from the server completes mid-payload. Likely cause is a server issue, though I do not know why no connection errors are raised.

There is no bad things that will occur from this parser error, as we just ignore it on our end, however we will include it as a valid request because it did have valid data.

Currently we do no require the stream to have [DONE] or finish_reason, though that could be a potential good thing to have the option.