Responses Ollama backend injects token 0 when no streamed token is ready

[sylvesterkaczmarek]

Summary

gpt_oss.responses_api.inference.ollama.infer_next_token() returns integer 0 when Ollama has not produced another token by the end of its short polling slice but the overall inactivity timeout has not expired:

return EOS_TOKEN if False else 0

The comment describes this as a harmless PAD/NOOP placeholder. The Responses API inference interface has no no-token sentinel, and the server appends every returned integer to self.tokens and passes it to the Harmony parser immediately.

Impact

A temporarily slow Ollama stream can inject token id 0 into the generated sequence. That synthetic token was never produced by the model and can corrupt parsing/output or alter the subsequent token history supplied to the backend.

Proposed resolution

Do not manufacture a token when no model token is ready. Keep waiting until one of the legitimate terminal conditions occurs:

• a real streamed token becomes available;
• the Ollama stream reports an error;
• the stream completes without another token;
• the existing overall inactivity timeout expires, in which case emit the existing EOS token.

Add regressions proving the normal no-token polling path never returns token 0 and that a delayed real token is returned unchanged.