Some output tokens went missing in AI responses

[gexgd0419]

Used the web version https://animalnots.github.io/BetterChatGPT-PLUS/ with OpenRouter API and gpt-4o-mini.

The response from AI is missing some words.

But checking the network activity, the response seems normal.

So the response "等差数列求和的公式可以用以下方式表示：" becomes "差列和的公式用以下方式：", and math formulas are broken as well.

[animalnots]

1. can you try enable markdown mode

2) in settings confirm inline latex checked in

[gexgd0419]

I've enabled the inline LaTeX setting. But not only the formulas are broken, some words in text are missing as well.

Here's another example:

Network activity:

[gexgd0419]

I tried the original version https://bettergpt.chat/, which I have used quite a while ago with an official OpenAI API key, and it worked at that time. But now with an OpenRouter key, it also started to drop some output tokens.

Since I don't have a usable OpenAI key now, I cannot test whether the key or endpoint specifically is causing the issue. The next step would be to experiment locally with a debugger.

[gexgd0419]

So here's my observations so far:

OpenRouter API sometimes will send : OPENROUTER PROCESSING at the beginning of the response stream. This is interestingly not shown in the "EventStream" tab, only in the raw response data. Seems that this is messing up the data parsing.

Here's the code that parses the text:

BetterChatGPT-PLUS/src/hooks/useSubmit.ts

Lines 195 to 217 in c7b7304

	while (reading && useStore.getState().generating) {
	const { done, value } = await reader.read();
	const result = parseEventSource(
	partial + new TextDecoder().decode(value)
	);
	partial = '';
	if (result === '[DONE]' || done) {
	reading = false;
	} else {
	const resultString = result.reduce((output: string, curr) => {
	if (typeof curr === 'string') {
	partial += curr;
	} else {
	if (!curr.choices || !curr.choices[0] || !curr.choices[0].delta) {
	// cover the case where we get some element which doesnt have text data, e.g. usage stats
	return output;
	}
	const content = curr.choices[0]?.delta?.content ?? null;
	if (content) output += content;
	}
	return output;
	}, '');

Assume that the text : OPENROUTER PROCESSING is read in the first line. Since parseEventSource cannot parse it as JSON, the string will be put as-is in the result array, and the rest of the elements are actual JSON objects that the code expected. After processing, that non-JSON string will be stored in partial.

During the next read, partial will be combined with the text read. Unfortunately, there are no delimeters between them, so the partial string : OPENROUTER PROCESSING will be stuck together permanently with the first JSON object string in this read, since parseEventSource can neither split them apart nor parse the combined string as JSON. Therefore, this token will be missing in the output text.

As more reads are processed, more JSON objects ends up being stuck into partial after the : OPENROUTER PROCESSING line. That's why tokens become missing randomly.

Just found in MDN docs that a colon (:) in front of a line indicates a comment, and should be ignored. This should be an easy fix.

[animalnots]

can u provide exact steps to replicate issue and suggested fix

[gexgd0419]

This issue happens when using OpenRouter APIs, since they send comment lines starting with a colon, which isn't handled properly in the code.

Suggested fix: #117.