Timeout / abortion handling

[hyperknot]

Thank you for this great library.

It works well, however, I have problem understanding how to implement timeout / abortion handling.

Here are 3 distinct timeout/abortion cases:

1. timeout during the initial fetch event (server not responding, etc.)
2. timeout during the SSE streaming. Sometimes providers just stop responding in the middle of the stream
3. user clicks a "Stop" kind of button and wants to abort the streaming.

I've tried my best to implement these, but I cannot yet see the logic how to do it.

Here is my snippet so far, using the standard OpenAI streaming format. Can you explain where should these be implemented?

I guess 2. has to be a second timer started/restarted in the middle of the for loop. The others I don't yet have an idea.

export async function* getOpenaiStream({ prompt, timeout = 1 }) {
  const controller = new AbortController()
  const timeoutId = setTimeout(() => controller.abort(), timeout * 1000)

  try {
    const es = createEventSource({
      url: `${BASE_URL}/chat/completions`,
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${API_KEY}`,
      },
      body: JSON.stringify({
        model: MODEL,
        messages: [
          {
            role: 'user',
            content: prompt,
          },
        ],
        stream: true,
      }),
    })

    try {
      for await (const event of es) {
        if (event.data === '[DONE]') {
          break
        }

        try {
          const data = JSON.parse(event.data)
          if (data?.error) {
            throw new ProviderError(data.error.message, 'OPENAI_API_ERROR')
          }
          const content = data.choices?.[0]?.delta?.content
          if (content) {
            yield content
          }
        } catch (e) {
          es.close()
          clearTimeout(timeoutId)
        }
      }
    } finally {
      es.close()
      clearTimeout(timeoutId)
    }
  } catch (error) {
    clearTimeout(timeoutId)
  }
}