Description
I believe this could be a duplicate of #452, but the discussion there diverged from the actual point.
Problem
When using the CreateChatCompletionStream
, and hitting rate limits errors (Code: 429), the returned error nor the stream doesn't contain the http headers. I need to retrieve 'Retry-After' headers to pause the job for the specified time.
Those headers are standard and provide the amount of time (seconds/time-stamp on 'Retry-After', milliseconds on 'Retry-After-Ms').
If a 429 is obtained, what I do with CreateChatCompletion
is to parse those headers and pause to go routine for the specified amount of time.
resp, err := client.client.CreateChatCompletion(ctx, *openaiReq)
if err != nil {
... check status code ...
parseAndPause(resp.Header()) // even with error, we still get `resp`
}
But for CreateChatCompletionStream
the above code could not work as:
stream, err := client.client.CreateChatCompletionStream(ctx, *openaiReq)
if err != nil {
... check status code ...
parseAndPause(stream.Header()) // stream.header could be nil, thus this panics
}
Proposed solution
I believe here:
resp, err := client.config.HTTPClient.Do(req) //nolint:bodyclose // body is closed in stream.Close()
if err != nil {
return new(streamReader[T]), err
}
if isFailureStatusCode(resp) {
return new(streamReader[T]), client.handleErrorResp(resp)
}
Those returns should add the response into the streamReader
struct.