Cannot get any streaming API's samples to work, all get an EOF

[joe-azps]

The ChatGPT streaming completion example.

c := openai.NewClient("your token")
ctx := context.Background()

req := openai.ChatCompletionRequest{
	Model:     openai.GPT3Dot5Turbo,
	MaxTokens: 20,
	Messages: []openai.ChatCompletionMessage{
		{
			Role:    openai.ChatMessageRoleUser,
			Content: "in 1000 words, tell me how the weather is during the summer in arizona.",
		},
	},
	Stream: true,
}
stream, err := c.CreateChatCompletionStream(ctx, req)
if err != nil {
	fmt.Printf("ChatCompletionStream error: %v\n", err)
	return
}
defer stream.Close()

fmt.Printf("Stream response: ")
for {
	response, err := stream.Recv()
	if errors.Is(err, io.EOF) {
		fmt.Printf("\nStream finished: %v\n", err)
		return
	}

	if err != nil {
		fmt.Printf("\nStream error: %v\n", err)
		return
	}

	fmt.Printf(response.Choices[0].Delta.Content)
}

This is getting an EOF.

Stream response: Title: The Scorching Summer Weather of Arizona: A Closer Look Introduction (72 words Stream finished: EOF

Also the GPT-3 streaming completion gets the same EOF

c := openai.NewClient("your token")
ctx := context.Background()

req := openai.CompletionRequest{
	Model:     openai.GPT3Ada,
	MaxTokens: 5,
	Prompt:    "in 1000 words, tell me how the weather is during the summer in arizona.",
	Stream:    true,
}
stream, err := c.CreateCompletionStream(ctx, req)
if err != nil {
	fmt.Printf("CompletionStream error: %v\n", err)
	return
}
defer stream.Close()

for {
	response, err := stream.Recv()
	if errors.Is(err, io.EOF) {
		fmt.Printf("Stream finished: %v\n", err)
		return
	}

	if err != nil {
		fmt.Printf("Stream error: %v\n", err)
		return
	}

	fmt.Printf("Stream response: %v\n", response)
}

This is getting an EOF error as well.

Stream response: {cmpl-7wwMeAqPs5v21hnVcB9P0sTmxrBkm text_completion 1694281732 ada [{ labour 0 {[] [] [] []}}] {0 0 0}} Stream response: {cmpl-7wwMeAqPs5v21hnVcB9P0sTmxrBkm text_completion 1694281732 ada [{t 0 {[] [] [] []}}] {0 0 0}} Stream response: {cmpl-7wwMeAqPs5v21hnVcB9P0sTmxrBkm text_completion 1694281732 ada [{rend 0 {[] [] [] []}}] {0 0 0}} Stream response: {cmpl-7wwMeAqPs5v21hnVcB9P0sTmxrBkm text_completion 1694281732 ada [{s 0 {[] [] [] []}}] {0 0 0}} Stream response: {cmpl-7wwMeAqPs5v21hnVcB9P0sTmxrBkm text_completion 1694281732 ada [{. 0 length {[] [] [] []}}] {0 0 0}} Stream finished: EOF

Environment:

• go-openai version: v1.15.2
• Go version: 1.21
• OpenAI API version: not sure how I find this out.
• OS: MacOS Ventura 13.5.1

Thanks for your time.

[ealvar3z]

@joe-azps try the same example and omit the MaxTokens field in your request object.

[ealvar3z]

@joe-azps you're asking for a 1000 words, but your max tokens is set at 5. Try changing MaxTokens or omit and it'll be limited to your openai-key; then report back.

[gou-jjjj]

This may be due to your network, and I have also encountered this issue before

[tobiassundman]

I am also getting this issue when looping too quickly over stream.Recv(). If I add a sleep between reads it always succeeds and if I add retries of io.EOF it also succeeds. Can we make it possible to distinguish between buffer empty io.EOF and response completion signalled by GPT io.EOF?