Skip to content

Commit f839497

Browse files
NickGaganclaude
andauthored
fix(gen-ai): remove max_tokens from external model verify request (opendatahub-io#7088)
max_tokens was hardcoded to 10 in the chat completions verification request, which could cause failures with models that don't support the field. Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
1 parent 1914770 commit f839497

1 file changed

Lines changed: 2 additions & 4 deletions

File tree

  • packages/gen-ai/bff/internal/integrations/externalmodels

packages/gen-ai/bff/internal/integrations/externalmodels/client.go

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,8 @@ import (
2121

2222
// chatCompletionRequest represents an OpenAI-compatible chat completion request
2323
type chatCompletionRequest struct {
24-
Model string `json:"model"`
25-
Messages []chatCompletionMessage `json:"messages"`
26-
MaxTokens int `json:"max_tokens,omitempty"`
24+
Model string `json:"model"`
25+
Messages []chatCompletionMessage `json:"messages"`
2726
}
2827

2928
// chatCompletionMessage represents a message in the chat completion request
@@ -235,7 +234,6 @@ func (c *ExternalModelsClient) VerifyModel(ctx context.Context, modelID string,
235234
Messages: []chatCompletionMessage{
236235
{Role: "user", Content: "test"},
237236
},
238-
MaxTokens: 10,
239237
}
240238
requestBody, err = json.Marshal(chatReq)
241239
if err != nil {

0 commit comments

Comments
 (0)