Vertex thinking changes to disable thinking mode for 2.5 models #1090
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Summary By MatterAI
🔄 What Changed
🔍 Impact of the Change
📁 Total Files Changed
🧪 Test Added
🔒 Security Vulnerabilities
Checklist
Sequence DiagramsequenceDiagram
participant GoogleVertexAI as Vertex
participant TransformConfig as Config
participant ChatComplete as Chat
Vertex->>Config: transformGenerationConfig()
Config-->>Vertex: Configure thinking mode
Vertex->>Chat: chatComplete()
Chat-->>Vertex: Process token usage
Note over Vertex, Chat: Token Tracking
Vertex->Chat: Extract thoughtsTokenCount
Chat-->Vertex: Return token details
|
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Summary By MatterAI
🔄 What Changed
🔍 Impact of the Change
📁 Total Files Changed
🧪 Test Added
🔒 Security Vulnerabilities
Type of Change
Checklist
Sequence DiagramsequenceDiagram
participant GoogleVertexAI as Vertex
participant ChatComplete as ChatAPI
Vertex->>ChatAPI: Request with thinking parameters
Note over Vertex: thinking.type = 'enabled'
Note over Vertex: thinking.budget_tokens configured
ChatAPI-->>Vertex: Response with thoughtsTokenCount
Vertex->>ChatAPI: Parse token details
Note over ChatAPI: Add completion_tokens_details
ChatAPI-->>Vertex: Return reasoning_tokens
|
VisargD
reviewed
May 14, 2025
| @@ -52,7 +52,8 @@ export function transformGenerationConfig(params: Params) { | |||
|
|
|||
| if (params?.thinking) { | |||
| const thinkingConfig: Record<string, any> = {}; | |||
| thinkingConfig['include_thoughts'] = true; | |||
| thinkingConfig['include_thoughts'] = | |||
There was a problem hiding this comment.
Based on this vertex doc: https://cloud.google.com/vertex-ai/generative-ai/docs/thinking#budget
In order to disable thinking, you need to set thinking budget tokens to 0. I do not see any parameter named include_thoughts available.
There was a problem hiding this comment.
I found this parameter looking inside the sdk, I'll find it and link it here
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
make changes to disable thinking mode in 2.5 models
also return thinking tokens in the response as per open ai format
example request body to disable thinking
{ "model": "gemini-2.5-flash-preview-04-17", "max_tokens": 2000, "stream": true, "messages": [ { "role": "user", "content": "What is the meanin of life, universe and everything" } ], "thinking": { "type": "disabled", "budget_tokens": 0 } }NOTE: users are required to send
type disabledandbudget_tokens: 0explicitly