Fix incorrect handling of max_tokens=0 in chat requests

[Chen-0210]

Motivation

Currently, a request with max_tokens=0 is treated as None due to the code "max_new_tokens": self.max_tokens or self.max_completion_tokens. When self.max_tokens=0 and self.max_completion_tokens=None, the expression 0 or None evaluates to None. As a result, the backend replaces None with a very large default value, causing the user-specified 0 to be ignored.

Modifications

Checklist

• Format your code according to the Format code with pre-commit.

[gemini-code-assist]

Summary of Changes

Hello @Chen-0210, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a bug where specifying max_tokens=0 in chat requests was not honored. The previous implementation inadvertently converted 0 to None, causing the system to default to a much larger token limit. The change introduces a more precise conditional assignment to ensure that max_tokens=0 is correctly passed through and applied, aligning with user expectations for controlling token generation.

Highlights

• Incorrect max_tokens=0 handling: Previously, a max_tokens=0 value in chat requests was incorrectly treated as None due to the use of the or operator, leading the backend to ignore the user-specified zero and apply a default large value instead.
• Corrected max_new_tokens assignment logic: The logic for assigning max_new_tokens has been updated to explicitly check if self.max_tokens is None, ensuring that 0 is now correctly respected as a valid user-specified value.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly fixes a bug where max_tokens=0 was being improperly handled in chat completion requests. The change from using a logical or to an explicit is not None check ensures that a value of 0 is respected. I've added one suggestion to further improve the logic by prioritizing the newer max_completion_tokens field over the deprecated max_tokens for better forward compatibility.

[gemini-code-assist]

While this change correctly fixes the bug for max_tokens=0, it prioritizes the deprecated max_tokens field. For better API design and forward compatibility, it's generally preferable to prioritize the newer max_completion_tokens field.

This ensures that if a client sends both the old and new parameters, the newer, non-deprecated one is used, which is a more robust behavior. Old clients that only send max_tokens will still function correctly.

Suggested change

	"max_new_tokens": (
	self.max_tokens
	if self.max_tokens is not None
	else self.max_completion_tokens
	),
	"max_new_tokens": (
	self.max_completion_tokens
	if self.max_completion_tokens is not None
	else self.max_tokens
	),