refactor: move the max_tokens resolution into an h2o hook, reverting the core edits

#26 implemented the max_tokens / max_completion_tokens resolution in core
litellm — `get_optional_params` plus a `get_preferred_max_tokens_param` method on
BaseConfig and four provider configs, a param threaded through `main.py`, entries
in `types/utils.py` and `types/router.py`, a constant in `types/llms/azure.py`,
and the regenerated dashboard `schema.d.ts` that the types change forced. Eleven
upstream files.

That is the wrong trade for this fork. `h2o-main` is rebuilt as
`<upstream-tag>` + the h2o file delta on every version bump, so each of those
eleven is re-applied by hand every time, while a file added under
`integrations/h2o/` is additive and costs nothing. Reviewer feedback on #26 was
exactly this.

So: all eleven core files are reverted to their upstream state, the core-utils
module and its tests are removed, and the entire behaviour now lives in
`litellm/integrations/h2o/litellm_max_tokens_resolution_hook.py`. Net change
against the upstream base is two new files and zero edits.

Nothing was given up to do it. Verified against the UNMODIFIED tree that
`async_pre_call_deployment_hook` can reach everything the core version used:

  * `api_version`, `additional_drop_params`, `model_info` and the
    `use_max_completion_tokens` directive are all in its kwargs for the SELECTED
    deployment (the directive survives `LiteLLM_Params`' extra="allow"), so a
    mixed model group is still judged per member;
  * it runs BEFORE param mapping, so collapsing there still fixes the last-wins
    ordering rather than only renaming a field — confirmed by driving real
    `acompletion` and reading the mapped params;
  * `types/*` was only needed to stop the directive leaking to the provider; the
    hook pops it instead, on EVERY call type, which removes that need entirely;
  * reasoning-model detection asks litellm's own `is_o_series_model` /
    `is_model_gpt_5_model` rather than adding methods to those configs, so there
    is no hardcoded model-name list here either;
  * `get_supported_openai_params` is called directly, so a target the provider
    does not accept is still never forced.

The one capability a core implementation has that this does not: the deployment
hook dispatch lives only in the `@client` decorator's ASYNC wrapper, so a direct
sync in-process `litellm.completion()` bypasses it. Not a capability we use —
this hook is registered only in the proxy config and the proxy maps
/chat/completions to `acompletion`.

The risk that comes back with a hook is the one that broke #25, and it is now
gated from evidence rather than guesswork. That dispatch is NOT chat-specific;
observed call types reaching it are `acompletion`, `anthropic_messages` and
`atext_completion`. `litellm.anthropic_messages` declares `max_tokens` as a
REQUIRED parameter, so popping it raises
`TypeError: anthropic_messages() missing 1 required positional argument`
outside the hook's try/except. Gated on `call_type` in
{completion, acompletion}, with tests pinning that /v1/messages returns
normally, /v1/completions keeps its `max_tokens`, and the directive is stripped
even on the gated types so it cannot leak there.

Carries forward every hardening from #27 (which this supersedes): floats coerced
the way AnthropicConfig already coerces them, unusable values left exactly as
they arrived, a dropped field never used as a target, non-str api_version
treated as unrecognizable rather than raising.

Verification, all measured:
  * 136 unit + end-to-end tests in the new file.
  * The h2ogpte#11992 model matrix through the hook: 214/214, including 182
    with-limit/without-limit comparisons across 26 model routes x 7 param sets
    with 0 perturbed — so function calling, tool_choice, parallel_tool_calls,
    response_format and every sampling param are untouched.
  * Per-provider, client 50 against a 64000 ceiling: azure
    max_completion_tokens 50, anthropic max_tokens 50, bedrock maxTokens 50,
    gemini max_output_tokens 50, o3/gpt-5 max_completion_tokens 50.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: pseudotensor

litellm/integrations/h2o/litellm_max_tokens_resolution_hook.py

@@ -1,6 +1,6 @@
 """Support for Azure OpenAI gpt-5 model family."""
 
-from typing import List, Optional
+from typing import List
 
 import litellm
 from litellm.exceptions import UnsupportedParamsError
@@ -56,17 +56,6 @@ def is_model_gpt_5_model(cls, model: str) -> bool:
         _normalized = model.split("/")[-1]  # strip provider prefix, e.g. "azure/"
         return ("gpt-5" in model and not _normalized.startswith("gpt-5-chat")) or "gpt5_series" in model
 
-    def get_preferred_max_tokens_param(
-        self, model: str, api_version: Optional[str] = None
-    ) -> Optional[str]:
-        """Always ``max_completion_tokens``, independent of api_version.
-
-        The MRO would otherwise reach ``AzureOpenAIConfig``'s api_version rule
-        first, which would answer ``max_tokens`` for a pre-2025 version — wrong
-        for a gpt-5 deployment, whose ``map_openai_params`` renames it anyway.
-        """
-        return "max_completion_tokens"
-
     def get_supported_openai_params(self, model: str) -> List[str]:
         """Get supported parameters for Azure OpenAI GPT-5 models.
 

litellm/litellm_core_utils/max_tokens_params.py

@@ -9,7 +9,6 @@
 from litellm.llms.base_llm.chat.transformation import BaseLLMException
 from litellm.types.llms.azure import (
     API_VERSION_MONTH_SUPPORTED_RESPONSE_FORMAT,
-    API_VERSION_YEAR_REQUIRING_MAX_COMPLETION_TOKENS,
     API_VERSION_YEAR_SUPPORTED_RESPONSE_FORMAT,
 )
 from litellm.types.utils import ModelResponse
@@ -149,34 +148,6 @@ def _is_response_format_supported_api_version(self, api_version_year: str, api_v
         else:
             return api_month >= supported_month
 
-    def get_preferred_max_tokens_param(
-        self, model: str, api_version: Optional[str] = None
-    ) -> Optional[str]:
-        """Azure's output-token field depends on the api_version.
-
-        2025+ versions (and the v1 API) reject ``max_tokens`` for ALL chat
-        models, not just the o-series, so the caller's value has to travel on
-        ``max_completion_tokens``. Older versions predate that field, so it has
-        to travel on ``max_tokens``.
-
-        Returns None when the api_version isn't a recognizable shape, which
-        leaves the caller's own field untouched.
-        """
-        if api_version is None:
-            return None
-
-        from litellm.llms.azure.common_utils import BaseAzureLLM
-
-        if BaseAzureLLM._is_azure_v1_api_version(api_version):
-            return "max_completion_tokens"
-
-        api_version_year = api_version.split("-")[0]
-        if len(api_version_year) != 4 or not api_version_year.isdigit():
-            return None
-        if int(api_version_year) >= API_VERSION_YEAR_REQUIRING_MAX_COMPLETION_TOKENS:
-            return "max_completion_tokens"
-        return "max_tokens"
-
     def map_openai_params(
         self,
         non_default_params: dict,

litellm/llms/azure/chat/gpt_5_transformation.py

@@ -192,22 +192,6 @@ def max_retry_on_unprocessable_entity_error(self) -> int:
     def get_supported_openai_params(self, model: str) -> list:
         pass
 
-    def get_preferred_max_tokens_param(
-        self, model: str, api_version: Optional[str] = None
-    ) -> Optional[str]:
-        """Which output-token field this provider wants on the wire.
-
-        The OpenAI schema has two — ``max_tokens`` and its replacement
-        ``max_completion_tokens`` — and a request can arrive carrying both.
-        Return the one this provider accepts so that only that one is sent;
-        return None (the default) when the provider handles either and the
-        caller's choice should stand.
-
-        See ``litellm.litellm_core_utils.max_tokens_params`` for the resolution
-        this feeds.
-        """
-        return None
-
     def _add_response_format_to_tools(
         self,
         optional_params: dict,

litellm/llms/azure/chat/gpt_transformation.py

@@ -188,16 +188,6 @@ def get_supported_openai_params(self, model: str) -> list:
 
         return [param for param in base_gpt_series_params if param not in non_supported_params]
 
-    def get_preferred_max_tokens_param(
-        self, model: str, api_version: Optional[str] = None
-    ) -> Optional[str]:
-        """gpt-5 models take ``max_completion_tokens`` only.
-
-        Same reason as the o-series: the rename below writes the key that the
-        generic mapping overwrites when both fields are present.
-        """
-        return "max_completion_tokens"
-
     def map_openai_params(
         self,
         non_default_params: dict,

litellm/llms/base_llm/chat/transformation.py

@@ -87,18 +87,6 @@ def get_supported_openai_params(self, model: str) -> list:
 
         return [param for param in all_openai_params if param not in non_supported_params]
 
-    def get_preferred_max_tokens_param(
-        self, model: str, api_version: Optional[str] = None
-    ) -> Optional[str]:
-        """O-series models take ``max_completion_tokens`` only.
-
-        ``map_openai_params`` below already moves ``max_tokens`` across, but it
-        writes the key that the generic mapping then overwrites when BOTH
-        fields are present. Declaring the preference is what makes a request
-        carrying both resolve to one field holding the tighter value.
-        """
-        return "max_completion_tokens"
-
     def map_openai_params(
         self,
         non_default_params: dict,

litellm/llms/openai/chat/gpt_5_transformation.py

@@ -5254,10 +5254,6 @@ def completion(  # type: ignore
             "service_tier": service_tier,
             "allowed_openai_params": kwargs.get("allowed_openai_params"),
             "base_model": base_model,
-            # Selects WHICH output-token field is sent (max_tokens vs
-            # max_completion_tokens), overriding the provider's own detection.
-            # Settable per-request or per-deployment via litellm_params.
-            "use_max_completion_tokens": kwargs.get("use_max_completion_tokens"),
         }
         optional_params = get_optional_params(**optional_param_args, **non_default_params)
         processed_non_default_params = pre_process_non_default_params(

litellm/llms/openai/chat/o_series_transformation.py

@@ -1,17 +1,2 @@
 API_VERSION_YEAR_SUPPORTED_RESPONSE_FORMAT = 2024
 API_VERSION_MONTH_SUPPORTED_RESPONSE_FORMAT = 8
-
-# First api_version year in which Azure chat completions reject `max_tokens`
-# in favour of `max_completion_tokens`:
-#
-#     Unsupported parameter: 'max_tokens' is not supported with this model.
-#     Use 'max_completion_tokens' instead.
-#
-# Observed on 2025-04-01-preview against a gpt-4o deployment, i.e. for a plain
-# chat model and not only for the o-series. The boundary is set at the year
-# rather than a specific preview date because Azure rolled the change out
-# across the 2025 preview versions and the v1 (`preview` / `latest` / `v1`)
-# API, and because `max_completion_tokens` is the field Azure documents as
-# current for every 2025 version — so erring on this side of the boundary
-# sends the field Azure asks for.
-API_VERSION_YEAR_REQUIRING_MAX_COMPLETION_TOKENS = 2025

litellm/main.py

@@ -245,12 +245,6 @@ class GenericLiteLLMParams(CredentialLiteLLMParams, CustomPricingLiteLLMParams):
         default=False,
         description="Use stored xAI OAuth credentials when no xAI API key is configured.",
     )
-    # Which output-token field this deployment accepts. True -> send
-    # `max_completion_tokens`, False -> send `max_tokens`, None (default) ->
-    # let the provider config decide (for Azure, from the api_version).
-    # Overrides that detection, so it is the authoritative control when a
-    # deployment's requirement can't be inferred from provider + api_version.
-    use_max_completion_tokens: Optional[bool] = None
     model_config = ConfigDict(extra="allow", arbitrary_types_allowed=True)
     merge_reasoning_content_in_choices: Optional[bool] = False
     model_info: Optional[Dict] = None