fix: harden the max_tokens resolution — four regressions found by red-teaming

Adversarially probing #26 against the pre-change tree (275646b) turned up
four behaviour regressions and one dead branch. Every row below is measured,
not reasoned.

1. A NUMERIC CEILING WAS SILENTLY DELETED. `_positive_int` rejected anything
   that was not an `int`, and the "strip an unusable value from a rejected
   field" clause then removed it. So on Azure 2025:

       max_tokens=50.0   before {'max_tokens': 50.0}  after {}   <- unbounded
       max_tokens='50'   before {'max_tokens': '50'}  after {}   <- unbounded
       max_tokens=0      before {'max_tokens': 0}     after {}
       max_tokens=True   before {'max_tokens': True}  after {}

   Floats genuinely reach litellm — `AnthropicConfig.map_openai_params` coerces
   one with `max(1, int(round(value)))`, and `anthropic max_tokens=50.0` really
   does go out as 50. Turning a caller's 50.0 into no ceiling at all is worse
   than the defect #26 fixed.

   Fixed two ways. Numeric values are now usable and coerced the same way
   Anthropic already coerces them (NaN and inf rejected). And rule 3 no longer
   strips anything: when no usable value can be derived, NOTHING is changed and
   the provider rejects the request as loudly as it always did. Garbage in,
   error out — that clause bought nothing (Azure rejects `max_tokens: 0` on its
   own merits either way) and cost a silent unbounding.

2. IT RESURRECTED A DROPPED PARAM. A field in `additional_drop_params` was
   still eligible as a target, so `drop: [max_completion_tokens]` plus a client
   `max_tokens` went out AS `max_completion_tokens` — defeating an explicit
   operator instruction:

       drop=[max_completion_tokens], MT=50
           before {'max_tokens': 50}   after {'max_completion_tokens': 50}

   The drop list is now consulted for every target choice, including the
   directive's. If both fields are dropped or unsupported, nothing happens.

3. A NON-STRING api_version RAISED A NEW EXCEPTION TYPE. The v1-api check does
   a set membership test, so a list or dict `api_version` raised
   `unhashable type` from param mapping where the pre-change tree raised
   `AttributeError` from its own `.split()`. Now anything that is not a `str`
   yields "no preference" — this lookup runs on every Azure chat request and
   must not invent a failure mode. The remaining int/bytes/list crashes are
   litellm's own pre-existing `api_version.split("-")` and are now byte-for-byte
   identical to pre-change.

4. THE PREFERENCE RESOLVED AGAINST A DIFFERENT api_version THAN THE REQUEST
   USED. The fallback to `litellm.api_version` / `AZURE_API_VERSION` /
   `AZURE_DEFAULT_API_VERSION` triggered on `is None`, while the azure branch
   below it uses `or`-falsiness. An `api_version=""` therefore resolved the
   preference against nothing while the request went out on the 2025 default —
   `max_tokens` to a version that rejects it. Now matches the same falsiness.

DEAD BRANCH REMOVED. The `extra_body` handling could never fire: `extra_body`
is not a mappable chat param, so it never appears in `non_default_params`.
Verified — with `extra_body={'max_completion_tokens': 64000}` neither the strip
nor the min() saw it. Code that claims a guarantee it does not provide is worse
than no code, so it is gone and the limitation is documented instead: a caller
who reaches for `extra_body` is explicitly bypassing param mapping, and the h2o
cap hook is where that channel is policed.

ALSO VERIFIED, NO CHANGE NEEDED. `azure_text` (the legacy /completions
endpoint, which has no `max_completion_tokens` at all) resolves to a different
provider config and is untouched — the same class of hazard that made the
earlier deployment-hook approach break /v1/messages. Now covered by a test.

Live re-verification through a litellm proxy against a capture server, client
sending max_tokens=50:

    azure 2025, mt=50            -> max_completion_tokens: 50
    azure 2025, mt=50.0          -> max_completion_tokens: 50
    azure 2025, drop=[MCT]       -> max_tokens: 50
    azure 2025, directive=false  -> max_tokens: 50
    anthropic ceiling 64000      -> max_tokens: 50

86 tests, up from 60. The float, drop-list, api_version-type and
leave-garbage-alone cases all fail on #26's head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: pseudotensor

litellm/litellm_core_utils/max_tokens_params.py

@@ -6,9 +6,9 @@
 ``max_tokens`` and its replacement ``max_completion_tokens``. A request can
 arrive with both — most commonly because a proxy deployment configures one of
 them as a ceiling in ``litellm_params`` while the client sends the other. When
-that happens today, what reaches the provider depends on the order the
-provider's ``map_openai_params`` happens to iterate, and the result is wrong in
-two different ways:
+that happens, what reaches the provider depends on the order the provider's
+``map_openai_params`` happens to iterate, and the result is wrong in two
+different ways:
 
   * Providers that collapse both onto one field do so with LAST-WINS
     semantics, because both branches assign the same output key. ``anthropic``
@@ -30,22 +30,43 @@
 
 THE RULES
 ---------
-1. TIGHTER WINS. When both fields carry a usable value, the smaller one is
+1. TIGHTER WINS. When more than one usable value is present, the smallest is
    kept. A deployment ceiling can therefore never be RAISED by a client, and a
    client's tighter request can never be widened to the ceiling.
 
-2. ONE FIELD OUT. When a preferred field is known, only that field survives.
-   The preference comes from the provider config
-   (``BaseConfig.get_preferred_max_tokens_param``), or from an explicit
+2. ONE FIELD OUT. When a target field is known, only that field survives. The
+   target comes from the provider config
+   (``BaseConfig.get_preferred_max_tokens_param``) or from an explicit
    per-request/per-deployment ``use_max_completion_tokens`` directive, which
    overrides the provider's own detection.
 
-3. NEVER INVENT A VALUE. If neither field holds a positive integer, nothing is
-   changed. A ``max_tokens`` of ``0``, ``None``, a string, or a bool must not
-   become a ``max_completion_tokens`` that truncates every response.
+3. NEVER SILENTLY DROP A CEILING. If no usable value can be derived — a
+   ``max_tokens`` of ``0``, ``"50"``, ``True``, ``None`` — NOTHING is changed.
+   The request goes on exactly as it would have without this module, and the
+   provider rejects it as loudly as it always did. Rewriting it here would
+   turn a garbage-in/error-out request into an unbounded one, which is a worse
+   failure than the one being fixed. Numeric values ARE usable and are coerced
+   to a positive int, matching what ``AnthropicConfig.map_openai_params``
+   already does with a float ``max_tokens``.
+
+4. NEVER RESURRECT A DROPPED PARAM. A field listed in
+   ``additional_drop_params`` is off limits as a target. An operator who
+   dropped a param meant it, and moving a value onto that field would defeat
+   the drop.
+
+OUT OF SCOPE: ``extra_body``
+----------------------------
+A copy of either field inside ``extra_body`` is NOT considered here, and
+deliberately so — ``extra_body`` is not a mappable chat param, so it never
+reaches ``non_default_params`` and cannot be read at this layer. It is merged
+into the request body further downstream, which means a caller who puts an
+output-token field there can still override what this resolves. That is the
+pre-existing behaviour of that channel (a caller reaching for ``extra_body`` is
+explicitly bypassing param mapping), and the h2o ``max_tokens`` cap hook is
+where ``extra_body`` is policed.
 """
 
-from typing import Any, Dict, Optional, Sequence
+from typing import Any, Dict, List, Optional, Sequence
 
 MAX_TOKENS_PARAM = "max_tokens"
 MAX_COMPLETION_TOKENS_PARAM = "max_completion_tokens"
@@ -59,30 +80,64 @@ def preferred_param_for_directive(
     """Translate the ``use_max_completion_tokens`` directive to a field name.
 
     ``None`` means "no directive given" — fall back to provider detection.
+    Only the exact booleans are honoured; anything else (a stray ``"false"``
+    string from a YAML config, say) is treated as "not given" rather than
+    coerced by truthiness into the opposite of what it reads like.
     """
-    if use_max_completion_tokens is None:
+    if use_max_completion_tokens is True:
+        return MAX_COMPLETION_TOKENS_PARAM
+    if use_max_completion_tokens is False:
+        return MAX_TOKENS_PARAM
+    return None
+
+
+def _usable_int(value: Any) -> Optional[int]:
+    """Return ``value`` as a positive int, or None if it isn't a usable limit.
+
+    ``bool`` is rejected explicitly because it is an ``int`` subclass. Floats
+    are accepted and rounded, because a float ``max_tokens`` really does reach
+    litellm — ``AnthropicConfig.map_openai_params`` coerces one with
+    ``max(1, int(round(value)))``, and this mirrors that so the two agree.
+    Strings are NOT coerced: a value litellm cannot interpret must be left for
+    the provider to reject (rule 3).
+    """
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
         return None
+    if isinstance(value, float):
+        if value != value or value in (float("inf"), float("-inf")):  # NaN / inf
+            return None
+        if value <= 0:
+            return None
+        return max(1, int(round(value)))
+    return value if value > 0 else None
+
+
+def _is_dropped(param: str, additional_drop_params: Optional[Sequence[str]]) -> bool:
+    """Mirror of ``litellm.utils._should_drop_param`` for these two fields."""
     return (
-        MAX_COMPLETION_TOKENS_PARAM
-        if use_max_completion_tokens
-        else MAX_TOKENS_PARAM
+        additional_drop_params is not None
+        and isinstance(additional_drop_params, (list, tuple))
+        and param in additional_drop_params
     )
 
 
-def _positive_int(value: Any) -> Optional[int]:
-    """Return ``value`` as a positive int, or None if it isn't one.
-
-    ``bool`` is rejected explicitly because it is an ``int`` subclass.
-    """
-    if isinstance(value, bool) or not isinstance(value, int):
-        return None
-    return value if value > 0 else None
+def _eligible(
+    param: str,
+    supported_params: Sequence[str],
+    additional_drop_params: Optional[Sequence[str]],
+) -> bool:
+    """A field can only be a target if the provider accepts it (rule 2) and the
+    operator has not dropped it (rule 4)."""
+    return param in supported_params and not _is_dropped(
+        param, additional_drop_params
+    )
 
 
 def resolve_max_tokens_params(
     non_default_params: Dict[str, Any],
     supported_params: Sequence[str],
     preferred_param: Optional[str] = None,
+    additional_drop_params: Optional[Sequence[str]] = None,
 ) -> Optional[str]:
     """Collapse ``max_tokens`` / ``max_completion_tokens`` in place.
 
@@ -94,6 +149,8 @@ def resolve_max_tokens_params(
             output-token ceiling at all.
         preferred_param: the field to keep, or None to let the caller's own
             field stand when only one was sent.
+        additional_drop_params: this deployment's drop list; a field in it is
+            never used as a target.
 
     Returns:
         The field that survived, or None when nothing was changed.
@@ -104,41 +161,50 @@ def resolve_max_tokens_params(
 
     values = [
         v
-        for v in (_positive_int(non_default_params[p]) for p in present)
+        for v in (_usable_int(non_default_params[p]) for p in present)
         if v is not None
     ]
     if not values:
-        # Rule 3 — nothing usable to move or tighten. Still strip a field the
-        # provider is known to reject: an unusable value (0, a string) is not a
-        # ceiling worth preserving, and leaving it on a field the provider
-        # rejects turns a garbage request into a guaranteed 400.
-        if preferred_param is not None and preferred_param in supported_params:
-            for param in MAX_TOKENS_PARAMS:
-                if param != preferred_param:
-                    non_default_params.pop(param, None)
+        # Rule 3 — nothing usable. Leave the request exactly as it was so the
+        # provider rejects it as loudly as it would have without us.
         return None
     resolved = min(values)
 
     target: Optional[str] = None
-    if preferred_param is not None and preferred_param in supported_params:
+    if preferred_param is not None and _eligible(
+        preferred_param, supported_params, additional_drop_params
+    ):
         target = preferred_param
     elif len(present) == 1:
-        # One field, no preference: the caller's choice already is canonical.
+        # One field, no provider preference: the caller's choice already is
+        # canonical.
         return None
     else:
-        # Both fields with no provider preference. Collapsing is still
-        # required — leaving both is what makes the last-wins provider maps
-        # pick the looser value. Prefer `max_tokens`: every provider
-        # understands it, and the reasoning-model configs that want
-        # `max_completion_tokens` rename it themselves downstream.
-        target = (
-            MAX_TOKENS_PARAM
-            if MAX_TOKENS_PARAM in supported_params
-            else MAX_COMPLETION_TOKENS_PARAM
-        )
+        # Both fields, no provider preference. Collapsing is still required:
+        # leaving both is what makes the last-wins provider maps pick the looser
+        # value. Prefer `max_tokens` — every provider understands it, and the
+        # reasoning-model configs that want `max_completion_tokens` rename it
+        # themselves downstream.
+        for candidate in MAX_TOKENS_PARAMS:
+            if _eligible(candidate, supported_params, additional_drop_params):
+                target = candidate
+                break
+        if target is None:
+            # Both fields are unsupported or dropped: there is nothing to move
+            # the value onto, and inventing one would defeat the drop.
+            return None
 
     for param in MAX_TOKENS_PARAMS:
         if param != target:
             non_default_params.pop(param, None)
     non_default_params[target] = resolved
     return target
+
+
+__all__ = [
+    "MAX_COMPLETION_TOKENS_PARAM",
+    "MAX_TOKENS_PARAM",
+    "MAX_TOKENS_PARAMS",
+    "preferred_param_for_directive",
+    "resolve_max_tokens_params",
+]

litellm/llms/azure/chat/gpt_transformation.py

@@ -160,9 +160,12 @@ def get_preferred_max_tokens_param(
         to travel on ``max_tokens``.
 
         Returns None when the api_version isn't a recognizable shape, which
-        leaves the caller's own field untouched.
+        leaves the caller's own field untouched. A non-str api_version (an int
+        year, bytes, a list) is treated as unrecognizable rather than allowed to
+        raise: this runs on every Azure chat request, so it must not turn a
+        misconfigured api_version into a traceback from param mapping.
         """
-        if api_version is None:
+        if not isinstance(api_version, str):
             return None
 
         from litellm.llms.azure.common_utils import BaseAzureLLM

litellm/utils.py

@@ -3883,10 +3883,11 @@ def _check_valid_arg(supported_params: List[str]):
     # limit to the deployment ceiling. See
     # litellm.litellm_core_utils.max_tokens_params for the rules.
     _max_tokens_api_version = api_version
-    if custom_llm_provider == "azure" and _max_tokens_api_version is None:
-        # Mirror the fallback chain the azure branch below applies, so the
-        # api_version-dependent preference is resolved against the version the
-        # request will actually be sent with.
+    if custom_llm_provider == "azure" and not _max_tokens_api_version:
+        # Mirror the fallback chain the azure branch below applies — including
+        # its `or`-based falsiness, so an empty-string api_version resolves the
+        # preference against the same version the request is actually sent with
+        # rather than against nothing.
         _max_tokens_api_version = (
             litellm.api_version
             or get_secret("AZURE_API_VERSION")
@@ -3903,6 +3904,7 @@ def _check_valid_arg(supported_params: List[str]):
         non_default_params=non_default_params,
         supported_params=supported_params,
         preferred_param=_preferred_max_tokens_param,
+        additional_drop_params=additional_drop_params,
     )
 
     _check_valid_arg(