Skip to content

[Bug]: vertex_ai region = global not supported for non-google models #11190

@taralika

Description

@taralika

What happened?

proxy_config.yaml:

- model_name: claude-sonnet-4@20250514
    litellm_params:
      model: vertex_ai/claude-sonnet-4@20250514
      vertex_project: "my-vertex-project"
      vertex_location: global

Expected:
"raw_request_api_base": "https://aiplatform.googleapis.com/v1/projects/my-vertex-project/locations/global/publishers/anthropic/models/claude-sonnet-4@20250514:rawPredict"

Actual:
"raw_request_api_base": "https://global-aiplatform.googleapis.com/v1/projects/my-vertex-project/locations/global/publishers/anthropic/models/claude-sonnet-4@20250514:rawPredict"

and this obviously leads to litellm.NotFoundError: VertexAIException

PR #10658 was a good start but it only tackled google models, now the same fix needs to be applied to non-google i.e., partner models here

Here are the partner models available on global: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations#global-preview

Image

Relevant log output

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.71.1-stable

Twitter / LinkedIn details

No response

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions