-
-
Notifications
You must be signed in to change notification settings - Fork 5.1k
Description
What happened?
proxy_config.yaml:
- model_name: claude-sonnet-4@20250514
litellm_params:
model: vertex_ai/claude-sonnet-4@20250514
vertex_project: "my-vertex-project"
vertex_location: global
Expected:
"raw_request_api_base": "https://aiplatform.googleapis.com/v1/projects/my-vertex-project/locations/global/publishers/anthropic/models/claude-sonnet-4@20250514:rawPredict"
Actual:
"raw_request_api_base": "https://global-aiplatform.googleapis.com/v1/projects/my-vertex-project/locations/global/publishers/anthropic/models/claude-sonnet-4@20250514:rawPredict"
and this obviously leads to litellm.NotFoundError: VertexAIException
PR #10658 was a good start but it only tackled google models, now the same fix needs to be applied to non-google i.e., partner models here
Here are the partner models available on global: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations#global-preview
Relevant log output
Are you a ML Ops Team?
Yes
What LiteLLM version are you on ?
v1.71.1-stable
Twitter / LinkedIn details
No response
