Skip to content

Fix Vertex AI model routing and v1beta1 for preview models#29

Merged
nyo16 merged 4 commits intomasterfrom
fix/vertex-ai-v1beta1-preview-models
Mar 12, 2026
Merged

Fix Vertex AI model routing and v1beta1 for preview models#29
nyo16 merged 4 commits intomasterfrom
fix/vertex-ai-v1beta1-preview-models

Conversation

@nyo16
Copy link
Copy Markdown
Owner

@nyo16 nyo16 commented Mar 10, 2026

Summary

  • Include model name in build_request_params — The Vertex AI override was missing the "model" key in the params map, causing chat/2 and chat_stream/2 to always fall back to "gemini-2.0-flash" regardless of which model was actually requested
  • Use v1beta1 API version for preview/experimental models — Preview models (e.g. gemini-3.1-pro-preview) return 404 on the v1 endpoint and require v1beta1
  • Add debug logging for Vertex AI request URLs to aid troubleshooting

Details

The root cause was that build_request_params/3 in Nous.Providers.VertexAI built a params map without a "model" key:

# Before (broken)
params = %{"contents" => contents}

# After (fixed)
params = %{"model" => model.model, "contents" => contents}

The Gemini provider already included this correctly. Without it, chat_stream/2 fell back to the default model name when building the URL.

Test plan

  • Verify gemini-2.0-flash (stable) uses v1 endpoint
  • Verify gemini-3.1-pro-preview uses v1beta1 endpoint
  • Verify the correct model name appears in the URL (not the default)
  • Run mix test — existing tests pass
  • Doctest examples for endpoint/3 and api_version_for_model/1

nyo16 added 3 commits March 11, 2026 12:58
…or preview models

Three fixes for Vertex AI provider:

1. Include "model" key in build_request_params output so chat/chat_stream
   use the correct model name in URL construction instead of falling back
   to the default "gemini-2.0-flash"

2. Use v1beta1 API version for preview/experimental models which require
   it (v1 returns 404 for these models)

3. Add debug logging for request URLs to aid troubleshooting
Changelog documents the Vertex AI model routing fix and v1beta1 support
for preview/experimental models.
@nyo16 nyo16 force-pushed the fix/vertex-ai-v1beta1-preview-models branch from 648748d to 51bc84d Compare March 11, 2026 16:59
The v1beta1 fix from 584f569 had a critical bug: Model.parse stored a
hardcoded v1 URL in model.base_url when GOOGLE_CLOUD_PROJECT was set,
bypassing the provider's v1beta1 selection logic. Preview models still
404'd in the most common setup.

Fix: default_base_url(:vertex_ai) now returns nil — URL is built at
request time by the provider with proper v1/v1beta1 selection.

Also adds:
- Global endpoint support (required for Gemini 3.x preview models)
- GOOGLE_CLOUD_LOCATION env var as fallback for GOOGLE_CLOUD_REGION
- Input validation for project ID and region with helpful error messages
- Comprehensive docs (README + moduledoc) with model table, setup guide
- Integration test exercising Flash + Pro on global endpoint
- Multi-region example script

Tested: 782 tests, 0 failures. Integration-tested with service account
against gemini-3.1-pro-preview and gemini-3-flash-preview on global endpoint.
@nyo16 nyo16 merged commit b6cc231 into master Mar 12, 2026
6 checks passed
@nyo16 nyo16 deleted the fix/vertex-ai-v1beta1-preview-models branch March 12, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant