Fix Vertex AI model routing and v1beta1 for preview models#29
Merged
Conversation
…or preview models Three fixes for Vertex AI provider: 1. Include "model" key in build_request_params output so chat/chat_stream use the correct model name in URL construction instead of falling back to the default "gemini-2.0-flash" 2. Use v1beta1 API version for preview/experimental models which require it (v1 returns 404 for these models) 3. Add debug logging for request URLs to aid troubleshooting
Changelog documents the Vertex AI model routing fix and v1beta1 support for preview/experimental models.
648748d to
51bc84d
Compare
The v1beta1 fix from 584f569 had a critical bug: Model.parse stored a hardcoded v1 URL in model.base_url when GOOGLE_CLOUD_PROJECT was set, bypassing the provider's v1beta1 selection logic. Preview models still 404'd in the most common setup. Fix: default_base_url(:vertex_ai) now returns nil — URL is built at request time by the provider with proper v1/v1beta1 selection. Also adds: - Global endpoint support (required for Gemini 3.x preview models) - GOOGLE_CLOUD_LOCATION env var as fallback for GOOGLE_CLOUD_REGION - Input validation for project ID and region with helpful error messages - Comprehensive docs (README + moduledoc) with model table, setup guide - Integration test exercising Flash + Pro on global endpoint - Multi-region example script Tested: 782 tests, 0 failures. Integration-tested with service account against gemini-3.1-pro-preview and gemini-3-flash-preview on global endpoint.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build_request_params— The Vertex AI override was missing the"model"key in the params map, causingchat/2andchat_stream/2to always fall back to"gemini-2.0-flash"regardless of which model was actually requestedv1beta1API version for preview/experimental models — Preview models (e.g.gemini-3.1-pro-preview) return 404 on thev1endpoint and requirev1beta1Details
The root cause was that
build_request_params/3inNous.Providers.VertexAIbuilt a params map without a"model"key:The Gemini provider already included this correctly. Without it,
chat_stream/2fell back to the default model name when building the URL.Test plan
gemini-2.0-flash(stable) usesv1endpointgemini-3.1-pro-previewusesv1beta1endpointmix test— existing tests passendpoint/3andapi_version_for_model/1