Skip to content

feat(vertexai): support meta's llama models#266

Open
CorieW wants to merge 11 commits into
genkit-ai:mainfrom
invertase:vertexai-support-llama
Open

feat(vertexai): support meta's llama models#266
CorieW wants to merge 11 commits into
genkit-ai:mainfrom
invertase:vertexai-support-llama

Conversation

@CorieW

@CorieW CorieW commented Apr 28, 2026

Copy link
Copy Markdown
Member

Support Meta's Llama models in VertexAI.

Testing

image image

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Meta models (e.g., Llama) on Vertex AI by implementing an OpenAI-compatible API client and necessary model options. The changes include adding chat completion and streaming support, defining the Meta model configuration schema, and updating the plugin to resolve and list these models. The review feedback highlights the importance of ensuring tool call IDs and tool response references are present for API compatibility, and suggests adding support for resolving 'gs://' URIs for image inputs, as the endpoint may not support them natively.

Comment thread packages/genkit_vertexai/lib/src/meta_model.dart
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart Outdated
@CorieW CorieW force-pushed the vertexai-support-llama branch from b35337f to 4db5379 Compare May 11, 2026 12:25
@CorieW

CorieW commented May 11, 2026

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Meta models (Llama) within the Vertex AI plugin. It adds a new VertexAiMetaOptions class for model configuration, implements request/response mapping for the OpenAI-compatible endpoints used by Meta models on Vertex AI, and updates the VertexAiPluginImpl to discover and resolve these models. Feedback includes an optimization to cache the API client to reduce overhead and suggestions to use GenkitException with specific status codes instead of UnimplementedError for better error handling consistency.

Comment thread packages/genkit_vertexai/lib/src/vertex_api_client.dart
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart Outdated
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart Outdated
@CorieW CorieW requested a review from cabljac May 11, 2026 12:40
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart Outdated
Comment thread packages/genkit_vertexai/lib/src/vertex_api_client.dart Outdated
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart Outdated
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart
@CorieW CorieW force-pushed the vertexai-support-llama branch from 28b33a6 to 4d20853 Compare May 11, 2026 14:06
Comment thread packages/genkit_vertexai/lib/src/vertex_api_client.dart
Comment thread packages/genkit_vertexai/lib/src/meta_model.dart
@CorieW CorieW force-pushed the vertexai-support-llama branch from b6cdbbf to 4d20853 Compare May 11, 2026 14:18
@CorieW CorieW marked this pull request as ready for review May 11, 2026 15:20
@CorieW CorieW requested a review from pavelgj May 11, 2026 15:20
@cabljac cabljac requested a review from kevmoo May 12, 2026 10:10
@cabljac cabljac enabled auto-merge (squash) May 12, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants