feat(azure): support separate deployment names and improve configuration validation#732
Open
Pranay772629 wants to merge 3 commits into
Open
feat(azure): support separate deployment names and improve configuration validation#732Pranay772629 wants to merge 3 commits into
Pranay772629 wants to merge 3 commits into
Conversation
…om model names Azure OpenAI uses the deployment name in chat/embeddings requests, while tiktoken expects a real model name for tokenizer lookup. The previous implementation implicitly assumed deployment name == model name, which breaks token estimation when users configure arbitrary deployment names in Azure. This change: - Introduces explicit deployment name fields for generative + embedding models - Uses *_MODEL_NAME for tokenizer/estimation and *_MODEL_DEPLOYMENT_NAME for Azure API calls - Adds AZURE_GENERATIVE_MODEL_DEPLOYMENT_NAME and AZURE_EMBEDDING_MODEL_DEPLOYMENT_NAME - Updates verify_environment() documentation Signed-off-by: Pranay Chandra Devasani <49197121+Pranay772629@users.noreply.github.com>
…method Extract environment variable retrieval and validation logic for Azure custom models into a reusable `_get_custom_params` method. This improves error handling by explicitly checking that both model name and deployment name are provided together, and reduces code duplication between the generative and embedding model getters. Changes: - Fix typos - Add `Literal` type import for stricter type hints - Create `_get_custom_params` method to handle EMBEDDING and GENERATIVE model configuration - Validate that both MODEL_NAME and MODEL_DEPLOYMENT_NAME are provided together - Raise explicit exception when only one of the pair is set - Update `get_schematic_generator` and `get_embedder` to use the new method Signed-off-by: Pranay Chandra Devasani <49197121+Pranay772629@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for custom Azure OpenAI deployment names separate from model names, and refactors configuration validation to improve error handling and reduce code duplication.
Problem
Azure OpenAI allows custom deployment names that differ from model names, but the previous implementation assumed
deployment_name == model_name, causing:tiktokenneeds actual model names)Changes
New Environment Variables
AZURE_GENERATIVE_MODEL_DEPLOYMENT_NAME- Deployment name in Azure PortalAZURE_EMBEDDING_MODEL_DEPLOYMENT_NAME- Deployment name in Azure PortalKey distinction:
*_MODEL_NAME→ For tokenizer initialization (e.g.,gpt-4o)*_MODEL_DEPLOYMENT_NAME→ For Azure API requests (can be any custom name)Refactoring
_get_custom_params(kind: Literal["EMBEDDING", "GENERATIVE"])method to centralize validationMODEL_NAMEandMODEL_DEPLOYMENT_NAMEare provided togetherget_schematic_generator()andget_embedder()to use shared validationLiteraltype import for stricter type safetyExample Configuration
Supports custom deployment names
export AZURE_GENERATIVE_MODEL_NAME="gpt-4o" # For tokenizer
export AZURE_GENERATIVE_MODEL_DEPLOYMENT_NAME="my-company-gpt4o" # For API