feat: complete AI provider configurations with universal model pattern support by johnlanni · Pull Request #234 · higress-group/higress-standalone

johnlanni · 2026-02-01T11:55:06Z

Summary

This PR completes the AI Gateway template configuration by adding initialization for all new providers introduced in PR #219, PLUS universal model pattern configuration support for ALL providers (not just new ones).

Background

PR #219 added configuration functions for many new AI providers in get-ai-gateway.sh, but:

The actual template generation in all-in-one/scripts/config-template/ai-gateway.sh was missing corresponding initialization code
Some new providers (like AWS Bedrock, Google Vertex AI, OpenRouter, etc.) have non-deterministic model sets
Existing providers (OpenAI, Claude, Gemini, etc.) also need flexible model configuration for complex routing scenarios

Changes

1. New Providers Added to Template (ai-gateway.sh) ✅

AWS Bedrock - with region and authentication configuration
Google Vertex AI - with project/region/service account
OpenRouter - multi-provider router
Cloudflare Workers AI - with account ID
DeepL - translation service
Dify - AI workflow platform
iFlyTek Spark - enabled
Tencent Hunyuan - with auth ID/key
Fireworks AI, GitHub Models, Grok, Groq, Together AI

2. Universal Model Pattern Configuration 🎯

ALL providers now support custom model patterns!

Environment Variables (Template)

Every provider now checks for a corresponding *_MODELS environment variable:

# Top providers
DASHSCOPE_MODELS="qwen"                    # default
DEEPSEEK_MODELS="deepseek"                 # default
MOONSHOT_MODELS="moonshot-.*|kimi-.*"      # default
ZHIPUAI_MODELS="GLM-"                      # default
MINIMAX_MODELS="abab"                      # default
AZURE_MODELS="gpt-.*|o1-.*|o3-.*"          # default
OPENAI_MODELS="gpt-.*|o1-.*|o3-.*"         # default

# Other providers
YI_MODELS="yi-"
AI360_MODELS="360GPT"
BAICHUAN_MODELS="Baichuan"
BAIDU_MODELS="ERNIE-"
CLAUDE_MODELS="claude-"
COHERE_MODELS="command|command-.*"
DOUBAO_MODELS="doubao-"
GEMINI_MODELS="gemini-"
MISTRAL_MODELS="open-mistral-.*|mistral-.*"
OLLAMA_MODELS="codellama.*|llama.*"
STEPFUN_MODELS="step-"

# Plus all new providers (Bedrock, Vertex, etc.)

CLI Parameters

# All providers now support --xxx-models parameters
./get-ai-gateway.sh start \
  --dashscope-key="$KEY" --dashscope-models="qwen-.*" \
  --deepseek-key="$KEY" --deepseek-models="deepseek-coder.*" \
  --openai-key="$KEY" --openai-models="gpt-4.*" \
  --claude-key="$KEY" --claude-models="claude-3.*" \
  --gemini-key="$KEY" --gemini-models="gemini-1.5.*"

Available CLI parameters:

--dashscope-models, --deepseek-models, --moonshot-models
--zhipuai-models, --minimax-models, --azure-models
--openai-models, --yi-models, --ai360-models
--baichuan-models, --baidu-models, --claude-models
--cohere-models, --doubao-models, --gemini-models
--mistral-models, --ollama-models, --stepfun-models
Plus all new providers (bedrock, vertex, openrouter, cloudflare, deepl, dify, fireworks, github, grok, groq, spark, hunyuan, togetherai)

Interactive Prompts

When configuring ANY provider interactively, users are now prompted:

→ Enter API Key for OpenAI: ***
Enter model pattern for routing (regex, e.g., 'gpt-.*|o1-.*|o3-.*'):
→ Model pattern (default: gpt-.*|o1-.*|o3-.*): gpt-4.*

→ Enter API Key for Claude: ***
→ Enter Claude Version (default: 2023-06-01): 
Enter model pattern for routing (prefix match, e.g., 'claude-'):
→ Model pattern (default: claude-): claude-3.*

3. Complete Configuration Functions Coverage

New functions added:

configureDashscopeProvider()
configureDeepSeekProvider()
configureMoonshotProvider()
configureZhipuAIProvider()
configureOpenAIProvider()
configureYiProvider()
configureAI360Provider()
configureBaichuanProvider()
configureBaiduProvider()
configureCohereProvider()
configureDoubaoProvider()
configureGeminiProvider()
configureMistralProvider()
configureStepfunProvider()
configureOpenRouterProvider()
configureFireworksProvider()
configureGitHubProvider()
configureGrokProvider()
configureGroqProvider()
configureTogetherAIProvider()

Updated existing functions with model pattern support:

configureAzureProvider()
configureMinimaxProvider()
configureClaudeProvider()
configureOllamaProvider()
configureBedrockProvider()
configureVertexProvider()
configureCloudflareProvider()
configureDeepLProvider()
configureDifyProvider()
configureSparkProvider()
configureHunyuanProvider()

4. Key Implementation Details

Conditional Initialization: Each provider is only initialized if properly configured (via *_CONFIGURED markers or API keys)
Sensible Defaults: Every provider has a default model pattern matching the original hard-coded values
Environment Variable Pattern: Model patterns are stored as PROVIDER_MODELS and properly passed to template via LLM_ENVS array
Better Organization:
- Reordered providers to match get-ai-gateway.sh (commonly used first)
- Comprehensive comments for each provider section
- Alphabetical ordering in "other providers" section

Use Cases

Use Case 1: Multi-Provider Model Routing

# Route specific OpenAI models to OpenAI, others to DeepSeek
./get-ai-gateway.sh start \
  --openai-key="$OPENAI_KEY" --openai-models="gpt-4-turbo.*" \
  --deepseek-key="$DEEPSEEK_KEY" --deepseek-models=".*"

Use Case 2: Multi-Cloud Setup

# Claude to Bedrock, Gemini to Vertex, GPT to Azure, others to OpenRouter
./get-ai-gateway.sh start \
  --bedrock-key="$BEDROCK_KEY" --bedrock-models="claude-.*" \
  --vertex-key="$VERTEX_KEY" --vertex-models="gemini-.*" \
  --azure-key="$AZURE_KEY" --azure-models="gpt-.*" \
  --openrouter-key="$OPENROUTER_KEY" --openrouter-models=".*"

Use Case 3: Version-Specific Routing

# Route only Claude 3.5 Sonnet to one provider, others elsewhere
./get-ai-gateway.sh start \
  --claude-key="$CLAUDE_KEY" --claude-models="claude-3-5-sonnet.*"

Use Case 4: Development vs Production

# Development: All to Ollama
./get-ai-gateway.sh start \
  --ollama-server-host="localhost" --ollama-models=".*"

# Production: Provider-specific routing
./get-ai-gateway.sh start \
  --openai-key="$KEY" --openai-models="gpt-4.*" \
  --deepseek-key="$KEY" --deepseek-models="deepseek-.*" \
  --moonshot-key="$KEY" --moonshot-models="kimi-.*"

Testing

This should be tested with:

Configuring providers via CLI with custom model patterns
Configuring providers interactively and verifying model pattern prompts
Verifying environment variables are properly passed to template
Testing model routing with various patterns

Benefits

✅ Universal: ALL providers now support custom model patterns
✅ Flexible: Both CLI and interactive configuration supported
✅ Backward Compatible: Defaults match original hard-coded patterns
✅ Comprehensive: Coverage for all 30+ providers
✅ User-Friendly: Sensible defaults with helpful examples

This commit completes the AI Gateway template configuration by adding initialization for all new providers introduced in PR higress-group#219: **New providers added:** - AWS Bedrock (with region and auth configuration) - Google Vertex AI (with project/region/service account) - OpenRouter (multi-provider router) - Cloudflare Workers AI (with account ID) - DeepL (translation service with target language) - Dify (AI workflow platform with bot type and variables) - iFlyTek Spark - Tencent Hunyuan (with auth ID/key) - Fireworks AI - GitHub Models - Grok (xAI) - Groq - Together AI **Key improvements:** 1. Added conditional initialization for all new providers based on configuration markers set in get-ai-gateway.sh 2. Implemented support for custom model patterns via environment variables (e.g., BEDROCK_MODELS, VERTEX_MODELS, etc.) for providers with non-deterministic model sets 3. Properly configured extra configs (regions, auth keys, account IDs) for providers requiring additional parameters 4. Maintained alphabetical ordering for better readability 5. Added comprehensive comments for each provider section **Environment variable pattern:** For providers with uncertain model sets, users can now customize the model matching pattern, e.g.: - BEDROCK_MODELS="claude-.*" for AWS Bedrock Claude models - VERTEX_MODELS="gemini-.*" for Vertex AI Gemini models - Default: ".*" (matches all models) Fixes the issue where PR higress-group#219 added provider configurations in get-ai-gateway.sh but the actual template generation in ai-gateway.sh was missing the corresponding initialization.

This commit adds comprehensive model pattern configuration support: **Interactive mode improvements:** - Added model pattern configuration prompts to all provider config functions - For providers with known model patterns (Vertex=gemini-.*, Bedrock=.*, etc.), provided sensible defaults - Users can specify custom regex patterns during interactive setup **CLI parameter support:** Added new CLI arguments for model pattern configuration: - --bedrock-models, --vertex-models, --openrouter-models - --cloudflare-models, --deepl-models, --dify-models - --fireworks-models, --github-models, --grok-models - --groq-models, --spark-models, --hunyuan-models - --togetherai-models **New configuration functions:** Added dedicated config functions for providers that previously only supported API key input: - configureOpenRouterProvider() - configureFireworksProvider() - configureGitHubProvider() - configureGrokProvider() - configureGroqProvider() - configureTogetherAIProvider() **Usage examples:** CLI mode: ```bash ./get-ai-gateway.sh start --bedrock-key="xxx" --bedrock-models="claude-.*" ./get-ai-gateway.sh start --openrouter-key="xxx" --openrouter-models=".*" ``` Interactive mode: Users are now prompted to enter model patterns after API credentials, with helpful examples and sensible defaults. All model pattern environment variables are properly passed to the ai-gateway.sh template via LLM_ENVS array.

This commit extends model pattern configuration to ALL providers, not just the newly added ones. **Template changes (ai-gateway.sh):** All providers now support custom model patterns via environment variables: - DASHSCOPE_MODELS (default: qwen) - DEEPSEEK_MODELS (default: deepseek) - MOONSHOT_MODELS (default: moonshot-.*|kimi-.*) - ZHIPUAI_MODELS (default: GLM-) - MINIMAX_MODELS (default: abab) - AZURE_MODELS (default: gpt-.*|o1-.*|o3-.*) - OPENAI_MODELS (default: gpt-.*|o1-.*|o3-.*) - YI_MODELS (default: yi-) - AI360_MODELS (default: 360GPT) - BAICHUAN_MODELS (default: Baichuan) - BAIDU_MODELS (default: ERNIE-) - CLAUDE_MODELS (default: claude-) - COHERE_MODELS (default: command|command-.*) - DOUBAO_MODELS (default: doubao-) - GEMINI_MODELS (default: gemini-) - MISTRAL_MODELS (default: open-mistral-.*|mistral-.*) - OLLAMA_MODELS (default: codellama.*|llama.*) - STEPFUN_MODELS (default: step-) - (Plus all previously added providers) **CLI parameter support:** Added corresponding --xxx-models parameters for all providers: --dashscope-models, --deepseek-models, --moonshot-models, --zhipuai-models, --minimax-models, --azure-models, --openai-models, --yi-models, --ai360-models, --baichuan-models, --baidu-models, --claude-models, --cohere-models, --doubao-models, --gemini-models, --mistral-models, --ollama-models, --stepfun-models **Interactive configuration functions:** Added/updated configuration functions for all providers: - configureDashscopeProvider() - configureDeepSeekProvider() - configureMoonshotProvider() - configureZhipuAIProvider() - configureOpenAIProvider() - configureYiProvider() - configureAI360Provider() - configureBaichuanProvider() - configureBaiduProvider() - configureCohereProvider() - configureDoubaoProvider() - configureGeminiProvider() - configureMistralProvider() - configureStepfunProvider() Updated existing functions with model pattern support: - configureAzureProvider() - configureMinimaxProvider() - configureClaudeProvider() - configureOllamaProvider() Each function now prompts for model patterns with sensible defaults matching the template configuration. **Usage examples:** ```bash # Customize models for standard providers ./get-ai-gateway.sh start \ --openai-key="$KEY" --openai-models="gpt-4.*" \ --dashscope-key="$KEY" --dashscope-models="qwen-.*" # Mix and match providers by model ./get-ai-gateway.sh start \ --deepseek-key="$KEY" --deepseek-models="deepseek-coder.*" \ --moonshot-key="$KEY" --moonshot-models="kimi.*" ``` Now EVERY provider supports custom model pattern configuration, enabling fine-grained control over model routing.

This commit adds a unified model pattern normalization system that supports multiple flexible input formats and intelligently converts them to optimal match types. **New normalizeModelPattern() function:** Supports the following input formats: 1. **Wildcard suffix**: `qwen-*` → PRE `qwen-` - Converts to prefix match for efficiency 2. **Exact model name**: `qwen3-max` → PRE `qwen3-max` - Uses prefix match for exact names 3. **Multiple wildcards**: `kimi-*,moonshot-*` → REGULAR `kimi-.*|moonshot-.*` - Comma-separated patterns with wildcards - Converts to regex with OR operator 4. **Multiple exact names**: `kimi-2.5,moonshot-k1` → REGULAR `kimi-2.5|moonshot-k1` - Comma-separated exact model names - Creates regex OR pattern 5. **Mixed patterns**: `qwen-*,gpt-4` → REGULAR `qwen-.*|gpt-4` - Supports mixing wildcards and exact names 6. **Match all**: `*` → REGULAR `.*` - Universal wildcard for all models **Smart type detection:** - Single pattern with `-*` suffix → PRE (prefix match, more efficient) - Multiple patterns (comma-separated) → REGULAR (regex with OR) - Complex wildcards → REGULAR (full regex power) - Empty pattern → REGULAR `.*` (match all) **Updated all provider configurations:** All providers now use normalizeModelPattern() to process their model patterns, enabling: - Consistent user experience across all providers - Flexible configuration via environment variables - Automatic optimization (PRE vs REGULAR) - Support for complex multi-model routing **Default pattern updates:** Changed defaults to use the more intuitive wildcard syntax: - `qwen-*` instead of `qwen` - `gpt-*,o1-*,o3-*` instead of `gpt-.*|o1-.*|o3-.*` - `moonshot-*,kimi-*` instead of `moonshot-.*|kimi-.*` Users can now configure models in the most natural way: ```bash # Simple prefix match DASHSCOPE_MODELS="qwen-*" # Multiple models OPENAI_MODELS="gpt-4-turbo,gpt-4o" # Mix exact and wildcard CLAUDE_MODELS="claude-3-5-sonnet-*,claude-opus-4" # Comma-separated wildcards MOONSHOT_MODELS="moonshot-*,kimi-*" ``` The normalization function handles all conversion automatically!

johnlanni · 2026-02-01T12:26:03Z

🎯 Latest Update: Universal Model Pattern Normalization

Added a powerful new normalizeModelPattern() function that makes model configuration much more flexible and user-friendly!

Supported Input Formats

Users can now configure models using any of these intuitive formats:

1️⃣ Wildcard Prefix (most common)

DASHSCOPE_MODELS="qwen-*"          # Matches: qwen-max, qwen-plus, qwen-turbo, etc.
OPENAI_MODELS="gpt-*"               # Matches: gpt-4, gpt-4-turbo, gpt-3.5-turbo, etc.
CLAUDE_MODELS="claude-*"            # Matches: claude-3-opus, claude-3-5-sonnet, etc.

→ Converted to PRE (prefix match) for efficiency

2️⃣ Exact Model Name

DASHSCOPE_MODELS="qwen3-max"        # Matches only: qwen3-max
OPENAI_MODELS="gpt-4-turbo"         # Matches only: gpt-4-turbo

→ Converted to PRE for exact match

3️⃣ Multiple Wildcard Patterns

MOONSHOT_MODELS="moonshot-*,kimi-*"     # Matches: moonshot-v1, kimi-2.5, etc.
OPENAI_MODELS="gpt-*,o1-*,o3-*"         # Matches: gpt-4, o1-preview, o3-mini, etc.
MISTRAL_MODELS="mistral-*,open-mistral-*"  # Matches both series

→ Converted to REGULAR with OR: moonshot-.*|kimi-.*

4️⃣ Multiple Exact Names

OPENAI_MODELS="gpt-4-turbo,gpt-4o,gpt-4o-mini"
CLAUDE_MODELS="claude-3-5-sonnet-20241022,claude-opus-4-20250514"

→ Converted to REGULAR with OR: gpt-4-turbo|gpt-4o|gpt-4o-mini

5️⃣ Mixed Patterns

# Mix wildcards and exact names
CLAUDE_MODELS="claude-3-5-sonnet-*,claude-opus-4-20250514"
OPENAI_MODELS="gpt-4-*,o1-preview"

→ Converted to REGULAR: claude-3-5-sonnet-.*|claude-opus-4-20250514

6️⃣ Match All

OPENROUTER_MODELS="*"     # Catch-all provider
BEDROCK_MODELS="*"        # Route all models to Bedrock

→ Converted to REGULAR: .*

Smart Optimization

The function automatically chooses the most efficient match type:

PRE (prefix match): For simple model-* or exact names → faster
REGULAR (regex): For complex patterns or multiple models → flexible

Real-World Examples

Multi-provider routing by model family:

./get-ai-gateway.sh start \
  --openai-key="$KEY" --openai-models="gpt-4-*" \
  --deepseek-key="$KEY" --deepseek-models="deepseek-*" \
  --claude-key="$KEY" --claude-models="claude-3-5-sonnet-*"

Route specific model versions:

./get-ai-gateway.sh start \
  --openai-key="$KEY" --openai-models="gpt-4-turbo,gpt-4o" \
  --claude-key="$KEY" --claude-models="claude-opus-4-20250514"

Catch-all with exceptions:

# Route GPT-4 to OpenAI, everything else to OpenRouter
./get-ai-gateway.sh start \
  --openai-key="$KEY" --openai-models="gpt-4-*" \
  --openrouter-key="$KEY" --openrouter-models="*"

Updated Defaults

All defaults now use the more intuitive wildcard syntax:

✅ qwen-* (was: qwen)
✅ gpt-*,o1-*,o3-* (was: gpt-.*|o1-.*|o3-.*)
✅ moonshot-*,kimi-* (was: moonshot-.*|kimi-.*)
✅ claude-* (was: claude-)

Benefits

✅ User-friendly: Natural * wildcard syntax
✅ Flexible: Mix exact names and wildcards freely
✅ Efficient: Auto-optimizes to PRE when possible
✅ Powerful: Full regex capability when needed
✅ Consistent: Same syntax across all providers

This makes Higress AI Gateway configuration as simple as possible while maintaining full power for complex scenarios! 🎉

Added detailed documentation for model pattern configurations in the --help output, including: **Format Documentation:** Documented all 6 supported input formats with examples: 1. Wildcard prefix: `qwen-*` (matches qwen-max, qwen-plus, etc.) 2. Exact model name: `gpt-4-turbo` (matches only this model) 3. Multiple wildcards: `moonshot-*,kimi-*` (matches multiple series) 4. Multiple exact names: `gpt-4-turbo,gpt-4o,gpt-4o-mini` 5. Mixed patterns: `claude-3-5-sonnet-*,claude-opus-4-20250514` 6. Match all: `*` (catch-all provider) Each format includes: - Clear example usage - What it matches - When to use it **Available Options:** Listed all 30+ --xxx-models parameters with provider names: - --dashscope-models, --deepseek-models, --moonshot-models - --zhipuai-models, --minimax-models, --azure-models - --openai-models, --openrouter-models, --yi-models - ... and all other providers **Practical Examples:** Added real-world usage examples: 1. Route specific models to specific providers: ```bash ./get-ai-gateway.sh start --non-interactive \ --openai-key sk-xxx --openai-models "gpt-4-*" \ --deepseek-key sk-xxx --deepseek-models "deepseek-*" \ --claude-key sk-xxx --claude-models "claude-3-5-sonnet-*" ``` 2. Multi-provider setup with exact model names: ```bash ./get-ai-gateway.sh start --non-interactive \ --openai-key sk-xxx --openai-models "gpt-4-turbo,gpt-4o" \ --claude-key sk-xxx --claude-models "claude-opus-4-20250514" ``` 3. Use OpenRouter as catch-all: ```bash ./get-ai-gateway.sh start --non-interactive \ --openai-key sk-xxx --openai-models "gpt-4-*" \ --openrouter-key sk-xxx --openrouter-models "*" ``` Users can now run `./get-ai-gateway.sh -h` to get comprehensive guidance on model pattern configuration!

johnlanni requested a review from CH3CHO as a code owner February 1, 2026 11:55

johnlanni added 2 commits February 1, 2026 20:01

johnlanni changed the title ~~feat: complete AI provider configurations in ai-gateway template~~ feat: complete AI provider configurations with universal model pattern support Feb 1, 2026

johnlanni merged commit 5616993 into higress-group:main Feb 1, 2026
3 checks passed

johnlanni mentioned this pull request Feb 1, 2026

feat: enhance Clawdbot integration SKILL with comprehensive provider list and independent setup script higress-group/higress#3435

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: complete AI provider configurations with universal model pattern support#234

feat: complete AI provider configurations with universal model pattern support#234
johnlanni merged 5 commits into
higress-group:mainfrom
johnlanni:feat/complete-ai-provider-configs

johnlanni commented Feb 1, 2026 •

edited

Loading

Uh oh!

johnlanni commented Feb 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

johnlanni commented Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background

Changes

1. New Providers Added to Template (ai-gateway.sh) ✅

2. Universal Model Pattern Configuration 🎯

Environment Variables (Template)

CLI Parameters

Interactive Prompts

3. Complete Configuration Functions Coverage

4. Key Implementation Details

Use Cases

Use Case 1: Multi-Provider Model Routing

Use Case 2: Multi-Cloud Setup

Use Case 3: Version-Specific Routing

Use Case 4: Development vs Production

Testing

Benefits

Related

Uh oh!

johnlanni commented Feb 1, 2026

🎯 Latest Update: Universal Model Pattern Normalization

Supported Input Formats

1️⃣ Wildcard Prefix (most common)

2️⃣ Exact Model Name

3️⃣ Multiple Wildcard Patterns

4️⃣ Multiple Exact Names

5️⃣ Mixed Patterns

6️⃣ Match All

Smart Optimization

Real-World Examples

Updated Defaults

Benefits

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

johnlanni commented Feb 1, 2026 •

edited

Loading