forked from ENTERPILOT/GoModel
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy path.env.template
More file actions
608 lines (522 loc) · 29.4 KB
/
Copy path.env.template
File metadata and controls
608 lines (522 loc) · 29.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
# Server Configuration
# PORT=8080
# Mount the whole gateway under a path prefix, e.g. https://example.com/g/
# BASE_PATH=/g
# Header used to read/write request user_path values (default: X-GoModel-User-Path)
# USER_PATH_HEADER=X-GoModel-User-Path
# Where the running gateway records its process id so `gomodel --reload` can find it
# (default: data/gomodel.pid next to a ./data directory, otherwise the per-user data
# directory). Set it per instance when several gateways share a host. Leaving this
# empty means "unset" and keeps the default, as everywhere else here; to write no pid
# file at all (which also disables --reload), set `server.pid_file: ""` in config.yaml.
# Changing it takes effect on the next restart, not on `gomodel --reload`.
# PID_FILE=data/gomodel.pid
# Reject unknown keys in config.yaml and in the JSON env vars that declare the same
# structures (VIRTUAL_MODELS, SET_RATE_LIMIT_*, SET_BUDGET_*). Default: true, so a
# typo or a misindented section fails startup instead of silently dropping providers,
# rate limits, budgets, or guardrails. Set false to downgrade unknown keys to
# warnings — intended for rolling a binary back under a newer config file. Malformed
# values stay fatal in either mode.
# CONFIG_STRICT=true
# Tagging based on headers: label every request from the listed headers (numbered
# from 1). Labels are recorded in usage tracking and audit logs. A header value can
# carry several labels split by the delimiter (default: ","). The optional prefix is
# trimmed from each extracted label only — the header itself is forwarded unchanged
# unless DONOTPASS=true. An env entry replaces the whole config.yaml `tagging:` entry
# with the same header name (leaving _PREFIX/_DONOTPASS/_DELIMITER unset resets those
# fields to their defaults, it does not inherit the YAML values); other env entries
# are appended. Both are read-only in the dashboard.
# TAGGING_HEADER_1=X-My-Tags
# TAGGING_HEADER_1_PREFIX=tag-
# TAGGING_HEADER_1_DONOTPASS=false
# TAGGING_HEADER_1_DELIMITER=,
# TAGGING_HEADER_2=X-Internal-Routing
# TAGGING_HEADER_2_DONOTPASS=true
# Session keeping: identify which requests belong to one client session, for sticky
# virtual-model load balancing and audit-log session grouping. On by default with a
# built-in registry of session headers/body fields known coding tools send (Claude
# Code, Codex CLI, OpenCode, Kilo Code, Goose, …) plus content-based auto-detection
# for untagged chat requests. Extra headers are numbered from 1 and merged over the
# built-ins; the optional TRANSFORM "session-uuid" extracts a session_<uuid> value.
# SESSION_KEEPING_ENABLED=true
# SESSION_AUTO_DETECT=true
# SESSION_BUILTIN_RULES=true
# SESSION_HEADER_1=X-My-Session
# SESSION_HEADER_1_TRANSFORM=
# Log output format: leave unset to auto-detect, or set to "json" / "text"
# LOG_FORMAT=text
# Log verbosity: "debug", "info" (default), "warn", or "error"
# LOG_LEVEL=info
# Maximum request body size (prevents DoS attacks)
# Accepts values like "10M", "1G", "500K" (default: 10M)
# BODY_SIZE_LIMIT=10M
# Enable/disable Swagger UI at /swagger/index.html (default: true)
# SWAGGER_ENABLED=true
# Enable/disable pprof profiling routes at /debug/pprof/* (default: false)
# PPROF_ENABLED=false
# Enable/disable provider-native passthrough routes under /p/{provider}/{endpoint} (default: true)
# ENABLE_PASSTHROUGH_ROUTES=true
# Allow optional /p/{provider}/v1/... passthrough aliases while keeping /p/{provider}/... canonical (default: true)
# ALLOW_PASSTHROUGH_V1_ALIAS=true
# Comma-separated list of provider types enabled for /p/{provider}/... passthrough (default: openai,anthropic,openrouter,kilo,zai,vllm,llmd,deepseek)
# Cohere native passthrough is opt-in; add cohere when those routes are needed.
# ENABLED_PASSTHROUGH_PROVIDERS=openai,anthropic,cohere,openrouter,kilo,zai,vllm,llmd,deepseek
# Enable the realtime (speech-to-speech) endpoints (default: true): the /v1/realtime
# websocket (and /p/{provider}/v1/realtime passthrough upgrade), the WebRTC SDP
# exchange at POST /v1/realtime/calls, and ephemeral browser credentials at
# POST /v1/realtime/client_secrets. Credentials are injected by the gateway, so
# clients connect without provider API keys; GET /v1/realtime?call_id=... attaches
# to an existing WebRTC call as a sideband channel. WebRTC media flows directly
# between client and provider; the gateway proxies the signaling and records usage
# via a sideband observer on each call it creates. WebRTC providers: OpenAI, xAI
# (calls are team-gated by xAI), and Azure OpenAI (GA v1 surface).
# REALTIME_ENABLED=true
# MCP gateway: aggregate upstream MCP (Model Context Protocol) servers behind the
# authenticated /mcp endpoint (default: true; a no-op until servers are declared here,
# in config.yaml under `mcp.servers`, or in the dashboard). Tools are namespaced as
# {server}_{tool}; /mcp/{server} exposes one upstream with original tool names.
# MCP_ENABLED=true
# MCP_SERVERS is a JSON object mapping server names to definitions; entries replace
# config.yaml `mcp.servers` entries with the same name and are read-only in the
# dashboard. Transports: "http" (streamable HTTP, default), "sse" (legacy), "stdio"
# (spawns a local subprocess — declarable only here or in config.yaml, never via the
# admin API, because runtime-registered subprocesses would be a code execution vector).
# MCP_SERVERS={"github":{"url":"https://api.githubcopilot.com/mcp","headers":{"Authorization":"Bearer ${GITHUB_PAT}"}}}
# HTTP Client Configuration (for upstream API requests)
# Values in seconds (or Go duration format like "10m", "1h30m")
# Overall request timeout (default: 600 = 10 minutes, matches OpenAI/Anthropic SDKs)
# HTTP_TIMEOUT=600
# Time to wait for response headers (default: 600)
# HTTP_RESPONSE_HEADER_TIMEOUT=600
# Security Configuration
# CRITICAL: Set this to secure your gateway from unauthorized access
# If not set, the server will run in UNSAFE MODE with a warning
# GOMODEL_MASTER_KEY=your-secret-key-here
# Metrics Configuration (Prometheus)
# Enable/disable Prometheus metrics collection and /metrics endpoint
# METRICS_ENABLED=false
# Custom metrics endpoint path (default: /metrics)
# METRICS_ENDPOINT=/metrics
# Cache Configuration
# Model cache uses the local filesystem by default.
# Set REDIS_URL to use Redis-backed caching instead.
# Add provider-native prompt-cache keys, breakpoints, and cached-content plans
# after routing (default: true). Set false as an operational kill switch; empty
# or invalid values keep the default. Client cache directives still pass through.
# PROVIDER_PROMPT_CACHE_PLANNER_ENABLED=true
# Redis Configuration
# REDIS_URL=redis://localhost:6379
# REDIS_KEY_MODELS=gomodel:models
# REDIS_TTL_MODELS=86400
# How often to refresh the model registry cache in seconds (default: 3600).
# Also sets how often provider health ("Last checked" in the dashboard) is
# re-checked; lower it to detect provider outages/recoveries faster.
# CACHE_REFRESH_INTERVAL=3600
# How often to re-probe only the providers whose last refresh failed, in
# seconds (default: 60, 0 disables). Detects provider recovery without
# waiting for the next full refresh.
# PROVIDER_RECHECK_INTERVAL=60
# REDIS_KEY_RESPONSES=gomodel:response:
# REDIS_TTL_RESPONSES=3600
# Opt-in when config.yaml has no cache.response.simple block (e.g. env-only deploys). Omit otherwise.
# RESPONSE_CACHE_SIMPLE_ENABLED=true
# Opt-in when config.yaml has no cache.response.semantic block (e.g. env-only deploys). Omit otherwise.
# SEMANTIC_CACHE_ENABLED=true
# Similarity threshold between 0 and 1 (default: 0.92)
# SEMANTIC_CACHE_THRESHOLD=0.92
# Semantic cache entry TTL in seconds (default: 3600)
# SEMANTIC_CACHE_TTL=3600
# Number of recent conversation messages to embed (default: 3)
# SEMANTIC_CACHE_MAX_CONV_MESSAGES=3
# Exclude the system prompt from semantic cache keys (default: false)
# SEMANTIC_CACHE_EXCLUDE_SYSTEM_PROMPT=false
# Embedding provider name used for semantic cache
# SEMANTIC_CACHE_EMBEDDER_PROVIDER=openai
# Optional embedding model override
# SEMANTIC_CACHE_EMBEDDER_MODEL=text-embedding-3-small
# Vector store backend: qdrant, pgvector, pinecone, or weaviate
# SEMANTIC_CACHE_VECTOR_STORE_TYPE=qdrant
# Qdrant
# SEMANTIC_CACHE_QDRANT_URL=http://localhost:6333
# SEMANTIC_CACHE_QDRANT_COLLECTION=gomodel_semantic
# SEMANTIC_CACHE_QDRANT_API_KEY=
# pgvector
# SEMANTIC_CACHE_PGVECTOR_URL=postgres://user:pass@localhost:5432/gomodel
# SEMANTIC_CACHE_PGVECTOR_TABLE=gomodel_semantic_cache
# SEMANTIC_CACHE_PGVECTOR_DIMENSION=1536
# Pinecone
# SEMANTIC_CACHE_PINECONE_HOST=https://your-index.svc.region.pinecone.io
# SEMANTIC_CACHE_PINECONE_API_KEY=
# SEMANTIC_CACHE_PINECONE_NAMESPACE=
# SEMANTIC_CACHE_PINECONE_DIMENSION=1536
# Weaviate
# SEMANTIC_CACHE_WEAVIATE_URL=http://localhost:8080
# SEMANTIC_CACHE_WEAVIATE_CLASS=GomodelSemanticCache
# SEMANTIC_CACHE_WEAVIATE_API_KEY=
# Optional: Custom cache directory for local file cache
# Default: ./.cache when it already exists, otherwise the OS per-user cache
# directory (~/.cache/gomodel, ~/Library/Caches/gomodel, %LocalAppData%\gomodel\cache)
# GOMODEL_CACHE_DIR=.cache
# External model metadata registry (provides pricing, capabilities, context window, etc.)
# Default: ENTERPILOT/ai-model-list on GitHub. Point this at an internal mirror
# for air-gapped installs. Setting it to an empty string here does NOT disable
# the fetch (empty env values are skipped, so the default survives) -- to
# disable it, set cache.model.model_list.url: "" in config.yaml.
# MODEL_LIST_URL=https://raw.githubusercontent.com/ENTERPILOT/ai-model-list/refs/heads/main/models.min.json
# Model Access Configuration
# Process-wide default for provider models when no persisted override exists (default: true)
# Set to false to keep models unavailable until a model override allows one or more user paths.
# MODELS_ENABLED_BY_DEFAULT=true
# Hide provider models from GET /v1/models and expose only enabled aliases (default: false).
# KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT=false
# How providers.<name>.models and <PROVIDER>[_SUFFIX]_MODELS affect provider inventory.
# fallback (default): use configured models only when upstream /models fails, is nil, or is empty.
# allowlist: expose only the configured models for providers that define a list, and skip their upstream /models calls.
# CONFIGURED_PROVIDER_MODELS_MODE=fallback
# Examples: OPENROUTER_MODELS=..., OPENROUTER_EU_MODELS=..., AZURE_MODELS=..., VLLM_MODELS=...
# Virtual models as infrastructure-as-code (JSON array). Declares redirects, load
# balancers, and access policies; overrides admin-store rows with the same source
# and is read-only in the dashboard. The same entries can be declared under
# `virtual_models:` in config.yaml. Each entry: source, target (single) or
# targets[] (load balanced), strategy (round_robin | cost), user_paths,
# slowdown (0 disables inherited slowdown; active factors are 0.1-10),
# description, enabled. A target has model and optional provider + weight.
# VIRTUAL_MODELS=[{"source":"smart","strategy":"round_robin","targets":[{"model":"openai/gpt-4o","weight":2},{"model":"anthropic/claude-sonnet-4-6"}]}]
# Failover & Workflow Configuration
# Failover is manual-only at runtime and enabled by default.
# FAILOVER_ENABLED=true
# JSON file mapping primary model selectors to ordered failover target lists
# FAILOVER_MANUAL_RULES_PATH=config/failover.example.json
# Inline JSON object mapping primary model selectors to ordered failover target arrays
# FAILOVER_RULES_JSON={"gpt-4o":["azure/gpt-4o","gemini/gemini-2.5-pro"]}
# JSON array or boolean object of primary model selectors where failover is disabled
# FAILOVER_DISABLED_MODELS_JSON=["claude-sonnet-4"]
# How often to refresh persisted workflows from storage (default: 1m)
# WORKFLOW_REFRESH_INTERVAL=1m
# LLM Client Resilience Configuration
# Retry attempts for upstream provider calls (default: 3)
# RETRY_MAX_RETRIES=3
# Initial retry backoff duration (default: 1s)
# RETRY_INITIAL_BACKOFF=1s
# Maximum retry backoff duration (default: 30s)
# RETRY_MAX_BACKOFF=30s
# Exponential backoff factor (default: 2.0)
# RETRY_BACKOFF_FACTOR=2.0
# Random jitter factor applied to retry delays (default: 0.1)
# RETRY_JITTER_FACTOR=0.1
# Consecutive failures before opening the circuit breaker (default: 5)
# CIRCUIT_BREAKER_FAILURE_THRESHOLD=5
# Consecutive successes required to close the circuit breaker (default: 2)
# CIRCUIT_BREAKER_SUCCESS_THRESHOLD=2
# Circuit breaker open-state timeout duration (default: 30s)
# CIRCUIT_BREAKER_TIMEOUT=30s
# =============================================================================
# Admin API & Dashboard Configuration
# =============================================================================
# Enable/disable admin REST API endpoints (default: true)
# When enabled, provides /admin/* REST endpoints
# ADMIN_ENDPOINTS_ENABLED=true
# Enable/disable admin dashboard UI (default: true)
# When enabled, provides /admin/dashboard UI
# Requires ADMIN_ENDPOINTS_ENABLED=true — if endpoints are disabled
# and UI is enabled, a warning is logged and UI is forced to disabled
# ADMIN_UI_ENABLED=true
# Enable/disable realtime dashboard log previews (default: true)
# When enabled, the dashboard opens /admin/live/logs and streams compact
# audit/usage lifecycle updates before the async DB flush completes.
# Requires ADMIN_ENDPOINTS_ENABLED=true and ADMIN_UI_ENABLED=true for the UI.
# DASHBOARD_LIVE_LOGS_ENABLED=true
# In-memory live log replay buffer in events (default: 10000)
# The effective buffer is capped at DASHBOARD_LIVE_LOGS_REPLAY_LIMIT + 1, since
# events older than the replay window can never be served; lower this below the
# replay limit only to shrink memory further at the cost of more replay resets.
# Buffered events hold compact previews only — request/response bodies are
# never retained (connected dashboards receive them live and hydrate history
# from the persisted audit entries).
# DASHBOARD_LIVE_LOGS_BUFFER_SIZE=10000
# Maximum live log events replayed to one reconnecting dashboard client (default: 1000)
# Also bounds the replay buffer above. If a cursor is older than the replay
# window, the dashboard resets from REST APIs.
# DASHBOARD_LIVE_LOGS_REPLAY_LIMIT=1000
# Heartbeat interval in seconds for idle live log streams (default: 15)
# Keeps proxies and poor connections from closing an otherwise quiet stream.
# DASHBOARD_LIVE_LOGS_HEARTBEAT_SECONDS=15
# =============================================================================
# Storage Configuration (used by audit logging, usage tracking, future IAM, etc.)
# =============================================================================
# Storage type: "sqlite" (default), "postgresql", or "mongodb"
# This determines where both audit logs and usage data are stored
# STORAGE_TYPE=sqlite
# SQLite Configuration (default, good for single instance)
# Default: ./data/gomodel.db when a ./data directory exists, otherwise the
# OS per-user data directory (~/.local/share/gomodel, ~/Library/Application
# Support/gomodel, %LocalAppData%\gomodel)
# SQLITE_PATH=data/gomodel.db
# PostgreSQL Configuration (for multi-instance deployments)
# POSTGRES_URL=postgres://user:password@localhost:5432/gomodel
# POSTGRES_MAX_CONNS=10
# MongoDB Configuration (recommended for high-volume logging)
# MONGODB_URL=mongodb://localhost:27017/gomodel
# MONGODB_DATABASE overrides the database named in MONGODB_URL (default: gomodel)
# MONGODB_DATABASE=gomodel
# =============================================================================
# Audit Logging Configuration
# =============================================================================
# Enable/disable audit logging (default: true)
# When enabled, all requests and responses are logged to the configured storage
# LOGGING_ENABLED=true
# Log full request/response bodies (default: true)
# WARNING: May contain PII, API keys in prompts, or sensitive data
# LOGGING_LOG_BODIES=true
# Log audio endpoint inputs/outputs: /v1/audio/speech text input + binary audio
# output (stored as base64 so the dashboard can play it back) and
# /v1/audio/transcriptions upload metadata. Requires LOGGING_LOG_BODIES=true (the
# master body-logging switch); when bodies are logged but this is off, audio
# responses are recorded as a lightweight placeholder instead of the full bytes.
# WARNING: stores full audio in the audit log and grows storage quickly (default: false)
# LOGGING_LOG_AUDIO_BODIES=false
# Log request/response headers (default: true)
# Sensitive headers (Authorization, Cookie, etc.) are automatically redacted
# LOGGING_LOG_HEADERS=true
# Log only model interactions, skip /health, /metrics, /admin endpoints (default: true)
# LOGGING_ONLY_MODEL_INTERACTIONS=true
# In-memory audit log queue capacity in entries/rows, not bytes (default: 1000)
# If the queue is full, new audit log entries are dropped with a warning
# LOGGING_BUFFER_SIZE=1000
# How often to flush buffered logs in seconds (default: 5)
# Logs are written at least every 5 seconds by default, and earlier when batches fill up
# LOGGING_FLUSH_INTERVAL=5
# When LOGGING_LOG_BODIES=true, captured request/response bodies are capped at 1MB each
# Streaming response content is also capped at 1MB of accumulated captured content
# Auto-delete logs older than N days, 0 = keep forever (default: 30)
# LOGGING_RETENTION_DAYS=30
# =============================================================================
# Token Usage Tracking Configuration
# =============================================================================
# Enable/disable token usage tracking (default: true)
# When enabled, token usage is tracked separately from audit logs
# USAGE_ENABLED=true
# Enable/disable the admin usage pricing recalculation action (default: true)
# Requires USAGE_ENABLED=true and supported storage; false always hides/blocks it
# USAGE_PRICING_RECALCULATION_ENABLED=true
# Enforce returning usage data in streaming responses (default: true)
# When true, stream_options: {"include_usage": true} is automatically added
# to streaming requests for OpenAI-compatible providers
# ENFORCE_RETURNING_USAGE_DATA=true
# Enable/disable guardrails globally (default: false)
# When enabled, configured guardrails can run for workflows that reference them
# GUARDRAILS_ENABLED=false
# Guardrails for inline batch processing (default: false)
# When true, guardrails are applied to inline /v1/batches request items
# (e.g. /v1/chat/completions and /v1/responses items).
# ENABLE_GUARDRAILS_FOR_BATCH_PROCESSING=false
# In-memory buffer size before flushing to storage (default: 1000)
# USAGE_BUFFER_SIZE=1000
# How often to flush buffered usage entries in seconds (default: 5)
# USAGE_FLUSH_INTERVAL=5
# Auto-delete usage data older than N days, 0 = keep forever (default: 90)
# USAGE_RETENTION_DAYS=90
# =============================================================================
# Rate Limits
# =============================================================================
# Cap requests, tokens, and in-flight concurrency per user_path subtree.
# Breaches return 429 with Retry-After and x-ratelimit-* headers. Counters are
# in-memory per gateway instance and reset on restart; token limits (tpm/tph/
# tpd) additionally require USAGE_ENABLED=true. Enabled by default; with no
# configured rules the check is a no-op.
# RATE_LIMITS_ENABLED=true
# Declare rules per user path with SET_RATE_LIMIT_<PATH> (double underscores
# separate path segments, like SET_BUDGET_*). Names: rpm/tpm (per minute),
# rph/tph (per hour), rpd/tpd (per day), concurrent (max in-flight requests).
# A JSON array of {period|period_seconds, max_requests, max_tokens} objects is
# also accepted for custom windows.
# SET_RATE_LIMIT_TEAM__ALPHA="rpm=100,tpm=50000,rpd=10000,concurrent=10"
# SET_RATE_LIMIT_="rpm=1000"
# Cap one provider across all consumers and models with
# SET_PROVIDER_RATE_LIMIT_<NAME> (same rpm/tpm/... syntax; underscores in the
# suffix become hyphens like provider-instance env vars). Load balancing and
# failover skip a saturated provider while another target has capacity.
# Per-model rules have no env form; declare them under rate_limits.models in
# config.yaml or via the admin API/dashboard.
# SET_PROVIDER_RATE_LIMIT_OPENAI="rpm=500,tpm=200000,concurrent=50"
# =============================================================================
# Provider API Keys (uncomment and set the ones you need)
# =============================================================================
# None of this is required to start GoModel. You can leave every provider env
# var below unset and instead add provider credentials from the admin
# dashboard's Providers page (/admin/dashboard) after starting the gateway —
# they take effect immediately, no restart. Env vars and config.yaml still
# take precedence when both are set for the same provider name (the dashboard
# marks that provider read-only).
#
# Add more instances with <PROVIDER>_<SUFFIX>_*.
# Example: OPENAI_EAST_API_KEY and OPENAI_EAST_BASE_URL register provider openai-east.
# Underscores in the suffix become hyphens in the provider name.
#
# Give ONE provider several API keys with <PROVIDER>_API_KEY_<n>, numbered from 2.
# Identified sessions stay on one key by default to preserve provider prompt-cache
# affinity; sessionless traffic still rotates. Set
# <PROVIDER>[_SUFFIX]_SESSION_STICKY_KEYS=false to rotate every request instead.
# Works for any provider that authenticates with an API key.
# OpenAI
# OPENAI_API_KEY=sk-...
# OPENAI_API_KEY_2=sk-...
# OPENAI_SESSION_STICKY_KEYS=false
# OPENAI_BASE_URL=https://api.openai.com/v1
# Anthropic
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_BASE_URL=https://api.anthropic.com/v1
# Anthropic /v1/messages requires max_tokens. When the caller omits it, GoModel
# injects this value. Default: 4096. Raise it for newer models that produce
# longer outputs (e.g. Sonnet 4.6 / Opus 4.7).
# ANTHROPIC_DEFAULT_MAX_TOKENS=4096
# Cohere
# COHERE_API_KEY=...
# COHERE_BASE_URL=https://api.cohere.com
# Google Gemini
# GEMINI_API_KEY=...
# Use Gemini's native generateContent API for chat/responses (default: true).
# Set to false to use Gemini's OpenAI-compatible API for chat/responses.
# Native mode supports inline image data via data: URLs, but GoModel does not
# fetch remote image URLs or upload them through Gemini Files API yet. Set this
# to false when you need OpenAI-compatible image_url pass-through behavior.
# USE_GOOGLE_GEMINI_NATIVE_API=true
# Gemini base URL. The official defaults are:
# - native chat/models: https://generativelanguage.googleapis.com/v1beta
# - OpenAI-compatible API: https://generativelanguage.googleapis.com/v1beta/openai
# If this ends in /openai, GoModel derives the native base by stripping /openai.
# GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
# xAI (Grok)
# XAI_API_KEY=...
# XAI_BASE_URL=https://api.x.ai/v1
# Groq
# GROQ_API_KEY=gsk_...
# GROQ_BASE_URL=https://api.groq.com/openai/v1
# Fireworks AI (default base URL: https://api.fireworks.ai/inference/v1)
# Model IDs are account-scoped paths, e.g. accounts/fireworks/models/gpt-oss-120b
# FIREWORKS_API_KEY=fw_...
# FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
# Optional configured model list; see CONFIGURED_PROVIDER_MODELS_MODE below
# FIREWORKS_MODELS=accounts/fireworks/models/gpt-oss-120b,accounts/fireworks/models/deepseek-v3p2
# Meta Model API (Muse Spark, default base URL: https://api.meta.ai/v1)
# META_API_KEY=...
# META_BASE_URL=https://api.meta.ai/v1
# Optional configured model list; see CONFIGURED_PROVIDER_MODELS_MODE below
# META_MODELS=muse-spark-1.1
# OpenRouter (default base URL: https://openrouter.ai/api/v1)
# OPENROUTER_API_KEY=sk-or-...
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# Optional configured model list; see CONFIGURED_PROVIDER_MODELS_MODE below
# OPENROUTER_MODELS=openai/gpt-oss-120b,anthropic/claude-sonnet-4
# OPENROUTER_SITE_URL=https://gomodel.enterpilot.io
# OPENROUTER_APP_NAME=GoModel
# Kilo AI Gateway (default base URL: https://api.kilo.ai/api/gateway)
# Model IDs use provider/model and pass through unchanged.
# KILO_API_KEY=...
# KILO_BASE_URL=https://api.kilo.ai/api/gateway
# Optional configured model list; see CONFIGURED_PROVIDER_MODELS_MODE below
# KILO_MODELS=anthropic/claude-sonnet-4.5,openai/gpt-5.5
# Z.ai (default base URL: https://api.z.ai/api/paas/v4)
# For GLM Coding Plan, use: https://api.z.ai/api/coding/paas/v4
# ZAI_API_KEY=...
# ZAI_BASE_URL=https://api.z.ai/api/paas/v4
# MiniMax (global: https://api.minimax.io/v1; China: https://api.minimaxi.com/v1)
# MINIMAX_API_KEY=...
# MINIMAX_BASE_URL=https://api.minimax.io/v1
# ElevenLabs (voice: text-to-speech + speech-to-text; default base URL: https://api.elevenlabs.io)
# The OpenAI "voice" field must be an ElevenLabs voice_id.
# ELEVENLABS_API_KEY=...
# ELEVENLABS_BASE_URL=https://api.elevenlabs.io
# Xiaomi MiMo (default base URL: https://api.xiaomimimo.com/v1)
# XIAOMI_API_KEY=...
# XIAOMI_BASE_URL=https://api.xiaomimimo.com/v1
# Optional configured model list; [1m] variants enable 1M context
# XIAOMI_MODELS=mimo-v2.5-pro,mimo-v2.5-pro[1m],mimo-v2.5-tts,mimo-v2.5-asr
# OpenCode Go (OpenCode Zen, default base URL: https://opencode.ai/zen/go/v1)
# OPENCODE_GO_API_KEY=sk-...
# OPENCODE_GO_BASE_URL=https://opencode.ai/zen/go/v1
# Optional configured model list
# OPENCODE_GO_MODELS=glm-5.1,kimi-k2.7-code,deepseek-v4-pro
# Models that must be routed to the Anthropic-native /messages endpoint instead
# of /chat/completions (default: qwen3.7-max). Temporary manual split until
# upstream exposes per-model endpoint metadata.
# OPENCODE_GO_MESSAGES_MODELS=qwen3.7-max
# Azure OpenAI
# AZURE_API_KEY=...
# AZURE_BASE_URL=https://your-resource.openai.azure.com/openai/deployments/your-deployment
# AZURE_API_VERSION=2024-10-21
# Oracle
# ORACLE_API_KEY=...
# ORACLE_BASE_URL=https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/v1
# Optional configured model list; comma-separated, whitespace around entries is ignored
# ORACLE_MODELS=openai.gpt-oss-120b,xai.grok-3
# Ollama (local LLM server)
# Note: Ollama doesn't require an API key, but one can be sent for secured deployments
# OLLAMA_API_KEY=...
# Set base URL to enable (default: http://localhost:11434/v1)
# OLLAMA_BASE_URL=http://localhost:11434/v1
# Alibaba Cloud Bailian (百炼)
# BAILIAN_API_KEY=...
# BAILIAN_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
# BAILIAN_MODELS=text-embedding-v3,text-embedding-v4
# vLLM (OpenAI-compatible server)
# VLLM_API_KEY is optional; set it only if vllm serve was started with --api-key.
# VLLM_API_KEY=token-abc123
# Set base URL to enable (default: http://localhost:8000/v1)
# VLLM_BASE_URL=http://localhost:8000/v1
# llm-d Router / Endpoint Picker (EPP)
# The route is required and usually points to the EPP service. Include /v1.
# LLMD_BASE_URL=http://quickstart-epp.llm-d.svc.cluster.local/v1
# Optional bearer token if the Gateway in front of llm-d requires one.
# LLMD_API_KEY=router-token
# Optional configured model list when GET /v1/models is not routed.
# LLMD_MODELS=Qwen/Qwen2.5-0.5B-Instruct
# Optional trusted InferenceObjective name injected by GoModel.
# LLMD_INFERENCE_OBJECTIVE=standard-traffic
# Derive the fairness ID from the effective user path (default: true).
# LLMD_FAIRNESS_FROM_USER_PATH=true
# SGLang (OpenAI-compatible server)
# SGLANG_API_KEY is optional; set it only if launch_server uses --api-key.
# SGLANG_API_KEY=token-abc123
# Set base URL to enable (default: http://localhost:30000/v1)
# SGLANG_BASE_URL=http://localhost:30000/v1
# LM Studio (local, OpenAI-compatible server)
# LM Studio speaks the OpenAI API (/v1/chat/completions, /v1/embeddings) and has
# NO native Ollama API. Attach it to the openai type via a suffix so it gets a
# descriptive name — here "openai-lmstudio". Do NOT use OLLAMA_BASE_URL for it:
# that routes embeddings to Ollama's native /api/embed, which LM Studio doesn't
# implement. The API key can be any non-empty value; LM Studio ignores it.
# OPENAI_LMSTUDIO_BASE_URL=http://localhost:1234/v1
# OPENAI_LMSTUDIO_API_KEY=lm-studio
# Optional: pin the served models (LM Studio only exposes loaded ones).
# OPENAI_LMSTUDIO_MODELS=text-embedding-nomic-embed-text-v1.5
# Amazon Bedrock
# Bedrock has no per-provider API key. Auth comes from the standard AWS
# credential chain — env vars (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/
# AWS_SESSION_TOKEN), AWS_PROFILE, IAM Identity Center, or instance/container
# roles. Set BEDROCK_BASE_URL to either an AWS region (e.g. "us-east-1") or a
# full https endpoint to enable the provider; otherwise the provider falls
# back to AWS_REGION / AWS_DEFAULT_REGION.
# Simplest single-secret setup: export an AWS Bedrock API key as
# AWS_BEARER_TOKEN_BEDROCK (the AWS SDK reads this name natively, so we keep
# it as-is rather than renaming to BEDROCK_API_KEY).
# AWS_BEARER_TOKEN_BEDROCK=ABSKBedrockAPIKey-...
# BEDROCK_BASE_URL=us-east-1
# Optional configured model allowlist (model IDs or inference profiles)
# BEDROCK_MODELS=anthropic.claude-3-5-haiku-20241022-v1:0,amazon.nova-lite-v1:0
# Amazon Bedrock Mantle (OpenAI-compatible)
# Use this separate provider for Responses-only models such as GPT-5.6.
# Authentication accepts a Bedrock API key or the AWS credential chain.
# BEDROCK_MANTLE_API_KEY=ABSK...
# BEDROCK_MANTLE_BASE_URL=us-east-1
# AWS_BEARER_TOKEN_BEDROCK from the Bedrock section above also works as a
# fallback bearer token. Set BASE_URL with it to enable provider discovery.
# Optional: auto (default), openai, or standard upstream API path selection.
# BEDROCK_MANTLE_API_MODE=auto
# BEDROCK_MANTLE_MODELS=openai.gpt-5.6-sol,openai.gpt-5.6-terra,openai.gpt-5.6-luna
# Kimi Code (OpenAI-compatible)
# Set the API key to enable. The base URL defaults to the Kimi Code endpoint.
# KIMICODE_API_KEY=sk-kimi-...
# KIMICODE_BASE_URL=https://api.kimi.com/coding/v1
# Optional configured model allowlist
# KIMICODE_MODELS=kimi-for-coding,bge_m3_embed