Skip to content

feat: map-based LLM provider config with pricing dialog #3492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JanCizmar merged 7 commits into from
Mar 12, 2026
Merged

feat: map-based LLM provider config with pricing dialog #3492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8439a43
feat: add map-based LLM provider configuration
JanCizmar Feb 20, 2026
3db281f
feat: add LLM provider pricing info dialog
JanCizmar Feb 20, 2026
bbb89dc
feat: add pricing info and description text to LLM provider dialog
JanCizmar Feb 25, 2026
40ea4aa
style: fix prettier formatting in LlmProviderPricingDialog
JanCizmar Feb 25, 2026
574930f
fix: address review feedback for LLM provider pricing
JanCizmar Feb 26, 2026
fbeb54c
fix: use explicit T components for static key extraction
JanCizmar Feb 26, 2026
48a5e82
fix: address PR review — deduplicate ProviderWithPricing type, extrac…
JanCizmar Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions backend/api/src/main/kotlin/io/tolgee/hateoas/llmProvider/LlmProviderSimpleModel.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@ open class LlmProviderSimpleModel(
var name: String,
var source: String?,
var type: LlmProviderType,
var tokenPriceInCreditsInput: Double?,
var tokenPriceInCreditsOutput: Double?,
) : RepresentationModel<LlmProviderModel>()
106 changes: 100 additions & 6 deletions ...d/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,11 @@ class LlmProperties : MachineTranslationServiceProperties {

@DocProperty(
description = """
List of LLM providers. Example:

List of LLM providers. When `provider-defaults` is also set, list entries are merged with the
matching map entry (by name). List values override map defaults only when explicitly set
(non-null for nullable fields, non-default for `type` / `maxTokens`).
`enabled` is always taken from the list entry.

``` yaml
providers:
- name: openai-gpt-4o-mini
Expand All @@ -34,9 +37,9 @@ class LlmProperties : MachineTranslationServiceProperties {
model: gpt-4o-mini
format: "json_schema"
```

or using environment variables:

```
TOLGEE_LLM_PROVIDERS_0_NAME=MySuperDuperAI
TOLGEE_LLM_PROVIDERS_0_TYPE=OPENAI
Expand All @@ -45,12 +48,45 @@ class LlmProperties : MachineTranslationServiceProperties {
TOLGEE_LLM_PROVIDERS_0_MODEL=gpt-4o-mini
TOLGEE_LLM_PROVIDERS_0_FORMAT=json_schema
```

Check [llm providers documentation](/platform/projects_and_organizations/llm-providers#self-hosted-server-configuration) for more information.
""",
)
var providers: MutableList<LlmProvider> = mutableListOf()

@DocProperty(
description = """
Map of provider defaults keyed by provider name. Use this to separate non-secret configuration
(model, prices, type) from secrets (API keys) in Kubernetes deployments.

``` yaml
provider-defaults:
gpt-5-mini:
type: OPENAI
model: gpt-5-mini
token-price-in-credits-input: 2.0
token-price-in-credits-output: 1.5
```

or using environment variables (in a ConfigMap):

```
TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_TYPE=OPENAI
TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_MODEL=gpt-5-mini
TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_TOKEN_PRICE_IN_CREDITS_INPUT=2.0
TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_TOKEN_PRICE_IN_CREDITS_OUTPUT=1.5
```
Comment on lines +71 to +78

@coderabbitai coderabbitai Bot Mar 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

Spring Boot official documentation: how are environment variable names derived from kebab-case @ConfigurationProperties names, and how are Map keys bound from environment variables?

💡 Result:

Deriving environment-variable names from canonical (kebab-case) property names

Spring Boot recommends using the canonical form for property names (lowercase kebab-case, with . for nesting). To convert a canonical property name to an environment variable name, Spring Boot documents these rules: replace . with _, remove -, then uppercase. For example, spring.main.log-startup-info becomes SPRING_MAIN_LOGSTARTUPINFO. [1]

Binding Map keys from environment variables

When binding from environment variables, Spring Boot lowercases the environment variable name before binding. This usually doesn’t matter, except for Map bindings: Map keys produced from env vars are always lowercase. Example: if you have @ConfigurationProperties("my.props") with a Map<String,String> values, then setting MY_PROPS_VALUES_KEY=value results in values containing {"key"="value"} (and if MY_PROPS_VALUES_KEY=VALUE, the key is still "key" but the value is "VALUE"). [1]

Source: Spring Boot Reference Guide → Externalized Configuration → “Binding From Environment Variables” and “Binding Maps From Environment Variables”. [1]

🏁 Script executed:

cat -n backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt | head -100

Repository: tolgee/tolgee-platform

Length of output: 4100

🏁 Script executed:

cat -n backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt | sed -n '100,180p'

Repository: tolgee/tolgee-platform

Length of output: 3757

🏁 Script executed:

cat -n backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt | sed -n '174,200p'

Repository: tolgee/tolgee-platform

Length of output: 1531

🏁 Script executed:

rg -A 5 "TOLGEE_LLM_PROVIDER_DEFAULTS" --type kt --type java

Repository: tolgee/tolgee-platform

Length of output: 92

🏁 Script executed:

rg -A 5 "TOLGEE_LLM_PROVIDER_DEFAULTS"

Repository: tolgee/tolgee-platform

Length of output: 1294

🏁 Script executed:

# Let me verify the Spring Boot relaxed binding rules by checking any tests or integration examples
fd -e properties -e yml -e yaml | xargs grep -l "provider-defaults" 2>/dev/null | head -5

Repository: tolgee/tolgee-platform

Length of output: 48

Fix the provider-defaults env-var example.

Spring Boot derives env-var names by replacing . with _, removing - entirely, and uppercasing. For the canonical property tolgee.llm.provider-defaults.gpt-5-mini.type, the correct env-var is TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINITYPE (no underscores between parts). Additionally, Map keys bound from env vars are always lowercased, so the map key would be gpt5mini, not gpt-5-mini. The documented form TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_TYPE does not follow these rules and will fail to bind to the intended map entry.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In
`@backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt`
around lines 71 - 78, Update the env-var examples in LlmProperties so they
follow Spring Boot's binding rules: use the canonical property
tolgee.llm.provider-defaults.gpt-5-mini.type and its env-var form
TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINITYPE (no dashes/underscores between parts)
and note the map key will be lowercased (gpt5mini). Replace the incorrect
examples (e.g. TOLGEE_LLM_PROVIDER_DEFAULTS_GPT_5_MINI_TYPE) with the correct
concatenated uppercase names for type/model and token price keys (e.g.
TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINITYPE,
TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINIMODEL,
TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINITOKENPRICEINCREDITSINPUT,
TOLGEE_LLM_PROVIDERDEFAULTSGPT5MINITOKENPRICEINCREDITSOUTPUT) and mention the
map key expected is "gpt5mini".


Then supply only the API key via the `providers` list (in a Secret):

```
TOLGEE_LLM_PROVIDERS_0_NAME=gpt-5-mini
TOLGEE_LLM_PROVIDERS_0_API_KEY=sk-proj-...
```
""",
)
var providerDefaults: MutableMap<String, LlmProviderDefaults> = mutableMapOf()

@DocProperty(
description = """
Named fallback mapping. When a provider is not found, Tolgee will try the fallback provider.
Expand All @@ -77,7 +113,7 @@ class LlmProperties : MachineTranslationServiceProperties {
@DocProperty(description = "User visible provider name")
override var name: String = "default",
@DocProperty(description = "Provider type, an API type")
override var type: LlmProviderType = LlmProviderType.OPENAI,
override var type: LlmProviderType = TYPE_DEFAULT,
@DocProperty(description = "Provider API Key (optional for some providers)")
override var apiKey: String? = null,
@DocProperty(description = "Provider API Url")
Expand Down Expand Up @@ -130,7 +166,65 @@ class LlmProperties : MachineTranslationServiceProperties {
}

companion object {
val TYPE_DEFAULT: LlmProviderType = LlmProviderType.OPENAI
const val MAX_TOKENS_DEFAULT: Long = 2000
}
}

class LlmProviderDefaults(
@DocProperty(description = "Enable/disable provider")
var enabled: Boolean = true,
@DocProperty(description = "Provider type, an API type")
var type: LlmProviderType = LlmProviderType.OPENAI,
@DocProperty(description = "Provider API Key (optional for some providers)")
var apiKey: String? = null,
@DocProperty(description = "Provider API Url")
var apiUrl: String? = null,
@DocProperty(description = "Provider model (optional for some providers)")
var model: String? = null,
@DocProperty(description = "Provider deployment (optional for some providers)")
var deployment: String? = null,
@DocProperty(
description = """Maximum number of tokens to generate.
`max_completion_tokens` option for OpenAI API.
`max_tokens` for Anthropic API.""",
)
var maxTokens: Long? = null,
@DocProperty(description = "ChatGPT reasoning effort")
var reasoningEffort: String? = null,
@DocProperty(description = "Set to `json_schema` if the API supports JSON Schema")
var format: String? = null,
@DocProperty(
description = "Load-balancing instruction HIGH = used for suggestions, LOW = used for batch operations",
)
var priority: LlmProviderPriority? = null,
@DocProperty(
description =
"Specify attempts timeout(s) (Example: [30, 30] - Tolgee will make two attempts, each with timeout of 30s)",
)
var attempts: List<Int>? = null,
@DocProperty(hidden = true)
var tokenPriceInCreditsInput: Double? = null,
@DocProperty(hidden = true)
var tokenPriceInCreditsOutput: Double? = null,
) {
fun toLlmProvider(name: String): LlmProvider {
return LlmProvider(
enabled = enabled,
name = name,
type = type,
apiKey = apiKey,
apiUrl = apiUrl,
model = model,
deployment = deployment,
maxTokens = maxTokens ?: LlmProvider.MAX_TOKENS_DEFAULT,
reasoningEffort = reasoningEffort,
format = format,
priority = priority,
attempts = attempts,
tokenPriceInCreditsInput = tokenPriceInCreditsInput,
tokenPriceInCreditsOutput = tokenPriceInCreditsOutput,
)
}
}
}
62 changes: 60 additions & 2 deletions backend/data/src/main/kotlin/io/tolgee/service/LlmPropertiesService.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package io.tolgee.service
import io.tolgee.api.EeSubscriptionProvider
import io.tolgee.configuration.tolgee.machineTranslation.LlmProperties
import io.tolgee.configuration.tolgee.machineTranslation.LlmProperties.LlmProvider
import io.tolgee.configuration.tolgee.machineTranslation.LlmProperties.LlmProviderDefaults
import io.tolgee.exceptions.InvalidStateException
import io.tolgee.model.enums.LlmProviderType
import org.springframework.stereotype.Service
Expand All @@ -25,9 +26,9 @@ class LlmPropertiesService(
}

fun getProviders(): List<LlmProvider> {
val result = llmProperties.providers.toMutableList()
val result = getMergedProviders().toMutableList()
if (subscriptionActive()) {
val hasTolgeeConfig = llmProperties.providers.find { it.type == LlmProviderType.TOLGEE } != null
val hasTolgeeConfig = result.find { it.type == LlmProviderType.TOLGEE } != null
if (!hasTolgeeConfig) {
result.add(
LlmProvider(
Expand All @@ -40,4 +41,61 @@ class LlmPropertiesService(
}
return result.filter { it.enabled }
}

fun getMergedProviders(): List<LlmProvider> {
val defaults = llmProperties.providerDefaults
if (defaults.isEmpty()) {
return llmProperties.providers.toList()
}

val result = mutableListOf<LlmProvider>()
val matchedDefaultNames = mutableSetOf<String>()

for (listEntry in llmProperties.providers) {
val mapEntry = defaults[listEntry.name]
if (mapEntry != null) {
matchedDefaultNames.add(listEntry.name)
result.add(mergeProviderWithDefaults(mapEntry, listEntry))
} else {
result.add(listEntry)
}
}

// Add map-only entries not matched by any list entry
for ((name, mapEntry) in defaults) {
if (name !in matchedDefaultNames) {
result.add(mapEntry.toLlmProvider(name))
}
}

return result
}

private fun mergeProviderWithDefaults(
defaults: LlmProviderDefaults,
listEntry: LlmProvider,
): LlmProvider {
val base = defaults.toLlmProvider(listEntry.name)
// enabled is always taken from the list entry
base.enabled = listEntry.enabled
// Nullable fields: override only if list value is non-null
listEntry.apiKey?.let { base.apiKey = it }
listEntry.apiUrl?.let { base.apiUrl = it }
listEntry.model?.let { base.model = it }
listEntry.deployment?.let { base.deployment = it }
listEntry.reasoningEffort?.let { base.reasoningEffort = it }
listEntry.format?.let { base.format = it }
listEntry.priority?.let { base.priority = it }
listEntry.attempts?.let { base.attempts = it }
listEntry.tokenPriceInCreditsInput?.let { base.tokenPriceInCreditsInput = it }
listEntry.tokenPriceInCreditsOutput?.let { base.tokenPriceInCreditsOutput = it }
// Non-nullable fields: override only if different from Spring default
if (listEntry.type != LlmProvider.TYPE_DEFAULT) {
base.type = listEntry.type
}
if (listEntry.maxTokens != LlmProvider.MAX_TOKENS_DEFAULT) {
base.maxTokens = listEntry.maxTokens
}
return base
}
}
Loading
Loading