Skip to content

Commit a434007

Browse files
authored
feat: server default LLM model selectable as "default" in prompts (#3838)
## What Introduces a **server default model** concept for AI translation: - New config property `tolgee.llm.default-provider` naming the provider that serves as the server default. When unset, the first enabled provider is used (previous behavior preserved). - The provider name `default` is now a reserved alias. Prompts that store it resolve to the server default **at call time** (interactive MT suggestions, batch, playground), so they automatically follow default-model upgrades without any prompt edits. Resolution goes through the existing fallback chain, and a real provider actually named `default` keeps precedence, so existing self-hosted configs relying on the Spring-default provider name are unaffected. - The default prompt now carries the `default` alias instead of pinning the first provider's name, and API responses keep the alias unresolved so the selection stays stable in the UI. - `GET .../llm-providers/all-available` prepends a synthetic `default` entry with a new `resolvesToName` field, and the AI playground model select renders it as **`default (<model>)`** (new translation key `ai_prompt_provider_server_default`, uploaded to Tolgee with screenshot). ## Why The production default model was pinned positionally (first entry of the providers list) and users wanting "the best model" had to edit their prompts whenever a new model came out. With this change, the deployment automation can repoint `tolgee.llm.default-provider` and every prompt on the `default` alias follows. Companion PRs in the deployment repo add the key to values.yaml and teach the weekly model automation to maintain it. ## Testing - Unit tests for `getDefaultProviderName` (configured/renamed-via-fallbacks/dangling/unset/disabled/empty). - Resolver unit tests: alias resolves to the configured default, real `default` provider wins, throws with no providers. - Controller tests: default prompt returns the alias, playground run works with the alias, saved prompts keep the alias unresolved, `all-available` contains/suppresses the synthetic entry correctly. - Verified end-to-end in a dev instance: select shows `default (gpt-5-mini)`, preselected for new prompts, playground run works. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable default provider selection for AI/LLM features. * Added fallback handling when the selected provider is unavailable, renamed, disabled, or unset. * The provider list now shows a clear “default (provider name)” entry. * Prompts can use the default provider alias while retaining that alias in saved prompt data. * **Documentation** * Updated the PostgreSQL autostart migration documentation link. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 81b173e commit a434007

14 files changed

Lines changed: 312 additions & 10 deletions

File tree

backend/api/src/main/kotlin/io/tolgee/hateoas/llmProvider/LlmProviderSimpleModel.kt

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
package io.tolgee.hateoas.llmProvider
22

3+
import io.swagger.v3.oas.annotations.media.Schema
34
import io.tolgee.model.enums.LlmProviderType
45
import org.springframework.hateoas.RepresentationModel
56
import org.springframework.hateoas.server.core.Relation
@@ -11,4 +12,11 @@ open class LlmProviderSimpleModel(
1112
var type: LlmProviderType,
1213
var tokenPriceInCreditsInput: Double?,
1314
var tokenPriceInCreditsOutput: Double?,
15+
@field:Schema(
16+
description =
17+
"Name of the concrete provider the server default (\"default\" provider) currently resolves to. " +
18+
"It is only set on the synthetic \"default\" entry and is always null for concrete providers. " +
19+
"Clients can rely on non-null resolvesToName to identify the server-default entry.",
20+
)
21+
var resolvesToName: String? = null,
1422
) : RepresentationModel<LlmProviderModel>()

backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/PostgresAutostartProperties.kt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ class PostgresAutostartProperties {
2626
"`EMBEDDED` is deprecated. Tolgee v4 removes the bundled PostgreSQL from the " +
2727
"`tolgee/tolgee` image, so setups using it have to move to an external database " +
2828
"before upgrading. See " +
29-
"[Migrate from the bundled database](/self_hosting/running_with_docker" +
29+
"[Migrate from the bundled database](/platform/self_hosting/running_with_docker" +
3030
"#migrate-from-the-bundled-database).",
3131
)
3232
var mode: PostgresAutostartMode = PostgresAutostartMode.DOCKER

backend/data/src/main/kotlin/io/tolgee/configuration/tolgee/machineTranslation/LlmProperties.kt

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,15 @@ class LlmProperties : MachineTranslationServiceProperties {
5454
)
5555
var providers: MutableList<LlmProvider> = mutableListOf()
5656

57+
@DocProperty(
58+
description = """
59+
Name of the provider used when a prompt selects the `default` provider. The name must match
60+
one of the configured providers (the `fallbacks` mapping is applied if the provider was renamed).
61+
When unset, the first enabled provider is used.
62+
""",
63+
)
64+
var defaultProvider: String? = null
65+
5766
@DocProperty(
5867
description = """
5968
Map of provider defaults keyed by provider name. Use this to separate non-secret configuration

backend/data/src/main/kotlin/io/tolgee/service/LlmPropertiesService.kt

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,29 @@ class LlmPropertiesService(
2525
return llmProperties.fallbacks[providerName]
2626
}
2727

28+
fun getDefaultProviderName(): String? {
29+
val providers = getProviders()
30+
val configured = resolveConfiguredDefault(providers)
31+
return configured ?: providers.firstOrNull()?.name
32+
}
33+
34+
private fun resolveConfiguredDefault(providers: List<LlmProvider>): String? {
35+
var current = llmProperties.defaultProvider ?: return null
36+
val tried = mutableSetOf<String>()
37+
repeat(MAX_FALLBACK_DEPTH) {
38+
if (providers.any { it.name == current }) {
39+
return current
40+
}
41+
tried.add(current)
42+
val fallback = llmProperties.fallbacks[current]
43+
if (fallback == null || fallback in tried) {
44+
return null
45+
}
46+
current = fallback
47+
}
48+
return null
49+
}
50+
2851
fun getProviders(): List<LlmProvider> {
2952
val result = getMergedProviders().toMutableList()
3053
if (subscriptionActive()) {
@@ -98,4 +121,9 @@ class LlmPropertiesService(
98121
}
99122
return base
100123
}
124+
125+
companion object {
126+
const val DEFAULT_PROVIDER_ALIAS = "default"
127+
private const val MAX_FALLBACK_DEPTH = 10
128+
}
101129
}
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
package io.tolgee.unit
2+
3+
import io.tolgee.configuration.tolgee.machineTranslation.LlmProperties
4+
import io.tolgee.configuration.tolgee.machineTranslation.LlmProperties.LlmProvider
5+
import io.tolgee.service.LlmPropertiesService
6+
import io.tolgee.testing.assert
7+
import org.junit.jupiter.api.Test
8+
9+
class LlmPropertiesServiceDefaultProviderTest {
10+
private fun createService(props: LlmProperties): LlmPropertiesService {
11+
return LlmPropertiesService(props, null)
12+
}
13+
14+
private fun props(
15+
vararg providerNames: String,
16+
defaultProvider: String? = null,
17+
fallbacks: Map<String, String> = emptyMap(),
18+
disabled: Set<String> = emptySet(),
19+
): LlmProperties {
20+
val props = LlmProperties()
21+
props.providers =
22+
providerNames
23+
.map { LlmProvider(name = it, enabled = it !in disabled) }
24+
.toMutableList()
25+
props.defaultProvider = defaultProvider
26+
props.fallbacks = fallbacks.toMutableMap()
27+
return props
28+
}
29+
30+
@Test
31+
fun `returns configured default when it exists`() {
32+
val service = createService(props("gpt-4o", "claude", defaultProvider = "claude"))
33+
service.getDefaultProviderName().assert.isEqualTo("claude")
34+
}
35+
36+
@Test
37+
fun `resolves renamed default through fallback chain`() {
38+
val service =
39+
createService(
40+
props(
41+
"gpt-6",
42+
defaultProvider = "gpt-4o",
43+
fallbacks = mapOf("gpt-4o" to "gpt-5", "gpt-5" to "gpt-6"),
44+
),
45+
)
46+
service.getDefaultProviderName().assert.isEqualTo("gpt-6")
47+
}
48+
49+
@Test
50+
fun `falls back to first provider when configured default is dangling`() {
51+
val service = createService(props("gpt-4o", "claude", defaultProvider = "removed-model"))
52+
service.getDefaultProviderName().assert.isEqualTo("gpt-4o")
53+
}
54+
55+
@Test
56+
fun `falls back to first provider when fallback chain is circular`() {
57+
val service =
58+
createService(
59+
props(
60+
"gpt-4o",
61+
defaultProvider = "old-a",
62+
fallbacks = mapOf("old-a" to "old-b", "old-b" to "old-a"),
63+
),
64+
)
65+
service.getDefaultProviderName().assert.isEqualTo("gpt-4o")
66+
}
67+
68+
@Test
69+
fun `returns first provider when unset`() {
70+
val service = createService(props("gpt-4o", "claude"))
71+
service.getDefaultProviderName().assert.isEqualTo("gpt-4o")
72+
}
73+
74+
@Test
75+
fun `returns null when no providers exist`() {
76+
val service = createService(props())
77+
service.getDefaultProviderName().assert.isNull()
78+
}
79+
80+
@Test
81+
fun `ignores disabled configured default`() {
82+
val service =
83+
createService(props("gpt-4o", "claude", defaultProvider = "claude", disabled = setOf("claude")))
84+
service.getDefaultProviderName().assert.isEqualTo("gpt-4o")
85+
}
86+
}

ee/backend/app/src/main/kotlin/io/tolgee/ee/api/v2/controllers/LlmProviderController.kt

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import io.tolgee.model.enums.OrganizationRoleType
1313
import io.tolgee.openApiDocs.OpenApiOrderExtension
1414
import io.tolgee.security.authorization.RequiresOrganizationRole
1515
import io.tolgee.security.authorization.UseDefaultPermissions
16+
import io.tolgee.service.LlmPropertiesService
1617
import jakarta.validation.Valid
1718
import org.springframework.hateoas.CollectionModel
1819
import org.springframework.web.bind.annotation.CrossOrigin
@@ -34,6 +35,7 @@ class LlmProviderController(
3435
private val providerService: LlmProviderService,
3536
private val providerModelAssembler: LlmProviderModelAssembler,
3637
private val providerSimpleModelAssembler: LlmProviderSimpleModelAssembler,
38+
private val llmPropertiesService: LlmPropertiesService,
3739
) {
3840
@GetMapping("all-available")
3941
@UseDefaultPermissions
@@ -60,7 +62,23 @@ class LlmProviderController(
6062
existing.add(it.name)
6163
}
6264
}
63-
return providerSimpleModelAssembler.toCollectionModel(result)
65+
val models = providerSimpleModelAssembler.toCollectionModel(result)
66+
val defaultModel = getServerDefaultModel(existing) ?: return models
67+
return CollectionModel.of(listOf(defaultModel) + models)
68+
}
69+
70+
private fun getServerDefaultModel(existingNames: Set<String>): LlmProviderSimpleModel? {
71+
if (LlmPropertiesService.DEFAULT_PROVIDER_ALIAS in existingNames) return null
72+
val defaultName = llmPropertiesService.getDefaultProviderName() ?: return null
73+
val provider = providerService.getAllServerProviders().find { it.name == defaultName } ?: return null
74+
return LlmProviderSimpleModel(
75+
name = LlmPropertiesService.DEFAULT_PROVIDER_ALIAS,
76+
source = "server",
77+
type = provider.type,
78+
tokenPriceInCreditsInput = provider.tokenPriceInCreditsInput,
79+
tokenPriceInCreditsOutput = provider.tokenPriceInCreditsOutput,
80+
resolvesToName = defaultName,
81+
)
6482
}
6583

6684
@GetMapping("")

ee/backend/app/src/main/kotlin/io/tolgee/ee/service/LlmProviderResolver.kt

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,22 @@ class LlmProviderResolver(
2323
return current
2424
}
2525
tried.add(current)
26-
val fallback = llmPropertiesService.getFallbackProviderName(current)
27-
if (fallback == null || fallback in tried) {
26+
val next = getNextName(current)
27+
if (next == null || next in tried) {
2828
throw LlmProviderNotFoundException(provider)
2929
}
30-
current = fallback
30+
current = next
3131
}
3232
throw LlmProviderNotFoundException(provider)
3333
}
3434

35+
private fun getNextName(current: String): String? {
36+
if (current == LlmPropertiesService.DEFAULT_PROVIDER_ALIAS) {
37+
return llmPropertiesService.getDefaultProviderName()
38+
}
39+
return llmPropertiesService.getFallbackProviderName(current)
40+
}
41+
3542
private fun providerExists(
3643
organizationId: Long,
3744
name: String,

ee/backend/app/src/main/kotlin/io/tolgee/ee/service/prompt/DefaultPromptHelper.kt

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,7 @@ import io.tolgee.service.LlmPropertiesService
55
import org.springframework.stereotype.Component
66

77
@Component
8-
class DefaultPromptHelper(
9-
private val llmPropertiesService: LlmPropertiesService,
10-
) {
8+
class DefaultPromptHelper {
119
fun getDefaultPrompt(): PromptDto {
1210
return PromptDto(
1311
name = "",
@@ -41,7 +39,7 @@ class DefaultPromptHelper(
4139
4240
{{fragment.translateJson}}
4341
""".trimIndent(),
44-
providerName = llmPropertiesService.getProviders().getOrNull(0)?.name ?: "default",
42+
providerName = LlmPropertiesService.DEFAULT_PROVIDER_ALIAS,
4543
)
4644
}
4745
}

ee/backend/app/src/main/kotlin/io/tolgee/ee/service/prompt/PromptServiceEeImpl.kt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import io.tolgee.model.Prompt
2323
import io.tolgee.model.enums.BasicPromptOption
2424
import io.tolgee.model.enums.LlmProviderPriority
2525
import io.tolgee.repository.PromptRepository
26+
import io.tolgee.service.LlmPropertiesService
2627
import io.tolgee.service.PromptService
2728
import io.tolgee.service.key.KeyService
2829
import io.tolgee.service.machineTranslation.MtServiceConfigService
@@ -295,9 +296,14 @@ class PromptServiceEeImpl(
295296
*
296297
* The entity is detached before mutation to prevent OSIV from flushing the resolved name
297298
* back to the DB — the stored value must remain the original.
299+
*
300+
* The "default" alias is intentionally NOT resolved — it must stay stable in API responses
301+
* so the frontend keeps showing the alias option instead of the concrete provider it
302+
* currently points to.
298303
*/
299304
private fun withResolvedProviderName(prompt: Prompt): Prompt {
300305
if (prompt.providerName.isEmpty()) return prompt
306+
if (prompt.providerName == LlmPropertiesService.DEFAULT_PROVIDER_ALIAS) return prompt
301307
// Access lazy association while still managed
302308
val organizationId = prompt.project.organizationOwner.id
303309
val resolvedName =

ee/backend/tests/src/test/kotlin/io/tolgee/ee/api/v2/controllers/LlmProviderControllerTest.kt

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ class LlmProviderControllerTest : AuthorizedControllerTest() {
4949
fun resetSsrfProperties() {
5050
internalProperties.disableUrlSsrfProtection = true
5151
webhookProperties.allowLocalAddresses = false
52+
llmProperties.defaultProvider = null
5253
}
5354

5455
@Test
@@ -63,6 +64,63 @@ class LlmProviderControllerTest : AuthorizedControllerTest() {
6364
}
6465
}
6566

67+
@Test
68+
fun `all-available contains synthetic default entry resolving to configured default`() {
69+
llmProperties.providers =
70+
mutableListOf(
71+
LlmProperties.LlmProvider(
72+
name = "gpt-6",
73+
type = LlmProviderType.OPENAI,
74+
apiUrl = "http://test.com",
75+
),
76+
LlmProperties.LlmProvider(
77+
name = "claude",
78+
type = LlmProviderType.ANTHROPIC,
79+
apiUrl = "http://test.com",
80+
),
81+
)
82+
llmProperties.defaultProvider = "claude"
83+
performAuthGet(
84+
"/v2/organizations/${testData.organization.self.id}/llm-providers/all-available",
85+
).andIsOk.andAssertThatJson {
86+
node("_embedded.providers").isArray.hasSize(4)
87+
node("_embedded.providers[0].name").isEqualTo("default")
88+
node("_embedded.providers[0].source").isEqualTo("server")
89+
node("_embedded.providers[0].resolvesToName").isEqualTo("claude")
90+
node("_embedded.providers[0].type").isEqualTo("ANTHROPIC")
91+
}
92+
}
93+
94+
@Test
95+
fun `synthetic default entry resolves to first provider when not configured`() {
96+
llmProperties.providers =
97+
mutableListOf(
98+
LlmProperties.LlmProvider(
99+
name = "gpt-6",
100+
type = LlmProviderType.OPENAI,
101+
apiUrl = "http://test.com",
102+
),
103+
)
104+
performAuthGet(
105+
"/v2/organizations/${testData.organization.self.id}/llm-providers/all-available",
106+
).andIsOk.andAssertThatJson {
107+
node("_embedded.providers[0].name").isEqualTo("default")
108+
node("_embedded.providers[0].resolvesToName").isEqualTo("gpt-6")
109+
}
110+
}
111+
112+
@Test
113+
fun `real provider named default suppresses the synthetic entry`() {
114+
// the setup's server provider keeps the Spring-default name "default"
115+
performAuthGet(
116+
"/v2/organizations/${testData.organization.self.id}/llm-providers/all-available",
117+
).andIsOk.andAssertThatJson {
118+
node("_embedded.providers").isArray.hasSize(2)
119+
node("_embedded.providers[1].name").isEqualTo("default")
120+
node("_embedded.providers[1].resolvesToName").isEqualTo(null)
121+
}
122+
}
123+
66124
@Test
67125
fun `prevents viewing other organization providers`() {
68126
this.userAccount = testData.projectReviewer.self

0 commit comments

Comments
 (0)