Fix: make auto book-language actually steer insight output#76
Merged
Conversation
The v6 language directive was a single weak line referencing an ISO *code*
(`Respond in the language identified by ISO 639-1 code "it"`) buried at the
end of an all-English prompt. Weak/structured-output models ignored it and
wrote English anyway — so "auto follows the book's language" didn't visibly
work.
Strengthen it (single-pass, no extra model call):
- Name the language ("Italian (ISO 639-1: it)"), not just the code.
- Lead the prompt with the directive instead of burying it; repeat a short
reminder at the end.
- Scope it explicitly: write the PROSE fields (intro, analysis, theme_analysis
text, craft_notes, distinctive_take, discussion_prompts, content_warnings,
author bio) in the target language, but keep the controlled fields exactly
as-is — `themes` (controlled English tags), `confidence`, and proper names
(titles, author/series names). This prevents the model from translating the
controlled vocabulary and breaking validation.
Bump PROMPT_VERSION 6 -> 7 (materially changes output; cache-bust). Tests
updated to assert the named-language, field-scoped directive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
You reported that auto book-language wasn't working — insights still came out in English for non-English books.
Root cause: the book language does reach the server (verified end-to-end for library EPUBs), but the v6 directive was too weak — a single line referencing an ISO code, buried at the end of an all-English prompt:
Weak / structured-output models ignore that and default to English.
Fix (single-pass, no extra model call)
Italian (ISO 639-1: it), not just"it".themes(controlled English tags),confidence, and proper names (titles, author/series names). This is critical — a naive "write everything in X" would translate the controlledthemesvocabulary and break validation.PROMPT_VERSION6 → 7 (cache-bust).Expected reliability
~70–85% on a weak self-hosted model (the test default hints at a Llama-class backend), higher on a stronger model. If your deployment's model still won't comply after this, the follow-up is a conditional prose-only translation-repair pass (only fires when the output is detectably not in the target language) — deliberately not included here to avoid a dependency + an extra model call against the daily budget unless it's actually needed.
Tests
353 server unit tests pass (new coverage: named-language directive, field-scoping that protects controlled fields, directive-leads-prompt) + AI integration suite;
ruffclean. Pure prompt output also verified directly.Not covered (separate, smaller gaps)
OPDS catalog books and the no-OPF fallback send no
languageat all, so auto can't resolve for them — pre-existing, tracked separately.