Add translation tracking - #602
Open
mcmah309 wants to merge 1 commit into
Open
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
7 issues found across 5 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="babeldoc/format/pdf/translation_config.py">
<violation number="1" location="babeldoc/format/pdf/translation_config.py:221">
P1: The executor request cannot enable translation tracking because `build_translation_config` never forwards `enable_translation_tracking` into `TranslationConfig`. Wiring the request field through, with a false default when absent, would make the opt-in reachable through the API.</violation>
<violation number="2" location="babeldoc/format/pdf/translation_config.py:381">
P2: Leaving tracking disabled still allocates paragraph/LLM tracking data and writes `translate_tracking.json` for normal translations, so the opt-in flag does not prevent its memory and disk overhead. Gating tracker creation and serialization on `enable_translation_tracking` would preserve the intended opt-in behavior.</violation>
<violation number="3" location="babeldoc/format/pdf/translation_config.py:534">
P1: The executor API never returns translation mappings even when `TranslateResult.translation_tracking` is populated, so the advertised tracking data is unavailable to API callers. Including `translation_tracking` in the payload when tracking is enabled would expose the new result data.</violation>
</file>
<file name="babeldoc/format/pdf/document_il/midend/il_translator.py">
<violation number="1" location="babeldoc/format/pdf/document_il/midend/il_translator.py:192">
P3: The new translation-tracking response is unverified for both `enable_translation_tracking=False` and `True`, including the new geometry and per-character fields; a regression could silently break API consumers. Adding focused serialization tests and documenting the opt-in test workflow would make the rollout verifiable.</violation>
</file>
<file name="babeldoc/format/pdf/document_il/midend/il_translator_llm_only.py">
<violation number="1" location="babeldoc/format/pdf/document_il/midend/il_translator_llm_only.py:182">
P3: The new LLM-only tracking path is not covered by a regression test, so the opt-in API output can silently regress without detection. A focused test could enable `enable_translation_tracking`, run a minimal LLM translation, and assert returned page/cross-page records include geometry and `layout_label`; the PR should also document how to exercise this flag.</violation>
</file>
<file name="babeldoc/format/pdf/high_level.py">
<violation number="1" location="babeldoc/format/pdf/high_level.py:1017">
P2: Normal translations now retain the entire translation-tracking object until `_do_translate_single` finishes, even when tracking is disabled, increasing peak memory usage for large PDFs and undermining the opt-in behavior. Capturing the tracker only when `enable_translation_tracking` is enabled would preserve the previous memory lifecycle.</violation>
</file>
<file name="babeldoc/format/pdf/result_merger.py">
<violation number="1" location="babeldoc/format/pdf/result_merger.py:201">
P3: Merging currently mutates and aliases the input parts' tracking data: callers retaining a part result will observe its batch IDs rewritten, and later edits to the merged tracking can modify that part. Copy each part's tracking structure before renumbering and appending it so aggregation does not change its inputs.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| metadata_extra_data: str | None = None, | ||
| term_pool_max_workers: int | None = None, | ||
| disable_same_text_fallback: bool = False, | ||
| enable_translation_tracking: bool = False, |
There was a problem hiding this comment.
P1: The executor request cannot enable translation tracking because build_translation_config never forwards enable_translation_tracking into TranslationConfig. Wiring the request field through, with a false default when absent, would make the opt-in reachable through the API.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/translation_config.py, line 221:
<comment>The executor request cannot enable translation tracking because `build_translation_config` never forwards `enable_translation_tracking` into `TranslationConfig`. Wiring the request field through, with a false default when absent, would make the opt-in reachable through the API.</comment>
<file context>
@@ -217,6 +218,7 @@ def __init__(
metadata_extra_data: str | None = None,
term_pool_max_workers: int | None = None,
disable_same_text_fallback: bool = False,
+ enable_translation_tracking: bool = False,
):
self.translator = translator
</file context>
| auto_extracted_glossary_path: Path | None | ||
| total_valid_character_count: int | None | ||
| total_valid_text_token_count: int | None | ||
| translation_tracking: dict | None |
There was a problem hiding this comment.
P1: The executor API never returns translation mappings even when TranslateResult.translation_tracking is populated, so the advertised tracking data is unavailable to API callers. Including translation_tracking in the payload when tracking is enabled would expose the new result data.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/translation_config.py, line 534:
<comment>The executor API never returns translation mappings even when `TranslateResult.translation_tracking` is populated, so the advertised tracking data is unavailable to API callers. Including `translation_tracking` in the payload when tracking is enabled would expose the new result data.</comment>
<file context>
@@ -517,6 +531,7 @@ class TranslateResult:
auto_extracted_glossary_path: Path | None
total_valid_character_count: int | None
total_valid_text_token_count: int | None
+ translation_tracking: dict | None
def __init__(
</file context>
| "cache_hit_prompt_tokens": 0, | ||
| } | ||
| self.disable_same_text_fallback = disable_same_text_fallback | ||
| self.enable_translation_tracking = enable_translation_tracking |
There was a problem hiding this comment.
P2: Leaving tracking disabled still allocates paragraph/LLM tracking data and writes translate_tracking.json for normal translations, so the opt-in flag does not prevent its memory and disk overhead. Gating tracker creation and serialization on enable_translation_tracking would preserve the intended opt-in behavior.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/translation_config.py, line 381:
<comment>Leaving tracking disabled still allocates paragraph/LLM tracking data and writes `translate_tracking.json` for normal translations, so the opt-in flag does not prevent its memory and disk overhead. Gating tracker creation and serialization on `enable_translation_tracking` would preserve the intended opt-in behavior.</comment>
<file context>
@@ -376,6 +378,7 @@ def __init__(
"cache_hit_prompt_tokens": 0,
}
self.disable_same_text_fallback = disable_same_text_fallback
+ self.enable_translation_tracking = enable_translation_tracking
if self.ocr_workaround:
</file context>
| il_translator = ILTranslator(translate_engine, translation_config) | ||
|
|
||
| il_translator.translate(docs) | ||
| translation_tracker = il_translator.translate(docs) |
There was a problem hiding this comment.
P2: Normal translations now retain the entire translation-tracking object until _do_translate_single finishes, even when tracking is disabled, increasing peak memory usage for large PDFs and undermining the opt-in behavior. Capturing the tracker only when enable_translation_tracking is enabled would preserve the previous memory lifecycle.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/high_level.py, line 1017:
<comment>Normal translations now retain the entire translation-tracking object until `_do_translate_single` finishes, even when tracking is disabled, increasing peak memory usage for large PDFs and undermining the opt-in behavior. Capturing the tracker only when `enable_translation_tracking` is enabled would preserve the previous memory lifecycle.</comment>
<file context>
@@ -994,13 +1007,14 @@ def _do_translate_single(
il_translator = ILTranslator(translate_engine, translation_config)
- il_translator.translate(docs)
+ translation_tracker = il_translator.translate(docs)
del il_translator
logger.debug(f"finish ILTranslator from {temp_pdf_path}")
</file context>
Suggested change
| translation_tracker = il_translator.translate(docs) | |
| if translation_config.enable_translation_tracking: | |
| translation_tracker = il_translator.translate(docs) | |
| else: | |
| il_translator.translate(docs) |
| return page | ||
|
|
||
| def to_json(self): | ||
| def to_dict(self) -> dict: |
There was a problem hiding this comment.
P3: The new translation-tracking response is unverified for both enable_translation_tracking=False and True, including the new geometry and per-character fields; a regression could silently break API consumers. Adding focused serialization tests and documenting the opt-in test workflow would make the rollout verifiable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/document_il/midend/il_translator.py, line 192:
<comment>The new translation-tracking response is unverified for both `enable_translation_tracking=False` and `True`, including the new geometry and per-character fields; a regression could silently break API consumers. Adding focused serialization tests and documenting the opt-in test workflow would make the rollout verifiable.</comment>
<file context>
@@ -164,7 +189,7 @@ def new_cross_column(self):
return page
- def to_json(self):
+ def to_dict(self) -> dict:
pages = []
for page in self.page:
</file context>
| def translate(self, docs: Document) -> DocumentTranslateTracker: | ||
| self.il_translator.docs = docs | ||
| tracker = DocumentTranslateTracker() | ||
| tracker = DocumentTranslateTracker( |
There was a problem hiding this comment.
P3: The new LLM-only tracking path is not covered by a regression test, so the opt-in API output can silently regress without detection. A focused test could enable enable_translation_tracking, run a minimal LLM translation, and assert returned page/cross-page records include geometry and layout_label; the PR should also document how to exercise this flag.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/document_il/midend/il_translator_llm_only.py, line 182:
<comment>The new LLM-only tracking path is not covered by a regression test, so the opt-in API output can silently regress without detection. A focused test could enable `enable_translation_tracking`, run a minimal LLM translation, and assert returned page/cross-page records include geometry and `layout_label`; the PR should also document how to exercise this flag.</comment>
<file context>
@@ -172,9 +177,11 @@ def find_title_paragraph(self, docs: Document) -> PdfParagraph | None:
+ def translate(self, docs: Document) -> DocumentTranslateTracker:
self.il_translator.docs = docs
- tracker = DocumentTranslateTracker()
+ tracker = DocumentTranslateTracker(
+ char_boxes=self.translation_config.enable_translation_tracking,
+ )
</file context>
| if batch_id not in batch_ids: | ||
| batch_ids[batch_id] = next_batch_id | ||
| next_batch_id += 1 | ||
| paragraph["multi_paragraph_id"] = batch_ids[batch_id] |
There was a problem hiding this comment.
P3: Merging currently mutates and aliases the input parts' tracking data: callers retaining a part result will observe its batch IDs rewritten, and later edits to the merged tracking can modify that part. Copy each part's tracking structure before renumbering and appending it so aggregation does not change its inputs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/result_merger.py, line 201:
<comment>Merging currently mutates and aliases the input parts' tracking data: callers retaining a part result will observe its batch IDs rewritten, and later edits to the merged tracking can modify that part. Copy each part's tracking structure before renumbering and appending it so aggregation does not change its inputs.</comment>
<file context>
@@ -170,6 +176,33 @@ def merge_results(
+ if batch_id not in batch_ids:
+ batch_ids[batch_id] = next_batch_id
+ next_batch_id += 1
+ paragraph["multi_paragraph_id"] = batch_ids[batch_id]
+ for section in TRACKING_SECTIONS:
+ merged[section].extend(tracking[section])
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Title
Adds translation tracking to the api so you can see how translations were mapped between the documents. Enabled through an opt-in config flag.
PR Type
Breaking Changes
Contributor Checklist
Testing Instructions
Screenshots (if applicable)
Additional Notes
Summary by cubic
Add translation tracking to the PDF translation pipeline. It maps source paragraphs to outputs with geometry and layout info, enabled via an opt-in flag.
New Features
enable_translation_trackinginTranslationConfigto collect tracking data.page_number,box,layout_label, placeholders, and optional per-characterchar_boxes.TranslateResult.translation_tracking.multi_paragraph_idto be unique.Migration
enable_translation_tracking=TrueinTranslationConfig.TranslateResult.translation_tracking(present only when enabled).Written for commit ac34156. Summary will update on new commits.