Skip to content

fix(translate): parallelize segments within a page - #378

Merged
youyongsong merged 1 commit into
mainfrom
feat/translate-page-parallel
Nov 15, 2026
Merged

youyongsong merged 1 commit into
mainfrom
feat/translate-page-parallel

Conversation

@youyongsong

@youyongsong youyongsong commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Translate independent segments of the same page concurrently with a bounded worker pool. Long pages previously waited for each segment to finish before starting the next one. The completed page is still assembled in source order.

In one measurement of the same 16-segment page, elapsed time changed from 1:55:13 to 0:45:26. Both runs passed the existing translation checks. This was one baseline/branch pair; gateway load and retry counts differed between runs, so the result does not establish a general speedup for other pages or entire sites.

Behavior and configuration

  • Reuse the existing document concurrency setting as the worker limit for segments within each page. Existing sites require no configuration changes.
  • Freeze verified cached segments before scheduling new translation work.
  • Apply the same worker limit to segments returned by the assembled-page checks.
  • Preserve retry budgets, quality checks, repair behavior, and source-index assembly order.
  • Stop scheduling new work after a worker error and wait for work already started before propagating the error.
  • Keep all model requests behind the existing shared limiter, including translation, judge, repair, and diagnostic requests. The configured global concurrency and requests-per-minute limits still apply.

Validation

The feature patch is unchanged after rebasing onto the current main branch: git range-diff reports an identical patch and both commits have the same stable patch ID.

On the rebased commit:

  • yarn install --immutable passed.
  • yarn build passed.
  • yarn lint passed with 0 errors and 3 warnings in unchanged theme files.
  • yarn test passed: 72 test files and 820 tests, including 36 segment-pipeline tests and 12 global-limiter tests.
  • yarn typecov passed at 100% (42,152 / 42,152).
  • yarn docs:build passed.
  • yarn docs:export exited before PDF generation because the Playwright browser executable is absent from the isolated cache. Local PDF export remains unverified.

Unit coverage includes the segment concurrency limit, out-of-order completion, worker failure handling, and reuse of verified cached segments. The patch changeset resolves to version 2.11.3 through the usual package release.

@changeset-bot

changeset-bot Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f76888b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@alauda/doom Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

yarn add https://pkg.pr.new/@alauda/doom@378.tgz
yarn add https://pkg.pr.new/@alauda/doom-export@378.tgz

commit: f76888b

@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.77419% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 69.69%. Comparing base (5afba0e) to head (f76888b).

Files with missing lines Patch % Lines
packages/doom/src/cli/translate-pipeline.ts 96.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #378      +/-   ##
==========================================
+ Coverage   69.45%   69.69%   +0.24%     
==========================================
  Files         104      104              
  Lines        4000     4029      +29     
  Branches     1193     1197       +4     
==========================================
+ Hits         2778     2808      +30     
+ Misses        973      972       -1     
  Partials      249      249              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@youyongsong
youyongsong force-pushed the feat/translate-page-parallel branch from 0ddf440 to f76888b Compare November 15, 2026 16:06
@youyongsong
youyongsong merged commit 8b541eb into main Nov 15, 2026
16 checks passed
@youyongsong
youyongsong deleted the feat/translate-page-parallel branch November 15, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant