Commit d8343e5
authored
feat(ai): multi-language + force regeneration in generate modal, and per-article AI overview board (#2804)
* docs(spec): 单文章 AI 汇总看板设计
* feat(ai): add force regeneration semantics to summary/insights/translation tasks
Thread force through the summary/insights/translation task payloads and
generation pipelines, mirroring the existing TTS force behavior: dedup keys
split force vs incremental so a forced task cannot be swallowed by an
in-flight regular one, and AiInFlightService.runWithStream gains
bypassResultCache to skip (and evict) a cached result on force. Translation
additionally drops block-level incremental reuse on force while still
reading the existing row for its other bookkeeping uses.
* fix(ai): propagate force into the PartialFailed translation retry payload
buildTranslationRetryTask rebuilt retryPayload without the original task's
force flag, so a force task that partially failed would silently retry as
incremental and hit the in-flight result cache instead of regenerating.
* feat(admin): support multi-language input and force regen in AI generate modal
parseLangInput normalizes comma-separated (incl. full-width comma) lang
input into a deduped, ordered list. GeneratePromptModal now renders live
lang chips with a count, caps input at 8 languages, and adds a "force
regenerate" checkbox shown across all four generate flows. Call sites
are updated in a follow-up task.
* fix(admin): only gate too-many-langs on the langs prompt branch
tooMany was computed unconditionally from the parsed lang input, but langs
are only actually submitted when promptForLang is true. A caller passing
defaultLangs with >8 entries alongside promptForLang: false silently
disabled the submit button with no visible error.
* chore: revert accidental page.tsx rename picked up by shared-worktree commit
An unrelated in-progress rename (apps/admin/src/views/(intelligence)/ai/page.tsx
-> ai/overview/page.tsx) from a concurrent session was staged in the shared
index and got swept into the prior commit. This restores the tracked path to
its pre-existing location without touching any working-tree file contents,
so that session's uncommitted work is unaffected.
* feat(admin): wire multi-lang + force into AI generate call sites
Connects GeneratePromptModal's {langs, force} result to all four AI
generate flows (summary, translation, tts, insights) plus the quick
actions menu and the article detail generate button. Adds
useAiDefaultLangs to prefill the modal from the AI config's
summary/translation target-language settings. createSummaryTask now
takes targetLanguages (array) instead of a single lang, matching the
backend DTO; createInsightsTask and createTranslationTask gain
force. Inline retranslate/regenerate row actions always pass
force: true.
* fix(admin): add missing useAiGenerateTask hook referenced by c7b038c
* feat(admin): per-article AI overview board
Adds a reverse view of the AI surfaces: given one article, which assets
exist, what is missing, and what the generation has cost.
Server: GET /ai/overview/grouped lists every article (posts, notes,
pages) newest first with a compact per-capability language projection;
GET /ai/overview/article/:id returns the assets, a per-resource-type
cost roll-up summed across every generation ever recorded, and the AI
tasks currently in flight for that article.
Admin: /ai/overview holds a coverage matrix of capability x language.
An empty cell dispatches the generation task, a filled one scrolls to
its asset row, and a queued one shows a spinner so a second click
cannot duplicate the work. Languages beyond the configured targets can
be added as ad-hoc columns.
* fix(ai): close inflight bypass race window and cap targetLanguages at 8
runWithStream's bypassResultCache path deleted resultKey before
attempting the lock, opening a window where an unrelated concurrent
follower for the same key could observe resultKey, errorKey, and
lockKey all empty and throw. Defer the delete until this instance
actually holds lockKey, and skip reading the cached result entirely
on bypass instead of racing a delete against the read.
Also give summary/translation task DTOs the same targetLanguages cap
TTS already enforces at runtime, sourced from one shared
MAX_LANGS_PER_TASK constant instead of a second literal.
* refactor(admin): rename runWithLangPrompt to runWithGeneratePrompt
The insights quick-action passes promptForLang: false yet calls the
same runWithLangPrompt helper, discarding its first arg as _langs.
Rename only — the helper drives every generate-prompt flow, not just
the language-prompting ones.
* fix(ai): fold force into the summary/insights/translation in-flight key
buildSummaryKey/buildInsightsKey/buildTranslationKey hashed only
articleId/lang/content, so a force request racing an in-flight plain
request for the same key lost the lock, became a follower, and
silently replayed the plain leader's incremental output — force
appeared to succeed but never regenerated anything. Fold force into
each hash so force and plain requests never share a lock/stream/result
key; repeated force requests still converge on the same key.
* docs: add admin AI generate modal multilang design spec
Records the design for the multi-language generate prompt modal and
core force-regeneration semantics, updated with the actual in-flight
key behavior (force and plain requests now use independent Redis
lock/stream/result keys instead of the originally-accepted "force
follows plain" degradation).
* feat(admin): active task list, language add control and overview sections
Extends the per-article AI overview board with an active-task list, an add-language control and a grouped overview section; core side gains active-task derivation for the overview payload.
* fix(admin): keep the AI overview polling until a generation actually settles
Clicking a coverage cell left a spinner that never resolved. Two causes,
both found by driving the board in a browser against a dev server whose
AI feature was off, so every task failed within a millisecond:
- refetchInterval is suspended while the window is unfocused, so the one
fetch that caught the task mid-queue was also the last one. The board
then displayed that frozen snapshot forever. refetchIntervalInBackground
keeps a progress view honest when the user switches away.
- An optimistic pending entry bridged the gap before the queue registers a
task, retired by conditions that did not always hold. It could outlive
every one of them and become a permanent phantom. The server now reports
live tasks and recent failures alike, so the client no longer invents
state; a dispatch grace window keeps polling alive until the queue has
something to say.
Polling now runs while anything is live or within 20s of a dispatch, and
stops once the queue is quiet.
* fix(ai): restore single in-flight lock and normalize target language codes
Folding `force` into the summary/insights/translation in-flight key hash
(8433347) fixed force silently joining a plain leader, but split the
mutex too: a force and a plain request for the same content now each ran
the model and upserted the same row, with whichever finished last
clobbering the other and doubling the bill.
Revert the key change and arbitrate on lock ownership instead. The lock
value now encodes the holder's mode (`force:`/`plain:`). A force request
that loses the race to a plain leader polls until the lock frees up (or
lockTtlSec elapses, degrading to a follower) instead of racing it with a
second writer; a lock already held by another force is joined immediately,
since two force runs converging on one leader is the desired outcome.
Plain-request behavior on a lost race is unchanged.
Also fold resolved target languages through parseLanguageCode before
dedup in the summary and translation task handlers, matching what TTS
already does — zh-CN and zh no longer produce two separate generations
for the same input.
* fix(admin): fold region-suffixed language codes in AI generate modal input
parseLangInput treated zh-CN and zh (or en_US and EN) as distinct
languages, so the multilang chips could lie about how many generations
would actually run. Normalize underscores to hyphens and drop a 2-letter
primary tag's region suffix before dedup — matching the backend's
authoritative parseLanguageCode without duplicating its alias table here.
* fix(ai): re-check lock holder each poll while a force request waits
waitForForceLock only inspected the lock holder once, when the initial
NX attempt failed. If a plain leader released the lock mid-wait and a
different force request won it via the ordinary leader path, this
instance kept polling blind until lockTtlSec instead of noticing the
lock was now force-held and joining immediately.
Re-check the holder after each failed retry so a mid-wait handoff to
another force is picked up right away, saving a redundant model call
and up to lockTtlSec of unnecessary waiting. Behavior when the holder
stays plain (or the lock is released outright) is unchanged.
* fix(ai): clear streamKey/errorKey when a force leader acquires the lock
Acquiring the lock only cleared resultKey before starting a fresh run.
streamKey (done frames live 600s) and errorKey (30s) from a previous run
on the same key survived, so a follower or converging force joining this
leader reads the stream from '0-0' and can hit the old `done`/`error`
entry first — resolving to the previous run's result or throwing a stale
error instead of waiting for the new one.
Delete all three together, still gated behind the lock so a concurrent
plain follower never observes them all empty at once.
* fix(ai): normalize target langs without clobbering unrecognized tokens
Two related language-normalization bugs, fixed together since they touch
the same call sites.
parseLanguageCode's fallback for anything it doesn't recognize is
DEFAULT_SUMMARY_LANG ('zh'). Folding summary/translation target languages
through it meant a free-typed token like "english" (the admin generate
modal takes arbitrary input) collapsed onto 'zh' and silently overwrote
an actual zh row instead of just being its own odd entry. Add
normalizeTargetLang (ai-language.util.ts): known codes/aliases still fold
via normalizeLanguageCode, but anything unrecognized passes through as
trim+lowercase instead of defaulting. Use it in both the summary task
handler and executeTranslationTask.
Since executeTranslationTask now generates against normalized language
codes, buildTranslationRetryTask's PartialFailed diff broke: it compared
raw payload.targetLanguages (e.g. 'zh-CN') against already-normalized
result.translations[].lang ('zh'), so a successful zh-CN run never
matched and got retried (with force inherited, at extra cost). Normalize
payload.targetLanguages the same way before diffing.
* fix(ai): reject blank target-language tokens instead of coining ''
normalizeTargetLang trimmed a blank/whitespace-only token and then, since
normalizeLanguageCode returns undefined for it, fell back to the trimmed
(still empty) string — so an empty target language silently became a
language named ''. The public task DTOs don't reject it either
(CreateSummaryTaskSchema / CreateTranslationTaskSchema only checked
z.string(), no non-empty constraint), so any direct API caller (not just
the admin modal, which already filters client-side) could push
targetLanguages: [''] through and generate against it.
Two-sided fix: normalizeTargetLang now returns undefined for a blank
token instead of '', and both call sites (summary handler, executeTranslationTask)
filter it out — dropping it rather than defaulting it to DEFAULT_SUMMARY_LANG,
which would reintroduce the same silent-overwrite problem the unrecognized-token
fix just closed. The DTOs add `.trim().min(1)` per element so a blank
entry 400s at the boundary instead of reaching the task handler at all.
* fix(ai): normalize target langs before hashing the summary/translation dedup key
computeAITaskDedupKey canonicalized languages for Tts (parseLanguageCode)
but not for Summary/Translation, which just sorted+joined the raw
targetLanguages. Now that generation itself normalizes (zh-CN and zh
produce the same result), two requests differing only in region suffix
enqueue as two distinct tasks at the queue layer — the loser runs for
nothing.
Add canonicalTargetLangs, using the same normalizeTargetLang the
handlers generate against (not parseLanguageCode — that would be a third
normalization scheme for the same data), and use it for the Summary and
Translation branches. Tts keeps its existing parseLanguageCode-based
canonicalization unchanged.
* fix(ai): give each in-flight leader run its own stream key
A forced regeneration used to delete the shared stream the moment it took
the lock, so a follower still draining the finished run lost its tail and
spliced the new run's tokens into the same response. Streams are now keyed
by run (`:stream:<runId>`, resolved from the lock value), so a completed
generation stays readable until its TTL retires it and force only clears
the result/error cache. Run-scoped keys also retire the stale-frame hazard
the delete existed for.
Also settles the follower result promise a lock-race test left dangling:
its idle timeout rejected ~1s later and failed the whole shard.
* fix(ai): canonicalize languages and carry force/targets through the overview board
- coverage compares canonicalized configured targets, so a `zh-CN` setting
no longer reports a gap the stored `zh` row can never close
- the article's active tasks include batch children and are looked up by
refId, so a busy queue can no longer push a live task off the first page
- insights translation accepts force end to end (payload, DTO, dedup key,
in-flight bypass)
- an insights task carries the language requested from a cell with no base
row and chains its translation once the base exists
- retry re-dispatches every language the task ran on, and keeps the
"use configured targets" case as an empty list instead of guessing one
- the language alias table moves to @mx-space/ai so admin folds exactly as
the server does
* feat(ai): unify summary and insights behind a shared multilang base-then-translate pipeline
- ai_summaries gains is_translation/source_summary_id/source_lang, lang backfill,
dedup and UNIQUE(ref_id, lang) via migration 0032
- new ai-multilang MultilangAdapter + MultilangGenerationService: reuse or
generate the source-language base, invalidate stale translations, then
translate remaining targets concurrently
- insights drops chained queue subtasks for inline concurrent translation and
resolves its source lang from meta.lang instead of the nonexistent article.lang
- summary drops the per-language loop, adds SummaryTranslation task type,
POST /ai/summaries/task/translate and SUMMARY_GENERATED
- insights task DTO gains the 8-language cap and blank-token rejection;
overview and admin dispatch follow the base/translation split
* refactor(admin): dispatch insights like summary — base first, translation only for a single-language retry1 parent c84d0bb commit d8343e5
104 files changed
Lines changed: 15798 additions & 1004 deletions
File tree
- apps
- admin/src
- api
- features/ai
- components
- article-grouped
- article-overview
- hooks
- routes
- utils
- i18n/resources
- query
- socket
- views
- (intelligence)/ai
- overview
- [id]
- core
- src
- constants
- database/migrations
- meta
- modules/ai
- ai-generation-metrics
- ai-inflight
- ai-insights
- ai-multilang
- ai-overview
- ai-summary
- ai-task
- ai-translation
- ai-tts
- prompts
- processors/task-queue
- utils
- test/src
- modules/ai
- ai-task
- processors/task-queue
- docs/superpowers/specs
- packages
- ai/src
- db-schema/src/schema
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
280 | 280 | | |
281 | 281 | | |
282 | 282 | | |
283 | | - | |
284 | | - | |
285 | | - | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
286 | 301 | | |
287 | 302 | | |
288 | 303 | | |
| |||
307 | 322 | | |
308 | 323 | | |
309 | 324 | | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
315 | 331 | | |
316 | 332 | | |
317 | 333 | | |
| 334 | + | |
318 | 335 | | |
319 | 336 | | |
320 | 337 | | |
321 | | - | |
| 338 | + | |
322 | 339 | | |
323 | 340 | | |
324 | 341 | | |
| |||
378 | 395 | | |
379 | 396 | | |
380 | 397 | | |
| 398 | + | |
381 | 399 | | |
382 | 400 | | |
383 | 401 | | |
384 | 402 | | |
385 | 403 | | |
386 | | - | |
| 404 | + | |
387 | 405 | | |
388 | 406 | | |
389 | 407 | | |
| |||
Lines changed: 10 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| |||
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
97 | | - | |
| 99 | + | |
98 | 100 | | |
99 | 101 | | |
100 | 102 | | |
| |||
130 | 132 | | |
131 | 133 | | |
132 | 134 | | |
| 135 | + | |
133 | 136 | | |
134 | | - | |
| 137 | + | |
135 | 138 | | |
136 | 139 | | |
137 | 140 | | |
138 | 141 | | |
139 | | - | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
140 | 147 | | |
141 | 148 | | |
142 | 149 | | |
| |||
Lines changed: 57 additions & 17 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| 9 | + | |
8 | 10 | | |
9 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
10 | 16 | | |
11 | 17 | | |
12 | 18 | | |
13 | 19 | | |
14 | 20 | | |
| 21 | + | |
15 | 22 | | |
16 | 23 | | |
17 | 24 | | |
18 | | - | |
| 25 | + | |
| 26 | + | |
19 | 27 | | |
20 | 28 | | |
21 | 29 | | |
22 | 30 | | |
23 | 31 | | |
24 | | - | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
25 | 39 | | |
26 | 40 | | |
27 | 41 | | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
35 | 47 | | |
36 | 48 | | |
37 | 49 | | |
38 | 50 | | |
39 | 51 | | |
40 | 52 | | |
41 | 53 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
49 | 81 | | |
50 | 82 | | |
51 | 83 | | |
52 | 84 | | |
53 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
54 | 94 | | |
55 | 95 | | |
56 | 96 | | |
57 | 97 | | |
58 | 98 | | |
59 | | - | |
| 99 | + | |
60 | 100 | | |
61 | 101 | | |
62 | 102 | | |
| |||
Lines changed: 4 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
| 72 | + | |
71 | 73 | | |
72 | 74 | | |
73 | | - | |
| 75 | + | |
| 76 | + | |
74 | 77 | | |
75 | 78 | | |
76 | 79 | | |
| |||
0 commit comments