Commit cc51b5f
authored
Implement translation storage refactor (#1142)
* feat(api): add ranking item foundation
Add method-neutral ranking item storage, content rows, external source rows, and translated-content storage needed for future ranking item translation. Runtime MaxDiff item paths now use ranking item tables internally while public MaxDiff route naming remains temporarily.
Add source and translated plaintext storage/backfill support for rich text. Frontend-controlled rich text sends plaintext alongside HTML where wired, while translated plaintext is derived server-side from sanitized translated HTML and kept storage-only.
Add explicit Flyway enum split for ranking_item translation source kind, generated schema artifacts, Python generated models, and a one-off TypeScript plaintext backfill script. Update shared HTML-to-text helpers to decode entities consistently and add focused tests.
Deploy: agora, api, content-translation-worker, scoring-worker
* fix(project): isolate activity card controls
Refactor project activity cards so the card navigation is handled by an overlay SpaLink instead of wrapping the entire card content. This keeps the translation toggle as a separate interactive button, preventing its press state from activating the card push effect or triggering card hover/navigation semantics.
The visual CTA remains non-interactive, while the overlay link carries the accessible navigation label.
Deploy: agora
* fix(agora): normalize metadata font weight
Replace stale font-weight token references with the canonical normal-weight token so shared metadata lines do not inherit bold parent styles. This keeps timestamps and edited labels visually consistent across conversation and project metadata surfaces.
Deploy: agora
* fix(agora): unify project contact details
Extract the project details sidebar into a shared component used by project and project-conversation pages.
This keeps attribution and contact rendering consistent across desktop layouts, reuses the same details content in the mobile project details sheet, and fixes contact-only/mobile cases so the contact section remains reachable without reserving an empty logo column.
Also tightens the contact/attribution language prop types and hardens contact card layout for empty subtitles and long names.
Deploy: agora
* fix(conversation): redirect importing fetches
Return an explicit importing state from /conversation/get when the authenticated owner opens an importing conversation directly, and keep anonymous/non-owner callers seeing the same not-found behavior as inaccessible conversations.
The frontend now parses the shared DTO response union, redirects importing responses to the existing import status route, and keeps direct conversation cache updates compatible with the ready-response shape. Metadata lookups also exclude importing conversations so action helpers do not operate on them.
Deploy: agora, api
* fix(agora): block prioritization exports
Add an Open in Agora action to project conversation menus and keep external-link trailing icons aligned with the regular action text color.
Hide export entry points for prioritization conversations, redirect direct export-page visits once the conversation metadata shows an unsupported ranking conversation, and avoid fetching export readiness/history for unsupported pages.
Reject unsupported prioritization export requests in the API with a typed unsupported_conversation_type business failure, and guard readiness/history direct calls from reporting export availability for ranking conversations.
Deploy: agora, api
* fix(project): show invitation-only activities
Include unindexed project conversations in project activity lists while keeping public conversations ordered first.
Make invitation-only activity cards non-interactive and omit their conversation slug IDs from the project activity DTO.
Add dev project-page access scenarios for public, mixed, and invitation-only activity lists.
Deploy: agora, api
* fix(project): unify display content
Refactor project page and project-context content resolution around a single displayContent payload instead of sending original and translated variants together. Project content now uses sourceVersion backed by project_content.public_id, and manual project localizations are treated as original-only content for that language without translation controls.
Changes:
- Add project_content.public_id and regenerated migration 80
- Add shared display-content helpers and project display-content DTOs
- Add /project/content/fetch for fetching original or translated project content on demand
- Update project page, project conversation shell, project activities, and normal conversation project-context pill to consume displayContent/sourceVersion
- Align project translation SSE cache updates with the new project content fetch/cache path
- Add pure backend resolver tests for source, manual localization, machine translation, missing translation, stale translation, and translation-status lookup scenarios
- Sync shared TypeScript and generated Python/OpenAPI artifacts
Deploy: agora, api, content-translation-worker
* feat(content): implement translation storage refactor
Implement the storage and API refactor described in AGENTS_TRANSLATION_NOTES: ranking item display content, dedicated conversation config/import metadata tables, persisted BWS ranking mode, and strict rich-text/plain-text backfills.
Changes:
- Move import metadata to conversation_import_source at runtime
- Move Polis/ranking config reads and writes to dedicated config tables
- Rename persisted ranking mode value from maxdiff to bws
- Route public BWS endpoints under /ranking/bws
- Add ranking item display content and translation handling
- Add destructive cleanup migration for legacy MaxDiff item storage
- Add explicit rich-text/plain-text operational backfill script
- Update API, frontend clients, workers, and generated models
- Remove generated OpenAPI Markdown docs from checked-in outputs
Deploy: agora, api, import-worker, math-updater, scoring-worker
* fix(agora): refresh analysis tab labels
Resolve stale subtab labels after changing display language by mapping tab identifiers to translation keys instead of caching translated strings at component setup.
This keeps analysis and MaxDiff result tabs reactive to locale changes while preserving type-checked coverage for all known tab identifiers.
Deploy: agora
* fix(workers): log AI retry lifecycle
Add retry-worker-specific Bedrock read timeout settings while keeping their default value at 12 seconds, matching the previous behavior and preserving the math-updater first-pass timeout separately.
Add startup and provider-batch lifecycle logs for AI description and translation retry workers so production logs show effective timeout, model, region, concurrency, claim selection, provider elapsed time, and failure timing.
Deploy: ai-description-retry-worker, description-translation-retry-worker
* fix(workers): tune Bedrock read timeouts
Increase math-updater first-pass Bedrock read timeout to 40 seconds so Mistral Large has enough room for normal latency variance while still failing fast relative to async retry processing.
Set retry worker Bedrock read timeout defaults and production compose overrides to 60 seconds for out-of-band backlog cleanup.
Deploy: math-updater, ai-description-retry-worker, description-translation-retry-worker
* fix(api): harden resource access checks
Tighten API resource validation across conversation subresources and generated shared schemas. Validate MaxDiff comparison invariants at the shared DTO boundary, scope import/export and live-resource reads consistently, and keep generated clients in sync.\n\nDeploy: api, agora
* fix(workers): add AI retry cooldown
Add last_error_at tracking to AI description lineage and translation work so retryable provider failures can be held out of claim scans for a configurable cooldown window.
The retry workers now pass retry_cooldown_seconds through claim discovery and claiming, log the configured cooldown, and re-scan the primary database after materialization so newly materialized work still goes through the same claimable/cooldown filters before processing.
Retryable and non-retryable AI description error codes are centralized, retryable failures record last_error_at, successful/non-processable cleanup clears it, and non-retryable failures remain controlled by the AI description epoch rather than the cooldown.
Regenerate the branch-local V0079/V0080 migration artifacts with last_error_at columns, existing error backfill, maxdiff-to-ranking enum rewrite, and deferred plaintext check validation.
Deploy: api, ai-description-retry-worker, description-translation-retry-worker
* fix(api): avoid replica-lag rereads
Refactor post-write flows to carry transaction-local data and update frontend caches without immediate replica-routed refetches.
Changes:
- Return known eager translation work IDs from conversation/project writes
- Build notification SSE payloads from inserted/known rows instead of rereads
- Seed export, comment, and survey frontend caches from mutation responses
- Return survey gate state for survey config update/delete
- Keep analysis primary fallbacks only where freshness is explicitly requested
- Document read-after-write guidance
Deploy: agora, api, import-worker
* refactor(api): split ranking cleanup migrations
Split the ranking/maxdiff migration flow into safe schema expansion, validation, and optional destructive cleanup phases.\n\nChanges:\n- Fold the ranking_item content translation enum addition into the regenerated V0079 migration\n- Keep V0079 focused on safe schema expansion and maxdiff-to-ranking enum normalization\n- Keep V0080 focused on post-backfill validation constraints using NOT VALID followed by VALIDATE CONSTRAINT\n- Move destructive legacy maxdiff table and conversation column cleanup into V0081\n- Update Drizzle snapshots and journal for the regenerated migration sequence\n\nDeploy: api
* fix(api): split ranking item enum migration
Move the content_translation_source_kind ranking_item enum value back into its own Flyway migration so PostgreSQL commits the enum addition before V0079 uses it in partial indexes and check constraints.\n\nThis avoids the unsafe use of new enum value error when Flyway runs V0079 in a transaction.\n\nDeploy: api
* fix(api): backfill normalized ranking conversations
V0079 now rewrites conversation_type from maxdiff to ranking before V0079.1 runs. Update the ranking config backfill to target the normalized ranking value so PostgreSQL does not reject the removed maxdiff enum value.\n\nDeploy: api1 parent 2db19b2 commit cc51b5f
1,275 files changed
Lines changed: 82080 additions & 60518 deletions
File tree
- scripts/infra
- services
- agora
- src
- api
- .openapi-generator
- docs
- components
- conversation/export
- features/conversation
- newConversation
- dialog
- onboarding/backgrounds
- post
- analysis
- display
- interactionBar
- list
- maxdiff
- report
- page
- project
- ui-library
- composables
- conversation
- draft
- report
- pages
- conversation
- [conversationSlugId]
- edit
- [postSlugId].embed
- [postSlugId].onboarding
- verify
- [postSlugId]
- new
- create
- seed
- dev
- moderate/conversation/[conversationSlugId]
- project
- [projectSlug]/conversation
- [postSlugId]
- stores
- utils
- actions/definitions
- api
- comment
- contentTranslation
- conversationExport
- maxdiff
- post
- survey
- project
- survey
- translation
- ai-description-retry-worker
- src/ai_description_retry_worker
- api
- database/flyway
- drizzle
- meta
- scripts
- src
- service
- administrator
- conversationExport
- generators
- conversationImport
- externalSource
- tests
- content-translation-worker
- src/content_translation_worker
- tests
- description-translation-retry-worker
- src/description_translation_retry_worker
- import-worker/src/import_worker
- load-testing
- src
- api
- .openapi-generator
- docs
- math-updater
- scoring-worker/src/scoring_worker
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
530 | 530 | | |
531 | 531 | | |
532 | 532 | | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
533 | 544 | | |
534 | 545 | | |
535 | 546 | | |
| |||
582 | 593 | | |
583 | 594 | | |
584 | 595 | | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
585 | 649 | | |
586 | 650 | | |
587 | 651 | | |
| |||
746 | 810 | | |
747 | 811 | | |
748 | 812 | | |
749 | | - | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
750 | 818 | | |
751 | 819 | | |
752 | 820 | | |
| |||
0 commit comments