Improve AI optimization workflows, chapter export, import parsing, and Gemini errors#142
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves end-to-end “AI optimization” workflows (routing + background tasks), enhances chapter export/import ergonomics, and hardens Gemini error handling so failures surface clearer reasons and can fall back to a default model when appropriate.
Changes:
- Add
/api/polishrouter support (including{content, usage}-shaped AI responses) and introduce background-task-based outline/character optimization. - Enhance chapter export (range selection + split ZIP) and chapter list filtering UI.
- Improve TXT import parsing and Gemini empty/safety-blocked response handling (including propagating analysis failure reasons).
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/types/index.ts | Updates request typing for the polish API payload. |
| frontend/src/services/api.ts | Adds export options (range/split) and new polish background-task endpoints. |
| frontend/src/pages/Outline.tsx | Adds inline polishing for outline generation inputs and background outline optimization UI/flow. |
| frontend/src/pages/Characters.tsx | Adds single-edit “AI optimize” and batch background optimization for character/org settings. |
| frontend/src/pages/Chapters.tsx | Adds chapter filters and a modal-driven export flow (range + ZIP split). |
| frontend/src/components/FloatingTaskPanel.tsx | Adds labels for new task types (outline/character optimize). |
| backend/app/services/txt_parser_service.py | Strengthens TXT chapter detection rules to reduce false positives. |
| backend/app/services/plot_analyzer.py | Tracks/propagates analysis failure reasons and avoids retrying non-retryable Gemini blocks. |
| backend/app/services/book_import_service.py | Grounds imported entity generation in source text; improves fallback summaries and parsing robustness. |
| backend/app/services/background_task_service.py | Skips tasks cancelled while queued (pre-start cancellation guard). |
| backend/app/services/ai_clients/gemini_client.py | Improves prompt-block/empty-response detection; normalizes finish reasons and usage metadata. |
| backend/app/security.py | Allows Clash-style fake-IP DNS results (198.18/15) to pass URL validation. |
| backend/app/schemas/polish.py | Updates polish schema (UUID project_id + instruction) and adds background-task request models. |
| backend/app/main.py | Mounts the polish router under /api. |
| backend/app/api/projects.py | Adds export query params (start/end/split) and implements split ZIP export. |
| backend/app/api/polish.py | Adds polish helpers + background optimization task creation and workers. |
| backend/app/api/chapters.py | Propagates analysis failure reasons and adds Gemini-block fallback to default model. |
| .gitignore | Ignores local Playwright CLI cache directory. |
| .dockerignore | Prevents nested host node_modules from leaking into Docker build context. |
Comments suppressed due to low confidence (1)
backend/app/api/polish.py:239
/polish在写入GenerationHistory时仅依赖请求体里的project_id，没有验证该 project 是否属于当前用户（也没有处理无效 UUID 的情况）。这允许登录用户把生成历史写入其他用户的项目（数据污染/潜在滥用），并在 project_id 不存在时触发 500（FK/IntegrityError）。建议在if request.project_id:分支里先await verify_project_access(request.project_id, user_id, db)，并将无效/无权限情况返回 400/403。
# 如果提供了项目ID，记录到历史
if request.project_id:
history = GenerationHistory(
project_id=request.project_id,
prompt=f"原文: {request.original_text[:100]}...",
generated_content=polished_text,
model=request.model or "default"
)
db.add(history)
await db.commit()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| text: string; | ||
| style?: string; | ||
| original_text: string; | ||
| project_id?: number; |
There was a problem hiding this comment.
已修复：PolishTextRequest.project_id 已改为 project_id?: string，与后端 UUID 字符串类型保持一致。npm run build 已通过。
| from app.database import get_db, get_engine | ||
| from app.models.generation_history import GenerationHistory | ||
| from app.schemas.polish import PolishRequest, PolishResponse | ||
| from app.models.background_task import BackgroundTask |
There was a problem hiding this comment.
已修复：未使用的 BackgroundTask import 已移除，避免 lint/静态检查噪音。
| def _build_import_source_text(self, chapters: Optional[list[BookImportChapter]]) -> str: | ||
| if not chapters: | ||
| return "" | ||
|
|
||
| parts: list[str] = [] | ||
| for chapter in chapters: | ||
| title = str(chapter.title or "").strip() | ||
| content = str(chapter.content or "").strip() | ||
| if title: | ||
| parts.append(title) | ||
| if content: | ||
| parts.append(content) |
There was a problem hiding this comment.
已修复：_build_import_source_text 现在按 max_chars 增量截断，默认最多 80k 字符；给 AI 的 prompt context 也限制为前 30 章、最多 10k 字符，避免大项目导入时无界拼接。
| for pattern in patterns: | ||
| for match in re.finditer(pattern, source_text): | ||
| name = self._normalize_import_source_name_candidate(match.group(1)) | ||
| if not name or name in stopwords: | ||
| continue | ||
| occurrences = source_text.count(name) | ||
| if occurrences >= 3: | ||
| counter[name] += occurrences | ||
|
|
There was a problem hiding this comment.
已修复：候选名提取已改为 match_counter 单次累积正则 match，再统一过滤出现次数，不再在每个 match 内反复 source_text.count(name)。source_text 本身也已有长度上限。
| resolved_ip = ipaddress.ip_address(info[4][0]) | ||
| if _is_forbidden_ip(resolved_ip): | ||
| if _is_forbidden_ip(resolved_ip) and not _is_dns_proxy_fake_ip(resolved_ip): | ||
| raise HTTPException(status_code=400, detail="URL解析到内网或保留地址") |
There was a problem hiding this comment.
已修复：fake-ip 放行现在受 ALLOW_DNS_PROXY_FAKE_IP / settings.allow_dns_proxy_fake_ip 控制，默认值为 false；生产默认仍拒绝保留地址，只有显式开启的本地代理场景才允许 198.18/15。
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
c141901 to
f3fa795
Compare
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
f3fa795 to
ddbffda
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 22 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| pollTaskUntilComplete( | ||
| response.task_id, | ||
| (status: TaskStatus) => { | ||
| if (status.status === 'pending' || status.status === 'running') { | ||
| message.loading({ |
There was a problem hiding this comment.
已修复于 6273040：大纲优化轮询现在保存到 outlineOptimizePollCancelRef，组件卸载时会清理，启动新优化轮询前也会先取消旧轮询，避免 stale timer/setState。npm run build 已通过。
| pollTaskUntilComplete( | ||
| response.task_id, | ||
| (status) => { | ||
| if (status.status === 'pending' || status.status === 'running') { | ||
| message.loading({ | ||
| key: messageKey, |
There was a problem hiding this comment.
已修复于 6273040：角色批量优化轮询现在保存到 characterOptimizePollCancelRef，组件卸载和新任务启动前都会取消旧轮询，避免卸载后继续更新状态。npm run build 已通过。
| } catch (error) { | ||
| if (error instanceof Error && error.message) { | ||
| message.error(`导出失败：${error.message}`); | ||
| } else if (error) { | ||
| message.error('导出失败，请重试'); | ||
| } | ||
| } finally { |
There was a problem hiding this comment.
已修复于 6273040：handleExportSubmit 现在会识别 AntD validateFields() 的 errorFields 校验异常并直接返回，只对真正的导出请求/下载错误展示“导出失败”。npm run build 已通过。
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
ddbffda to
6273040
Compare
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
6273040 to
18fc9de
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 23 out of 24 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| function extractJsonObject(text: string): Record<string, unknown> | null { | ||
| const cleaned = text.trim() | ||
| .replace(/^```(?:json)?\s*/i, '') | ||
| .replace(/```$/i, '') | ||
| .trim(); | ||
|
|
||
| try { | ||
| return JSON.parse(cleaned) as Record<string, unknown>; | ||
| } catch { | ||
| const match = cleaned.match(/\{[\s\S]*\}/); | ||
| if (!match) return null; | ||
|
|
||
| try { | ||
| return JSON.parse(match[0]) as Record<string, unknown>; | ||
| } catch { | ||
| return null; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
已修复：新增 frontend/src/utils/jsonExtract.ts，将 extractJsonObject 抽成共享 util；Characters.tsx 和 Organizations.tsx 现在都复用同一实现，避免后续解析策略漂移。
|
|
||
| def _polish_max_tokens(text: str, *, has_instruction: bool = False) -> int: | ||
| minimum = 4096 if has_instruction else 2048 | ||
| return max(len(text or "") * 2, minimum) |
There was a problem hiding this comment.
已修复：_polish_max_tokens() 现在会在保留现有最小值的基础上将 max_tokens clamp 到 16000，避免长输入超过 provider 限制或产生异常成本。已通过 python3 -m py_compile backend/app/api/polish.py。
| disabled={chapters.length === 0} | ||
| block={isMobile} | ||
| > | ||
| 导出为TXT |
There was a problem hiding this comment.
已修复：章节页按钮文案已从“导出为TXT”改为更通用的“导出章节”，弹窗内继续区分“合并TXT / 分章ZIP”。npm run build 已通过。
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
2 times, most recently
from
255bcf2 to
96a6246
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 31 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const handleAIGenerateRelationships = async (values: { | ||
| relationship_count: number; | ||
| requirements?: string; | ||
| }) => { | ||
| setIsAIModalOpen(false); | ||
| setAiGenerating(true); | ||
| setAiProgress(0); | ||
| setAiMessage('开始分析大纲和章节...'); | ||
|
|
||
| try { | ||
| const response = await fetch('/api/relationships/generate-stream', { | ||
| method: 'POST', | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| }, | ||
| credentials: 'include', | ||
| body: JSON.stringify({ | ||
| project_id: projectId || '', | ||
| relationship_count: values.relationship_count, | ||
| requirements: values.requirements?.trim() || '', | ||
| }), | ||
| }); | ||
|
|
||
| if (!response.ok || !response.body) { | ||
| message.error(`请求失败: ${response.status}`); | ||
| return; | ||
| } | ||
|
|
||
| const reader = response.body.getReader(); | ||
| const decoder = new TextDecoder(); | ||
| let buffer = ''; | ||
|
|
||
| while (true) { | ||
| const { done, value } = await reader.read(); | ||
| if (done) break; | ||
|
|
||
| buffer += decoder.decode(value, { stream: true }); | ||
| const lines = buffer.split('\n'); | ||
| buffer = lines.pop() || ''; | ||
|
|
||
| for (const line of lines) { | ||
| if (!line.startsWith('data: ')) continue; | ||
| try { | ||
| const data = JSON.parse(line.slice(6)); | ||
| if (data.type === 'progress') { | ||
| setAiProgress(data.progress || 0); | ||
| setAiMessage(data.message || ''); | ||
| } else if (data.type === 'done') { | ||
| message.success('AI关系生成完成'); | ||
| loadData(); | ||
| } else if (data.type === 'error') { | ||
| message.error(data.error || data.message || '生成失败'); | ||
| return; | ||
| } | ||
| } catch { | ||
| // Ignore raw generation chunks. | ||
| } | ||
| } | ||
| } | ||
| } catch (error) { | ||
| message.error(error instanceof Error ? error.message : '启动生成失败'); | ||
| } finally { | ||
| setAiGenerating(false); | ||
| setAiProgress(0); | ||
| setAiMessage(''); | ||
| } | ||
| }; |
There was a problem hiding this comment.
已修复：关系生成现在使用 relationshipGenerateAbortRef 持有 AbortController，启动新生成会先 abort 旧请求；SSEProgressModal.onCancel 和组件卸载都会真正 abort()，不再只是把 aiGenerating 置为 false。已通过 npm run build。
| character_by_name = {character.name: character for character in characters} | ||
| existing_pairs = { | ||
| frozenset((relationship.character_from_id, relationship.character_to_id)) | ||
| for relationship in existing_relationships | ||
| } |
There was a problem hiding this comment.
已修复：关系生成提示词现在要求 AI 返回角色 ID，并在保存前通过 character_by_id 精确解析；角色名仅作为兼容兜底，且只接受项目内唯一名称，重名时不会绑定，直接跳过该条关系，避免写入到错误角色。已通过 python3 -m py_compile backend/app/api/relationships.py。
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
96a6246 to
5aab1d5
Compare
felixchaos
force-pushed
the
codex/ai-workflow-export-import-fixes
branch
from
5aab1d5 to
af805d0
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 32 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
frontend/src/utils/sseClient.ts:62
SSEClientOptions新增了signal?: AbortSignal，但SSEClient（EventSource 版本）的connect()/close()逻辑没有监听该 signal。这样调用方即使传入 signal 也无法真正取消连接，接口语义会误导。建议要么在connect()里在 signal abort 时调用this.close()并 reject/resolve，要么把signal从SSEClientOptions中拆分成仅SSEPostClient使用的选项。
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Closes #141
Summary
This PR implements the local fixes and feature improvements described in #141, then iterates on the follow-up Copilot review feedback and the later runtime bugs found during manual use.
Main changes
/api/polishrouter, including support for AI responses shaped as{ content, usage }and safer project access validation before writing generation history.new_textresponses are still applied, and empty responses no longer leave the modal stuck in a fake success state.ALLOW_DNS_PROXY_FAKE_IPsetting for local Clash/fake-IP DNS development while keeping production default behavior locked down.node_modulesand local browser automation cache from being included accidentally.Follow-up fixes after review
PolishTextRequest.project_idtyping to match backend UUID strings.Verification
npm run buildpython -m py_compile backend/app/api/projects.py backend/app/api/chapters.py backend/app/api/polish.py backend/app/api/relationships.py backend/app/schemas/polish.py backend/app/services/ai_clients/gemini_client.py backend/app/services/background_task_service.py backend/app/services/book_import_service.py backend/app/services/plot_analyzer.py backend/app/services/txt_parser_service.py backend/app/security.py backend/app/main.pygit diff --check