Skip to content

Commit 7f321fc

Browse files
committed
feat: OpenRouter OneShot, Quick Chat, dimension/rerank/transcription fixes
- OpenRouter OneShot dialog with model classification (tool-calling, vision, embedding) - OpenRouter transcription adapter (base64 JSON, no multipart) - Quick Chat side panel with Direct RAG mode, session per collection - Embedding dimension follows collection's actual Qdrant vector size - Collection-aware embedding overrides in DirectQueryModule and recall - Reranker fixes: base_url in OneShot, skip Qwen chat fallback for non-Qwen models - Transcription preload: skip when external provider active, skip when already loading - Collection creation: child_chunk_size default 256, ADVANCED collapsible section - Recall filesMap: resolve source→display_name for search results and eval cases - Cleanup stale selectedCollections IDs on fetch - Various settings page improvements (auto-save, one-per-row layout, model folding) - favicon.svg replaced
1 parent b0f74ae commit 7f321fc

67 files changed

Lines changed: 2359 additions & 2098 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.template

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# SinkDuce environment variables
2+
# Copy this file to .env and modify as needed:
3+
# cp .env.template .env
4+
5+
# Backend API port (default: 18900)
6+
API_PORT=18900
7+
8+
# Frontend dev server port (default: 5173)
9+
UI_PORT=5173
10+
11+
# Qdrant vector database HTTP port (default: 6343)
12+
QDRANT_HTTP_PORT=6343
13+
14+
# Qdrant vector database gRPC port (default: 6334)
15+
QDRANT_GRPC_PORT=6334

README.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -40,15 +40,25 @@ docker compose up -d --build
4040

4141
Docker rebuilds the image with the latest code while preserving your `data/` directory (database, config, history).
4242

43-
### Recommended Out-of-the-Box Setup (DashScope)
43+
### Recommended Out-of-the-Box Setup
4444

45-
The project is highly optimized for [Alibaba Cloud DashScope (Bailian/阿里云百炼)](https://bailian.console.aliyun.com/). Go to **Settings****LLM Providers** → click **OneShot Setting (DashScope API)**, enter your API Key, and all services (LLM, Embedding, Reranker, STT) will automatically auto-configure with optimal defaults:
45+
Two OneShot configuration paths are available:
46+
47+
**DashScope** (Alibaba Cloud): Go to **Settings****LLM Providers****OneShot Setting (DashScope API)**:
4648

4749
* **LLM**: `deepseek-v4-flash`
48-
* **Embedding**: `text-embedding-v4`
50+
* **Embedding**: `text-embedding-v4` (1024d)
4951
* **Reranker**: `qwen3-rerank`
5052
* **Transcription**: `fun-asr` / `fun-asr-realtime`
5153

54+
**OpenRouter**: Go to **Settings****LLM Providers****OneShot Setting (OpenRouter API)**. Enter your API Key; models are auto-fetched and classified:
55+
56+
* **LLM**: `deepseek/deepseek-v4-flash`
57+
* **Chat** (function-calling, tools-filtered): `deepseek/deepseek-v4-pro`
58+
* **Visual** (vision models): `xiaomi/mimo-v2.5`
59+
* **Embedding**: `qwen/qwen3-embedding-4b` (default 1536d)
60+
* **Reranker**: `cohere/rerank-v3.5`
61+
5262
## 🏗️ Core Pillars
5363

5464
### ⚡ 01. Spark: Fluid Friction Capture & Synthesis
@@ -58,8 +68,10 @@ The project is highly optimized for [Alibaba Cloud DashScope (Bailian/阿里云
5868
* **Full-Featured Markdown Workspace (Collection Notes)**: Create structured personal notes explicitly bound to specific operational contexts, featuring full WYSIWYG editing with markdown support (headings, tables, task lists, code blocks, images, YouTube embeds).
5969
* **Intelligent Note Distillation (Drag-to-Distill)**: Drag an existing old note or document into your current workspace, and the system automatically extracts the core insights into a dense citation block, seamlessly aggregating scattered ideas without manual rewriting.
6070
* **AI Image Ingestion**: One click prompts the AI to generate a precise contextual text description, weaving visual data into your markdown memory map for vector indexing. Supports image pasting, drag-and-drop, and resizing with inline captions.
61-
* **Audio Scribing & Tri-Fold Pipeline**: Upload recordings or record live meetings/lectures via WebSockets. Transcription (STT) can be handled via the embedded local engine or flexible external API endpoints. The system then generates a structured tri-fold artifact: *Summary* (semantic synthesis), *To-Do List/Action Items*, and *Detail* (deep-dive information extraction that filters conversational noise while preserving core intent).
71+
* **Audio Transcription & Multi-Phase Pipeline**: Upload recordings or record live meetings/lectures via WebSockets. Transcription (STT) can be handled by the embedded local FunASR engine or external APIs (DashScope, OpenRouter, etc.). After transcription, a multi-phase analysis runs: **Pass 1 — General Summary + Blueprint** (SSE streaming, builds section structure); **Pass 2 — Section Summary** (one-click custom sections, one keyword generates description and summary); **Pass 3 — Final Merge** producing Summary, To-Do List, and Detail artifacts.
6272
* **Hot Words**: Manage custom vocabulary libraries for domain-specific terminology to improve transcription accuracy.
73+
* **Quick Chat Panel**: A floating side panel in the Collection view for rapid Q&A via Direct RAG. Diamond button with animation, per-collection sessions, SSE streaming with thinking step display.
74+
* **Multi-turn Agentic Chat**: The main Chat supports multi-turn conversation with tool-calling Agent that autonomously decides whether, where, and how to search — exposing the full Agentic RAG capability through conversational interaction.
6375

6476
### 📥 02. Sink: Anti-Hoarding Ingestion Pipeline
6577

@@ -70,13 +82,14 @@ The project is highly optimized for [Alibaba Cloud DashScope (Bailian/阿里云
7082
* **Granular Document Parsing & Chunking**: Utilizes the natively embedded parsing engine (supporting 12 format parsers) or links to powerful cloud parsing APIs (e.g., *MinerU*). Supports advanced **Parent-Child chunking** by headings, paragraphs, and max token configurations while keeping word boundaries intact.
7183
* **Context Enrichment Engine**: When enabled, an LLM evaluates each split chunk and injects its missing global context, mitigating the "chunk isolation" effect during retrieval.
7284
* **Auto-Summarization & Consolidation**: Every document gets a structured summary. Collections get consolidated overviews with automatic conflict detection when documents contradict each other.
85+
* **Auto-Updating Collection Coverage**: Collection-level coverage maps update automatically after each ingestion, comparing new and existing documents to keep retrieval context aligned with the latest knowledge boundaries.
7386

7487
### 🧠 03. Educe: High-Dimensional Contextual Reasoning
7588

7689
**Educe** implements a cutting-edge retrieval architecture to turn cold data into active intelligence.
7790

7891
* **Advanced Hybrid Retrieval**: Supports dense vector similarity search, keyword-semantic hybrid querying (BM25 + Dense via Qdrant), and advanced **Reranking** algorithms to surface top-tier context.
79-
* **Iterative Agentic RAG Pipeline**: Moves far beyond naïve single-shot semantic search. Sinkduce orchestrates a multi-step Agentic loop: *Analyze → Route → Retrieve → Grade → Decompose → Rerank → Synthesize* until it locks onto the exact underlying truth.
92+
* **Agentic RAG Pipeline**: Complex queries are decomposed into atomic sub-queries, routed to multiple Collections in parallel, each executing a Variant loop (Rewrite + Grade), then aggregated and synthesized. Full event stream observability (`decompose → variant_generation → scoring → synthesize_merge`).
8093
* **Multi-Collection Federated Search**: Context isolation does not limit high-dimensional synthesis. Users can choreograph inquiries spanning multiple explicit Collections simultaneously, harmonizing cross-domain shards via top-level reasoning.
8194
* **Absolute Source Traceability (3-Layer Traceability)**: Build bulletproof trust in AI answers by drilling straight to the raw text. Instantly inspect the source lineage across three deep-dive levels: the specific *Vector Chunk*, the *Full-Text Context*, or the *Original Source File*.
8295
* **Recall Evaluation**: Built-in benchmarking with adjustable parameters to evaluate retrieval recall and precision.
@@ -145,7 +158,7 @@ The MCP server uses **stdio** transport — Claude Code launches it as a subproc
145158

146159
### Backend & Frontend Stack
147160

148-
Python 3.11+, FastAPI, React 19, Vite, TypeScript, Tailwind CSS, Qdrant, FunASR, Zustand, Shadcn UI.
161+
Python 3.11+, FastAPI, React 19, Vite, TypeScript, Tailwind CSS, Qdrant, FunASR, Zustand, Shadcn UI, SSE Streaming, Session-based Chat, Tool-calling LLM Agent.
149162

150163
### Directory Layout
151164

README_CN.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,15 +44,25 @@ docker compose up -d --build
4444

4545
Docker 将使用最新代码重新构建镜像,同时完整保留您的 `data/` 目录(包含数据库、配置及历史记录)。
4646

47-
### 推荐开箱即用配置 (阿里云百炼 DashScope)
47+
### 推荐开箱即用配置
4848

49-
本项目对 [阿里云百炼 (DashScope)](https://bailian.console.aliyun.com/) 进行了深度优化。前往 **Settings****LLM Providers** → 点击 **OneShot Setting (DashScope API)** 并输入您的 API Key,系统将自动一键配置所有服务,并匹配最佳默认模型:
49+
项目提供两套 OneShot 一键配置方案:
50+
51+
**阿里云百炼 (DashScope)**:前往 **Settings****LLM Providers****OneShot Setting (DashScope API)**,输入 API Key 自动配置全部服务:
5052

5153
* **LLM**: `deepseek-v4-flash`
52-
* **Embedding**: `text-embedding-v4`
54+
* **Embedding**: `text-embedding-v4`(1024 维)
5355
* **Reranker**: `qwen3-rerank`
5456
* **Transcription**: `fun-asr` / `fun-asr-realtime`
5557

58+
**OpenRouter**:前往 **Settings****LLM Providers****OneShot Setting (OpenRouter API)**,输入 API Key 后自动拉取模型列表并分类:
59+
60+
* **LLM**: `deepseek/deepseek-v4-flash`
61+
* **Chat**(自动筛选支持工具调用的模型): `deepseek/deepseek-v4-pro`
62+
* **Visual**(自动筛选视觉模型): `xiaomi/mimo-v2.5`
63+
* **Embedding**: `qwen/qwen3-embedding-4b`(默认 1536 维)
64+
* **Reranker**: `cohere/rerank-v3.5`
65+
5666
## 🏗️ 三大核心支柱
5767

5868
### ⚡ 01. Spark: Fluid Friction Capture & Synthesis
@@ -62,8 +72,10 @@ Docker 将使用最新代码重新构建镜像,同时完整保留您的 `data/
6272
* **Full-Featured Markdown Workspace (Collection Notes)**: 创建明确绑定到特定业务上下文的结构化个人笔记。支持全功能所见即所得(WYSIWYG)编辑及 Markdown 语法(标题、表格、任务列表、代码块、图片和 YouTube 视频内嵌)。
6373
* **Intelligent Note Distillation (Drag-to-Distill)**: 直接将现有的旧笔记或文档拖入当前工作区,系统会自动将其核心洞察提取为高密度的引用块,无需手动重写即可无缝聚合零散灵感。
6474
* **AI Image Ingestion**: One click 触发 AI 生成精准的图像上下文文本描述,将视觉数据织入您的 Markdown 记忆网络中以供向量索引。支持图片粘贴、拖拽和带行内字幕的尺寸调整。
65-
* **音频转录与“三折页”流水线**: 支持上传录音或通过 WebSockets 录制实时会议/讲座。语音转文字(STT)可由内置的本地引擎或灵活的外部 API 端点处理。系统随后会生成结构化的三折页伪影:*Summary(语义合成摘要)**To-Do List/Action Items(清晰的行动项)* 以及 *Detail(深度信息提取,过滤口语杂音并保留核心意图)*
75+
* **音频转录与多阶段智能管道**: 支持上传录音或通过 WebSockets 录制实时会议/讲座。语音转文字(STT)可由内置本地 FunASR 引擎或外部 API(DashScope、OpenRouter 等)处理。转写完成后自动运行多阶段分析:**Pass 1 — General Summary + Blueprint(章节结构发现)**,SSE 流式输出,动态构建章节骨架;**Pass 2 — Section Summary(逐章节深度总结)**,支持一键添加自定义 Section,仅需一个关键词即可智能生成该章节的描述与摘要;**Pass 3 — 合并产出**,最终生成 Summary、To-Do List 及 Detail 三折页产物
6676
* **Hot Words**: 管理特定专业领域的自定义词汇库,以显著提高特定术语的语音转写准确率。
77+
* **Quick Chat Panel**: Collection 视图中通过悬浮菱形按钮呼出的侧边栏,使用 Direct RAG 快速问答。每个 Collection 独立会话,支持 SSE 流式输出与 thinking 过程展示。
78+
* **Multi-turn Agentic Chat**: 主 Chat 支持多轮对话,Agent 通过工具调用(Tool-calling)自主决策是否检索、检索哪些 Collection、以何种模式检索,将 Agentic RAG 的完整能力暴露在对话交互中。
6779

6880
### 📥 02. Sink: Anti-Hoarding Ingestion Pipeline
6981

@@ -74,13 +86,14 @@ Docker 将使用最新代码重新构建镜像,同时完整保留您的 `data/
7486
* **Granular Document Parsing & Chunking**: 利用内置的本地解析引擎(支持 12 种格式解析器)或链接到强大的云端解析 API(如 *MinerU*)。支持基于标题、段落和最大 Token 配置的高级 **Parent-Child chunking**,同时保持完整的词边界。
7587
* **Context Enrichment Engine**: 开启后,LLM 将评估每个切分出的数据块,并为其注入缺失的全局上下文,从而缓解检索过程中的“分块孤立”效应。
7688
* **Auto-Summarization & Consolidation**: 每个文档都会获得一个结构化摘要。同时,集合会生成固化综述,并在文档间出现事实冲突时自动进行冲突检测与提示。
89+
* **Auto-Updating Collection Coverage**: 集合在每次文档入库后自动触发摘要固化与变化检测,对比新旧文档并更新集合级别的主题覆盖图(Coverage Map),确保检索上下文始终反映最新的知识边界。
7790

7891
### 🧠 03. Educe: High-Dimensional Contextual Reasoning
7992

8093
**Educe** 实现了前沿的检索架构,将静态的数据转化为活跃的智能。
8194

8295
* **Advanced Hybrid Retrieval**: 支持标准稠密向量相似度检索、关键词-语义混合查询(基于 Qdrant 的 BM25 + Dense),并结合先进的 **Reranking算法** 来锁定顶级上下文。
83-
* **Iterative Agentic RAG Pipeline**: 远超传统朴素的单次语义检索。Sinkduce 编排了一个多步骤的 Agentic RAG 循环:*Analyze → Route → Retrieve → GradeDecomposeRerankSynthesize*,直到精准锁定底层事实
96+
* **Agentic RAG Pipeline**: 复杂查询 → 拆解为原子子查询 → 并行路由至多个 Collection → 各自执行 Variant(Rewrite + Grade loop)→ 聚合合成。全程事件流可观测(`decomposevariant_generationscoringsynthesize_merge`),每阶段结果可追溯
8497
* **Multi-Collection Federated Search**: Context-Isolated 架构并不会限制高维度的知识合成。用户可以设计同时跨越多个明确集合的联合查询,系统将触发相互隔离的多路检索流水线,并通过顶层推理协调跨领域碎片。
8598
* **Absolute Source Traceability (3-Layer Traceability)**: 允许直接穿透至原始文本,建立对 AI 回答的铁证信任。您可以立即向下钻取三层源头脉络:特定 *Vector Chunk**Full-Text Context**Original Source File*
8699
* **Recall Evaluation**: 内置基准测试功能,可通过可调参数评估检索的召回率(Recall)和精确度(Precision)。
@@ -149,7 +162,7 @@ JSON
149162

150163
### Backend & Frontend Stack
151164

152-
Python 3.11+, FastAPI, React 19, Vite, TypeScript, Tailwind CSS, Qdrant, FunASR, Zustand, Shadcn UI.
165+
Python 3.11+, FastAPI, React 19, Vite, TypeScript, Tailwind CSS, Qdrant, FunASR, Zustand, Shadcn UI, SSE Streaming, Session-based Chat, Tool-calling LLM Agent.
153166

154167
### Directory Layout
155168

frontend/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "frontend",
33
"private": true,
4-
"version": "0.0.0",
4+
"version": "1.0.0",
55
"type": "module",
66
"scripts": {
77
"dev": "vite",

frontend/public/favicon.png

-542 KB
Loading

0 commit comments

Comments
 (0)