You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor: port tabbed Playground/Dashboard UI from zeabur-rag
- Merge Chat, Report, Learn into tabbed index.html
- Add Inspect, Triage, Edit tabs to Playground
- Simplify nav to [Playground | Dashboard] pill
- Move Settings/Refresh to tab bar as icon-only buttons
- Add tooltip help on stat cards (Triage, Dashboard, Query Signals)
- Add source color legend in Query Signals
- Fix formatAuditChanges to read old_value/new_value
- Rename "Similarity" to "Score" with mode-aware tooltips
- Redirect /report and /learn to /?tab=xxx
- Remove standalone learn.html and report.html
- Add zeabur-rag-feedback skill
- Sync skill doc fixes (RRF vs cosine explanations)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1.**Call `zeabur-rag-triage`** to get the pending items list.
18
-
2.**Present results** as a numbered list grouped by category (🔴 Reports, 🟡 Unverified, 🟠 Low-similarity, 🔵 Negative feedback). Show counts.
18
+
2.**Present results** as a numbered list grouped by category (🔴 Reports, 🟡 Unverified, 🟠 Low-score, 🔵 Negative feedback, ⚪ Needs-frontmatter). Show counts.
19
19
3.**If all categories are empty:** tell the user "The knowledge base has no pending items — looks healthy ✅" and stop.
20
20
4.**Wait for user** to pick an item by number or description.
21
21
5.**Run the decision tree** for that item type (see below).
description: Send feedback on RAG search results — thumbs-up (+1) or thumbs-down (-1) to indicate whether the answer was helpful. Use after calling zeabur-rag-search when you can evaluate the quality of the results. Positive feedback reinforces good results; negative feedback flags poor matches for review. Requires write:feedback scope.
4
+
---
5
+
6
+
# RAG Feedback
7
+
8
+
Send feedback on search results to improve knowledge base quality over time.
9
+
10
+
Base URL: `$ZEABUR_RAG_URL`
11
+
Auth: `Authorization: Bearer $RAG_API_KEY`
12
+
13
+
## API
14
+
15
+
```bash
16
+
curl -s -X POST "$ZEABUR_RAG_URL/api/feedback" \
17
+
-H "Authorization: Bearer $RAG_API_KEY" \
18
+
-H "Content-Type: application/json" \
19
+
-d '{
20
+
"query": "how to deploy to zeabur",
21
+
"answer": "the generated answer text",
22
+
"chunks": ["DOC-1234", "SUP-5678"],
23
+
"score": 1,
24
+
"signal_id": "optional-signal-id",
25
+
"comment": "optional explanation"
26
+
}'
27
+
```
28
+
29
+
| Field | Required | Description |
30
+
|-------|----------|-------------|
31
+
|`query`| Yes | The original search query |
32
+
|`answer`| No | The generated RAG answer |
33
+
|`chunks`| No | Array of chunk IDs that were returned |
34
+
|`score`| Yes |`1` (helpful) or `-1` (not helpful) |
35
+
|`signal_id`| No | Signal ID from the search response — links feedback to the specific query signal |
36
+
|`comment`| No | Free-text explanation of why the result was good or bad |
37
+
38
+
## When to send feedback
39
+
40
+
| Situation | Score |
41
+
|-----------|-------|
42
+
| Answer correctly addressed the question |`1`|
43
+
| Retrieved chunks were relevant and accurate |`1`|
44
+
| Answer was wrong, incomplete, or misleading |`-1`|
45
+
| Retrieved chunks were irrelevant to the query |`-1`|
46
+
| No useful results were returned |`-1`|
47
+
48
+
## Notes
49
+
50
+
- If `signal_id` is provided, the feedback score is written back to the `rag_query_signals` table, linking it to the original query for analytics.
51
+
- Negative feedback surfaces in the Triage review queue for admin follow-up.
52
+
- Feedback is also appended to `data/user-feedback.jsonl` as a backup log.
Copy file name to clipboardExpand all lines: skills/zeabur-rag-search/SKILL.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,7 +34,7 @@ curl -s -X POST "$ZEABUR_RAG_URL/api/query" \
34
34
35
35
**Always pass `client: "claude-code"` when calling from Claude Code** so queries are trackable in the dashboard.
36
36
37
-
Response includes `chunks[]` (each with `id`, `title`, `answer`, `tags`, `similarity`, `source`, `verified`) and `signal_id` for linking feedback.
37
+
Response includes `chunks[]` (each with `id`, `title`, `answer`, `tags`, `similarity` (relevance score), `source`, `verified`) and `signal_id` for linking feedback. Note: `similarity` is an RRF score in hybrid mode (~0–0.02) or cosine similarity in semantic mode (0–1). The scale depends on the search mode — do not compare values across modes.
Copy file name to clipboardExpand all lines: skills/zeabur-rag-triage/SKILL.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
---
2
2
name: zeabur-rag-triage
3
-
description: List pending knowledge base maintenance items — open reports, unverified learned chunks, low-similarity queries, and negative feedback. Use when checking KB health, reviewing the maintenance queue, or starting a curation session. Triggers on "review queue", "what needs attention in the KB", "pending reports", "unverified chunks", "failed queries", "KB health check". Requires admin scope.
3
+
description: List pending knowledge base maintenance items — open reports, unverified learned chunks, low-score queries, and negative feedback. Use when checking KB health, reviewing the maintenance queue, or starting a curation session. Triggers on "review queue", "what needs attention in the KB", "pending reports", "unverified chunks", "failed queries", "KB health check". Requires admin scope.
4
4
---
5
5
6
6
# RAG — Triage
7
7
8
-
Show what needs attention in the knowledge base. Returns 4 categories of pending items in a single call.
8
+
Show what needs attention in the knowledge base. Returns 5 categories of pending items in a single call.
|`days`|`14`| Time window for signals (reports and learned have no time window) |
23
-
|`similarity_threshold`|`0.4`| Queries with `top_similarity` below this are flagged |
23
+
|`similarity_threshold`|`0.4`| Queries with `top_similarity` below this are flagged. Note: in hybrid mode (RRF), scores are ~0–0.02 so the default 0.4 catches all hybrid queries. Adjust to ~0.005 for meaningful hybrid-mode filtering.|
24
24
|`limit`|`20`| Max items per category |
25
25
26
26
## Response
27
27
28
-
Returns JSON with 4 arrays + `counts` (true totals, not truncated) + `has_more` (boolean per category):
28
+
Returns JSON with 5 arrays + `counts` (true totals, not truncated) + `has_more` (boolean per category):
29
29
30
30
-**`open_reports`** — reports with `status='open'`, no time window. Fields: `id`, `type`, `chunk_id`, `query`, `detail`, `created_at`.
31
31
-**`unverified_learned`** — learned chunks with `status='unverified'`, no time window. Fields: `id`, `title`, `tags`, `created_at`, `age_days`.
32
-
-**`low_similarity_signals`** — queries where the best result scored below the threshold, within the time window, deduped by query text. Fields: `id`, `query`, `top_similarity`, `top_chunk_ids`, `client`, `created_at`.
32
+
-**`low_similarity_signals`** — queries where the top score fell below the threshold, within the time window, deduped by query text. The score is RRF in hybrid mode (~0–0.02) or cosine in semantic mode (0–1). Fields: `id`, `query`, `top_similarity`, `top_chunk_ids`, `client`, `created_at`.
33
33
-**`negative_feedback_signals`** — queries that got negative user feedback, within the time window. Fields: `id`, `query`, `feedback_score`, `feedback_comment`, `top_chunk_ids`, `created_at`.
34
+
-**`needs_frontmatter`** — chunks whose tags include `"needs-frontmatter"`, meaning they were ingested from a source file that lacks proper frontmatter and got a temporary ID. Fields: `id`, `title`, `tags`, `url`, `source`, `created_at`.
34
35
35
36
If a category query fails, its array is empty and a `<category>_error` field appears with the error message.
0 commit comments