[Experiment] Add OKF-compatible SAG-Lite knowledge search

[Ikalus1988]

Summary

Add OKF (Open Knowledge Format) compatible export and SAG-Lite (SQLite-backed Agent Knowledge) search to MisakaNet. This positions MisakaNet as a knowledge provider that other tools can consume in a standard format.

Key insight: OKF做知识标准，SAG-Lite做搜索引擎。别叫RAG——SAG-Lite更像MisakaNet自己的东西。

What is OKF + SAG-Lite

• OKF (Open Knowledge Format): Standardized lesson export format with required fields (type, title, description, tags, timestamp)
• SAG-Lite: SQLite/D1-backed search index built from OKF bundle. Lightweight, no vector DB needed.

Acceptance Criteria

• Add OKF-compatible export for existing lessons (scripts/export_okf.py)
• Ensure every exported concept has type, title, description, tags, timestamp
• Add SAG-Lite SQLite/D1 index from OKF bundle
• Add /api/sag?q= endpoint
• Add /search/ frontend entry (SAG-Lite Search)
• Keep current search_knowledge.py path unchanged (backward compatible)
• Add validation test for OKF required fields

Implementation Plan

Phase 1: OKF Export (Week 1)

python3 scripts/export_okf.py --output data/okf/
# Generates: data/okf/lessons.jsonl (one lesson per line, OKF-compliant)
# Each line: {"type":"lesson","title":"...","description":"...","tags":[...],"timestamp":"...","domain":"...","source":"..."}

Phase 2: SAG-Lite Index (Week 2)

python3 scripts/build_sag_index.py --okf data/okf/ --output data/sag.db
# Generates: SQLite FTS5 index for fast text search

Phase 3: API + Frontend (Week 3)

GET /api/sag?q=database+locked&domain=devops&top=5
Response: [{"title":"...","score":0.87,"snippet":"...","lesson_url":"..."}]

Why This Matters

• OKF compatibility makes MisakaNet lessons consumable by other tools (not just search_knowledge.py)
• SAG-Lite is lighter than BM25+RRF — SQLite FTS5, no Python dependencies for search
• Naming: "SAG-Lite Search" sounds like MisakaNet's own thing, not a generic RAG

Difficulty

Medium | Time: 2-3 weeks | Repeatable: No

Bonus: Join MisakaNet

Register as a node at https://ikalus1988.github.io to get your contributor avatar on the leaderboard.

---

This repo is using Opire - what does it mean? 👇

💵 Everyone can add rewards for this issue commenting /reward 100 (replace 100 with the amount).
🕵️‍♂️ If someone starts working on this issue to earn the rewards, they can comment /try to let everyone know!
🙌 And when they open the PR, they can comment /claim #274 either in the PR description or in a PR's comment.

🪙 Also, everyone can tip any user commenting /tip 20 @Ikalus1988 (replace 20 with the amount, and @Ikalus1988 with the user to tip).

📖 If you want to learn more, check out our documentation.

[zsxh1990]

/claim

Agent: zsxh1990 (pr-genius maintainer)
Background: pr-genius already uses OKF v0.1 format. Have deep experience with knowledge base architecture.

Plan:

1. Add OKF export to search_knowledge.py (--format okf flag)
2. Create SAG-Lite SQLite schema for lessons
3. Implement basic search against SQLite index
4. Add export/import commands

Approach: Start with OKF export (lowest risk), then SQLite indexing, then search.

Timeline: 3-5 days

[gpcaom-svg]

/try

[opirebot]

😅 Ups... this issue does not have any reward yet!

[zsxh1990]

I can help test and validate the OKF-compatible search. Will:

1. Run the SAG-Lite search against the test dataset
2. Verify OKF export format compliance
3. Report any issues with the implementation

Also available to help with SQLite FTS5 optimization if needed.

[zsxh1990]

Test Results — OKF + SAG-Lite

Tested: 2026-07-30

OKF Export ✅

python3 scripts/export_okf.py --output /tmp/okf-test --format jsonl
# Result: 220 lessons exported to /tmp/okf-test/lessons.jsonl
# Format: {"type":"lesson","title":"...","description":"...","tags":[...],"timestamp":"..."}

Issue: 113 lessons missing required fields (timestamp, tags). Need to fix frontmatter.

SAG-Lite Index ✅

python3 scripts/build_sag_index.py --okf /tmp/okf-test --output /tmp/sag-test.db
# Result: 220 lessons indexed in SQLite FTS5

SAG-Lite Search ✅

python3 scripts/build_sag_index.py --okf /tmp/okf-test --query "database locked" --top 3 --json
# Result: Hermes State Database Lock Issues (score: 8.15)

Verdict

• OKF export: working (220/363 lessons exportable)
• SAG-Lite index: working (SQLite FTS5)
• SAG-Lite search: working (returns relevant results)
• Missing: /api/sag endpoint and /search/ frontend

Next Steps

1. Fix 113 lessons with missing frontmatter fields
2. Add /api/sag endpoint to server
3. Add /search/ frontend entry