naikan 内観

Distill years of messy self-notes into a structured personal context pack for AI assistants.

If you have been journaling to yourself — in a LINE chat with yourself, a notes app, a diary — you are sitting on the highest-quality personalization data that exists for you. naikan (内観, introspection) turns that raw archive into two artifacts:

1. A context pack (PERSONA.md + per-theme notes): an LLM-distilled portrait of your recurring themes, values, and how they shifted over time — written as briefing context you can paste into Claude, ChatGPT, or an agent's memory file.
2. A queryable index (index.json): every entry, clustered by theme, searchable in milliseconds from a single-binary CLI.

Everything runs locally. The only network call is one optional LLM request for the distillation step — and the pipeline degrades gracefully without it.

 raw exports                    pipeline (Python)                     outputs
┌──────────────┐   ┌────────────────────────────────────┐   ┌─────────────────────┐
│ LINE talk    │   │ parse → normalize → TF-IDF          │   │ PERSONA.md          │
│ exports .txt ├──▶│ → spherical k-means themes          ├──▶│ themes/*.md         │
│ app JSON     │   │ → LLM distillation (Claude, 1 call) │   │ index.json ◀─┐      │
│ plain text   │   └────────────────────────────────────┘   └──────────────┼──────┘
└──────────────┘                                                           │
                                              search / stats (Go) ─────────┘

Why this exists

Agent memory is usually built forward: the assistant accumulates notes as you use it. But most people already have years of backlog. naikan builds memory backward — it bootstraps an assistant's understanding of you from the archive you already have, and keeps you as the only editor: the output is plain Markdown you review before any model ever sees it in a system prompt.

Quick start

# 1. Build the context pack (works without an API key — LLM step auto-skips)
cd pipeline
pip install -e .
naikan build ../examples/diary.txt -o out

# 2. With an API key, one Claude call labels the themes and writes the persona
export ANTHROPIC_API_KEY=sk-ant-...
naikan build ../examples/diary.txt -o out

# 3. Search your history at native speed
cd ../cli
go build -o naikan-cli .
./naikan-cli -index ../pipeline/out/index.json search "何を大切にしたいか"
./naikan-cli -index ../pipeline/out/index.json stats

Design notes

• No morphological analyzer, no embedding service. Japanese/English mixed text is tokenized with a small unicode-aware scheme (word tokens for Latin runs, character bigrams for CJK runs), implemented identically in Python and Go so the index writer and the search CLI agree byte-for-byte. Clustering is TF-IDF + spherical k-means in plain NumPy; retrieval is BM25 in stdlib Go. The whole thing is auditable in an afternoon — a deliberate choice for a tool that reads your diary.
• One LLM call, structured output. Theme labeling and persona synthesis is a single Claude request (claude-opus-4-8 by default) with a JSON-schema-constrained response. The prompt instructs the model to refer to people by role, never by name.
• Frozen contract. index.json v1 is the interface between the two halves; either side can be replaced independently (swap in real embeddings, a TUI, a vector DB) without touching the other.

Privacy

Your notes never leave your machine except for the excerpts sent in the single distillation call (skippable with --no-llm). Nothing in this repository contains real journal data — examples/diary.txt is fictional. Review PERSONA.md before pasting it anywhere. It is your context; you are its only editor.

Layout

Path	What
pipeline/	Python — parsing, clustering, LLM distillation, pack output
cli/	Go — BM25 search, stats, theme browsing over index.json
examples/	Fictional sample input

日本語で一言

LINEの「自分専用トーク」やメモアプリに溜めた言語化メモを、AIアシスタントに渡せる 「自分の取扱説明書」に蒸留するツールです。処理はすべてローカル、LLM呼び出しは任意の 1回だけ。生成物はただのMarkdownなので、共有する前に自分の目で編集できます。

License

MIT