Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
4ccfa4e
docs: add semantic search design spec
wesm Jul 4, 2026
fb395ae
docs: address codex review of semantic search spec
wesm Jul 4, 2026
d3ed4ba
docs: align semantic search spec with kit v0.2.0 API
wesm Jul 5, 2026
63403cc
docs: pin DeleteVectors cleanup in mirror refresh
wesm Jul 5, 2026
457f34e
docs: add semantic search implementation plan
wesm Jul 5, 2026
bc90e76
build: pin go.kenn.io/kit v0.2.0 and smoke-test sqlitevec
wesm Jul 5, 2026
779a787
feat(config): add [vector] configuration section
wesm Jul 5, 2026
f893216
feat(vector): add OpenAI-compatible embeddings encoder
wesm Jul 5, 2026
1b49356
feat(vector): add vectors.db index core with generation lifecycle
wesm Jul 5, 2026
c313893
feat(vector): mirror refresh with resync-stable doc keys
wesm Jul 5, 2026
69d67c8
fix(vector): delete evicted slot occupant's vectors on mirror refresh
wesm Jul 5, 2026
1c0b7e3
fix: address review findings in vector config, coverage, scan, and do…
wesm Jul 5, 2026
a3b870b
docs: address review findings in semantic search spec and plan
wesm Jul 5, 2026
c551bbd
feat(vector): build orchestration with generations and progress
wesm Jul 5, 2026
de99ed3
feat(vector): semantic search over the mirror index
wesm Jul 5, 2026
94a2683
docs(vector): correct Search doc comment error names
wesm Jul 5, 2026
4720c12
feat(db): semantic content-search mode with capability gating
wesm Jul 5, 2026
6369525
feat(db): hybrid RRF content-search mode
wesm Jul 5, 2026
0de2efd
feat(cli): session search --semantic and --hybrid modes
wesm Jul 5, 2026
a466400
fix(vector): address checkpoint-2 review findings in mirror, build, a…
wesm Jul 5, 2026
e6455fd
fix(search): checkpoint-2 findings in semantic redaction, backend val…
wesm Jul 5, 2026
e605a3f
feat: symmetric message windows with role filtering across backends
wesm Jul 5, 2026
4182793
feat: inline context windows on content search
wesm Jul 5, 2026
dfe2add
docs(service): correct negative-context reachability comment
wesm Jul 5, 2026
b39f05c
feat(mcp): semantic search mode and around windows
wesm Jul 5, 2026
ea1d472
feat(server): embeddings build lifecycle endpoints
wesm Jul 5, 2026
ab65f11
fix(vector): serialize generation lifecycle and recover build panics
wesm Jul 5, 2026
b6797ea
feat(cli): embeddings build/list/activate/retire commands
wesm Jul 5, 2026
d298815
test(cli): cover embeddings daemon dispatch, progress ticker, and act…
wesm Jul 5, 2026
689ff60
fix: checkpoint-3 findings in eviction accounting, generation 404s, a…
wesm Jul 5, 2026
3448095
fix(service): cap around windows and redact search context messages
wesm Jul 5, 2026
22bd7ef
feat(serve): wire vector index, searcher, and after-sync embed scheduler
wesm Jul 5, 2026
1facc2f
docs: semantic search usage and architecture notes
wesm Jul 5, 2026
1c7fe19
docs: correct semantic-search limitations and update CLI references
wesm Jul 5, 2026
aa9cc75
fix: final-review findings in daemon embeddings path, locking, and er…
wesm Jul 5, 2026
209d07c
fix: close around-cap overflow, scheduler tee for periodic sync, and …
wesm Jul 5, 2026
f7e9263
fix: legacy ended_at rescan, bind-list batching, and build-semantics …
wesm Jul 5, 2026
481c052
fix(vector): harden builds and query path against endpoint failures
wesm Jul 5, 2026
cb7db04
feat(vector): include_automated build scope for the embedding index
wesm Jul 5, 2026
77b4323
fix(vector): scope flag override, legacy scope upgrade, and build hyg…
wesm Jul 5, 2026
106dacb
docs: fold semantic-search design notes into internals page, drop wor…
wesm Jul 5, 2026
49cb1e2
fix(vector): abort builds on auth-status encode errors instead of ski…
wesm Jul 5, 2026
8438279
docs: add skills pack design spec
wesm Jul 5, 2026
a0fa314
docs: add skills pack implementation plan
wesm Jul 5, 2026
4a5d3cc
fix(vector): preserve retries and guard semantic search
wesm Jul 5, 2026
44d2369
feat(cli): relative --since filter for session search and list
wesm Jul 5, 2026
514cbe3
feat(skills): embedded finding-history skill template and renderer
wesm Jul 5, 2026
aef186c
feat(cli): skills install and list commands
wesm Jul 5, 2026
2bf440a
docs: skills pack usage docs, drop working spec
wesm Jul 5, 2026
420f9be
fix: address open review findings across skills, vector, and docs
wesm Jul 5, 2026
4895a2e
docs: add run-grouped embeddings design spec
wesm Jul 5, 2026
f44e89a
docs: revise run-grouped embeddings spec per review
wesm Jul 5, 2026
ccf98cd
docs: make run-grouped embeddings spec seams and migration explicit
wesm Jul 5, 2026
df005ff
docs: define scope precedence and hit compatibility in embeddings spec
wesm Jul 5, 2026
1e81896
docs: correct resolver index note in embeddings spec
wesm Jul 5, 2026
41ce538
docs: add run-grouped embeddings implementation plan
wesm Jul 5, 2026
23491e8
feat(db): embeddable unit reducer for run-grouped embeddings
wesm Jul 5, 2026
c7b8579
feat(vector): versioned mirror schema with reset and read sentinel
wesm Jul 5, 2026
9bf4230
feat(vector): mirror unit documents with run doc keys
wesm Jul 5, 2026
5496a76
feat(vector): run_v1 fingerprint params and 15 percent chunk overlap
wesm Jul 6, 2026
fd61899
feat(vector): anchor semantic hits to ordinal ranges
wesm Jul 6, 2026
bc30f3d
docs: specify semantic-only subordinate penalty and anchor tie semantics
wesm Jul 6, 2026
229e1e8
fix(vector): version-gate search path, anchor-local snippets, identit…
wesm Jul 6, 2026
ef3ead0
feat(search): unit-granularity hybrid fusion with subordinate penalty
wesm Jul 6, 2026
97c2976
feat(search): scope filter with include_children precedence
wesm Jul 6, 2026
e2ddb94
docs: carve child sessions out of one-shot gate for semantic scope
wesm Jul 6, 2026
b7a7cc2
fix(search): exempt child sessions from one-shot gate in semantic modes
wesm Jul 6, 2026
2be52e5
feat(api): ordinal-range semantic hits and rebuild-required wiring
wesm Jul 6, 2026
8a302cc
docs: run-grouped embeddings docs, drop working spec
wesm Jul 6, 2026
09a570d
docs: correct single-message hit shape and one-shot carve-out wording
wesm Jul 6, 2026
c9d4d4f
fix: enforce read-only vector opens, batch hybrid FTS leg, MCP scope …
wesm Jul 6, 2026
7df5558
fix: final review minors — DSN escaping, parked-row reads, doc gaps
wesm Jul 6, 2026
24ccfdf
fix(db): enforce read-only sessions.db connections with file: DSN scheme
wesm Jul 6, 2026
e6ea71b
docs: add conversation-unit citations design spec
wesm Jul 6, 2026
cfbece0
docs: exact run endpoints, honest bench acceptance, event-row cardina…
wesm Jul 6, 2026
163dd49
docs: move cardinality note after derived-unit rules
wesm Jul 6, 2026
3b0cc14
docs: name the two omission-pin tests precisely in citations spec
wesm Jul 6, 2026
73165c6
docs: add conversation-unit citations implementation plan
wesm Jul 6, 2026
64f995d
test(db): content-search page benchmarks for citation baseline
wesm Jul 6, 2026
93a8bf3
feat(db): conversation-unit range derivation with SQLite seam
wesm Jul 6, 2026
b43376c
test(db): pin is_system flip predicate with sidechain system row
wesm Jul 6, 2026
e90c1ec
feat(api): ordinal_range citation field replaces ordinal_start/end
wesm Jul 6, 2026
78751a9
feat(db): derived conversation-unit citations on lexical search
wesm Jul 6, 2026
fabd60d
perf(db): batch unit-range probes per page and group seam scans
wesm Jul 6, 2026
9bcca52
test(db): cross-chunk span coverage and post-tuning doc corrections
wesm Jul 6, 2026
33fe932
feat(postgres): conversation-unit citations on PG content search
wesm Jul 6, 2026
6372c97
fix(db): stop forcing WAL journal mode on read-only connections
wesm Jul 6, 2026
d0231d4
fix(search): close candidate cursors before unit derivation
wesm Jul 6, 2026
95c219b
fix(search): classify hybrid unit-less hits before scope and fusion
wesm Jul 6, 2026
b6edc6d
fix(duckdb): self-range ordinal_range placeholder on content search
wesm Jul 6, 2026
12c1060
test(db): pin FTS benchmark page to the long-run corpus region
wesm Jul 6, 2026
e5f7136
test(db): drop temporary page-composition probe
wesm Jul 6, 2026
f019a7f
chore(frontend): regenerate API client for ordinal_range
wesm Jul 6, 2026
9c51f0d
docs: checkpoint wording fixes for citations spec and plan
wesm Jul 6, 2026
7a444d5
perf(db): correlated point-lookup run extents with run sharing
wesm Jul 6, 2026
8b3e0b4
refactor(db): share unit-range resolution across backend seams
wesm Jul 6, 2026
fa74f80
feat(duckdb): conversation-unit citations on DuckDB content search
wesm Jul 6, 2026
a1c7181
feat(mcp): conversation-unit citation fields on search results
wesm Jul 6, 2026
aaa4f27
docs: conversation-unit citations, drop working spec
wesm Jul 6, 2026
723fd69
docs: unit-less hybrid penalty accuracy and no-auto-disable policy
wesm Jul 6, 2026
8e88a5b
fix(postgres): mirror SQLite session-role index for dense boundary fetch
wesm Jul 6, 2026
cc229a5
test(db): self-enforcing bench page fill and post-derivation comments
wesm Jul 6, 2026
c6d91d0
feat(vector): add input_suffix for models needing a client-side termi…
wesm Jul 6, 2026
3ed9d18
docs: cover semantic search and citations in overview pages
wesm Jul 6, 2026
1798d6f
fix(sync): skip local git discovery for foreign-machine sessions
wesm Jul 7, 2026
5fe3d48
test(sync): neutral machine name in identity discovery fixture
wesm Jul 7, 2026
67c73a0
fix(export): skip macOS automount namespaces in root path resolution
wesm Jul 7, 2026
1d9fab3
feat(vector): request base64 embeddings with float fallback
wesm Jul 7, 2026
a24be57
feat(vector): parallel document encodes via [vector.embeddings] concu…
wesm Jul 7, 2026
2ee1fbe
fix(vector): bump kit for the sqlitevec KNN join-order fix
wesm Jul 7, 2026
eda8dc9
feat(vector): named embeddings servers with per-build --using selection
wesm Jul 7, 2026
a8fba36
docs: named embeddings servers, default_server, and build --using
wesm Jul 7, 2026
7d46369
fix(server): map unknown embeddings server to 400, regen API client
wesm Jul 7, 2026
b2d050b
fix(vector): open vectors.db with modernc driver on Windows, kit v0.2.1
wesm Jul 7, 2026
f4d1cab
fix(vector): seed parking sentinel below leftover parked ordinals
wesm Jul 7, 2026
0129a34
test(vector): per-driver var-limit seam, promote modernc to direct dep
wesm Jul 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 29 additions & 24 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ CLI (agentsview) -> Config -> DB (SQLite/FTS5)
- `internal/parser/` - Per-agent session file parsers and content extraction.
- `internal/server/` - HTTP handlers, SSE, middleware, search, and export.
- `internal/sync/` - Sync engine, file watcher, discovery, and hashing.
- `internal/vector/` - Semantic search: embeddings encoder, `vectors.db`
mirror/index, build orchestration, and semantic/hybrid search.
- `internal/timeutil/` - Time parsing utilities.
- `internal/web/` - Embedded frontend copied from `frontend/dist/` at build
time.
Expand All @@ -128,30 +130,33 @@ CLI (agentsview) -> Config -> DB (SQLite/FTS5)

## Key Files

| Path | Purpose |
| -------------------------------- | --------------------------------------------- |
| `cmd/agentsview/main.go` | CLI entry point, server startup, file watcher |
| `cmd/agentsview/pg.go` | `pg` command group: push, status, serve |
| `internal/server/server.go` | HTTP router and handler setup |
| `internal/server/sessions.go` | Session list/detail API handlers |
| `internal/server/search.go` | Full-text search API |
| `internal/server/events.go` | SSE event streaming |
| `internal/db/db.go` | Database open, migrations, schema |
| `internal/db/sessions.go` | Session CRUD queries |
| `internal/db/search.go` | FTS5 search queries |
| `internal/sync/engine.go` | Sync orchestration |
| `internal/parser/types.go` | Agent registry with one `AgentDef` per agent |
| `internal/parser/*.go` | Per-agent session parsers |
| `internal/postgres/connect.go` | Connection setup, SSL checks, DSN helpers |
| `internal/postgres/schema.go` | PG DDL and schema management |
| `internal/postgres/push.go` | Push logic and fingerprinting |
| `internal/postgres/sync.go` | Push sync lifecycle |
| `internal/postgres/store.go` | PostgreSQL read-only store |
| `internal/postgres/sessions.go` | PG session queries on the read side |
| `internal/postgres/messages.go` | PG message queries and ILIKE search |
| `internal/postgres/analytics.go` | PG analytics queries |
| `internal/postgres/time.go` | Timestamp conversion helpers |
| `internal/config/config.go` | Config loading and flag registration |
| Path | Purpose |
| -------------------------------- | --------------------------------------------------------- |
| `cmd/agentsview/main.go` | CLI entry point, server startup, file watcher |
| `cmd/agentsview/pg.go` | `pg` command group: push, status, serve |
| `cmd/agentsview/embeddings.go` | `embeddings` command group: build, list, activate, retire |
| `internal/server/server.go` | HTTP router and handler setup |
| `internal/server/sessions.go` | Session list/detail API handlers |
| `internal/server/search.go` | Full-text search API |
| `internal/server/events.go` | SSE event streaming |
| `internal/db/db.go` | Database open, migrations, schema |
| `internal/db/sessions.go` | Session CRUD queries |
| `internal/db/search.go` | FTS5 search queries |
| `internal/vector/index.go` | `vectors.db` schema, generations, staleness gate |
| `internal/vector/search.go` | Semantic + hybrid search, RRF merge |
| `internal/sync/engine.go` | Sync orchestration |
| `internal/parser/types.go` | Agent registry with one `AgentDef` per agent |
| `internal/parser/*.go` | Per-agent session parsers |
| `internal/postgres/connect.go` | Connection setup, SSL checks, DSN helpers |
| `internal/postgres/schema.go` | PG DDL and schema management |
| `internal/postgres/push.go` | Push logic and fingerprinting |
| `internal/postgres/sync.go` | Push sync lifecycle |
| `internal/postgres/store.go` | PostgreSQL read-only store |
| `internal/postgres/sessions.go` | PG session queries on the read side |
| `internal/postgres/messages.go` | PG message queries and ILIKE search |
| `internal/postgres/analytics.go` | PG analytics queries |
| `internal/postgres/time.go` | Timestamp conversion helpers |
| `internal/config/config.go` | Config loading and flag registration |

## Development

Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,11 @@ agentsview stats --include-git-outcomes
| ![Search](https://agentsview.io/assets/generated/screenshots/search-results.png) | ![Heatmap](https://agentsview.io/assets/generated/screenshots/heatmap.png) |

- **Full-text search** across all message content (FTS5)
- **Semantic search** (opt-in) -- index session content with any
OpenAI-compatible embeddings endpoint and search by meaning with
`agentsview session search --semantic` or `--hybrid`; every content-search
match cites the conversation unit it came from
([docs](https://agentsview.io/semantic-search/))
- **Token usage and cost dashboard** -- per-session and per-model cost
breakdowns, daily spend charts, all in the web UI
- **Analytics dashboard** -- activity heatmaps, tool usage, velocity metrics,
Expand Down
2 changes: 2 additions & 0 deletions cmd/agentsview/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,12 +115,14 @@ func newRootCommand() *cobra.Command {
root.AddCommand(newActivityCommand())
root.AddCommand(newPGCommand())
root.AddCommand(newDuckDBCommand())
root.AddCommand(newEmbeddingsCommand())
root.AddCommand(newSessionCommand())
root.AddCommand(newMCPCommand())
root.AddCommand(newStatsCommand())
root.AddCommand(newParseDiffCommand())
root.AddCommand(newClassifierCommand())
root.AddCommand(newSecretsCommand())
root.AddCommand(newSkillsCommand())
root.AddCommand(newDoctorCommand())
root.AddCommand(newVersionCommand())
root.AddCommand(newOpenAPICommand())
Expand Down
7 changes: 6 additions & 1 deletion cmd/agentsview/doctor.go
Original file line number Diff line number Diff line change
Expand Up @@ -202,12 +202,17 @@ func inspectDoctorDB(path string) doctorDBInspection {
return insp
}

// doctorReadOnlyDSN builds a read-only sqlite3 DSN. The file: scheme is
// required for mattn/go-sqlite3 to honor mode=ro (a bare path silently opens
// read-write), and the path is percent-encoded so `%`, `?`, or `#` in a real
// path cannot be misparsed as URI syntax.
func doctorReadOnlyDSN(path string) string {
params := url.Values{}
params.Set("mode", "ro")
params.Set("_busy_timeout", "5000")
params.Set("_foreign_keys", "ON")
return path + "?" + params.Encode()
escaped := (&url.URL{Path: path}).EscapedPath()
return "file:" + escaped + "?" + params.Encode()
}

func listDoctorResyncTempFiles(dbPath string) []string {
Expand Down
Loading