feat(data): first-class Data mode for project inventory and reclassification (#1166)

## Summary

Adds a top-level Data mode for reviewing project identity across an archive and managing folder-to-project mappings. It replaces the mapping editor in Settings with an inventory, a project workspace, and centralized rules that can be previewed and applied atomically.

## Behavior and correctness

- Keys inventory rows by opaque project identity, so sanitized-label collisions remain distinct and every deep link and reclassification targets the exact project.
- Presents observed folders in a compact workspace suited to projects with many source directories. Unknown identities appear as Unclassified, and filesystem roots are not suggested as automatic mappings.
- Binds reclassification previews to the normalized draft, collision state, mapping revision, and exact affected-session snapshot. Changed state returns a conflict instead of applying a stale preview.
- Reconciles identity evidence when sessions move between projects while preserving trustworthy historical Git evidence for missing sources and orphaned sessions.
- Emits project/session changes after mutations and refreshes Data, Sessions, and Activity metadata across clients. Background rule refreshes update the list without discarding unsaved edits.
- Keeps mapping mutations local to writable SQLite archives. PostgreSQL and DuckDB expose the same inventory and governance data as read-only views.

## Publication and storage

Filtered PostgreSQL and DuckDB publication keeps mappings, snapshots, provenance, and tombstones within each configured project scope. Incremental watch work is bounded by changed sessions, project moves publish in both directions, placeholder snapshots are excluded, and mirrored sessions retain their source machine, file path, and exact database generation.

The SQLite data version advances from 76 to 77 to reparse project-identity source snapshots while preserving live and orphaned archive data through non-destructive migration. PostgreSQL gains source-scoped publication metadata and session provenance fields. DuckDB schema v8 adds source archive and mapping provenance; existing disposable mirrors rebuild rather than migrate in place.

Co-authored-by: Wes McKinney <wesm@users.noreply.github.com>

Author: wesm

.github/workflows/ci.yml

@@ -118,6 +118,7 @@ jobs:
         run: |
           bash scripts/install_test.sh
           bash scripts/retry_test.sh
+          bash scripts/e2e-server_test.sh
           bash scripts/check_desktop_release_health_test.sh
           bash scripts/check_desktop_release_health_from_event_test.sh
           bash desktop/scripts/test-repair-appimage-diricon.sh
@@ -366,3 +367,9 @@ jobs:
         env:
           E2E_PREBUILT_FIXTURE: /tmp/testfixture
           E2E_PREBUILT_SERVER: /tmp/agentsview
+
+      - name: Run DuckDB E2E tests
+        run: make e2e-duckdb
+        env:
+          E2E_PREBUILT_FIXTURE: /tmp/testfixture
+          E2E_PREBUILT_SERVER: /tmp/agentsview

.gitignore

@@ -67,9 +67,9 @@ docs/.env*.local
 testdata/ssh/test_key
 
 # Local data
-data/
-sessions/
-html/
+/data/
+/sessions/
+/html/
 .superset/
 .github/hooks/
 .superpowers/

Makefile

@@ -378,7 +378,8 @@ e2e:
 # Run focused Playwright smoke tests against duckdb serve.
 e2e-duckdb:
 	cd frontend && AGENTSVIEW_E2E_BACKEND=duckdb npx playwright test \
-		e2e/duckdb-backend.spec.ts e2e/session-list.spec.ts --project=chromium
+		e2e/duckdb-backend.spec.ts e2e/data-mode.spec.ts \
+		e2e/session-list.spec.ts --project=chromium
 
 # Vet
 vet: pricing-snapshot ensure-embed-dir

cmd/testfixture/main.go

@@ -12,6 +12,7 @@ import (
 
 	"go.kenn.io/agentsview/internal/db"
 	duckdbsync "go.kenn.io/agentsview/internal/duckdb"
+	"go.kenn.io/agentsview/internal/export"
 	"go.kenn.io/agentsview/internal/money"
 )
 
@@ -119,6 +120,12 @@ func main() {
 		log.Fatalf("creating recent-edits fixture: %v", err)
 	}
 
+	if err := createProjectReclassificationFixture(
+		database, base.Add(120*time.Hour),
+	); err != nil {
+		log.Fatalf("creating project-reclassification fixture: %v", err)
+	}
+
 	fmt.Printf("Fixture DB written to %s\n", *out)
 	if *duckDBOut != "" {
 		if err := writeDuckDBMirror(database, *duckDBOut); err != nil {
@@ -128,6 +135,80 @@ func main() {
 	}
 }
 
+func createProjectReclassificationFixture(
+	database *db.DB, start time.Time,
+) error {
+	const (
+		machine      = "remote-example-host"
+		project      = "wrong_branch_label"
+		worktreeRoot = "/srv/worktrees/github.com/example-org/sample-service/example-worktree"
+		model        = "claude-sonnet-4-20250514"
+	)
+	cwds := []struct {
+		suffix string
+		cwd    string
+	}{
+		{suffix: "root", cwd: worktreeRoot},
+		{suffix: "nested", cwd: worktreeRoot + "/cmd/server"},
+	}
+	ctx := context.Background()
+	for index, item := range cwds {
+		sessionID := "test-session-project-reclassification-" + item.suffix
+		startedAt := start.Add(time.Duration(index) * time.Hour)
+		endedAt := startedAt.Add(12 * time.Minute)
+		firstMessage := "Inspect the sample service worktree."
+		session := db.Session{
+			ID:               sessionID,
+			Project:          project,
+			Machine:          machine,
+			Agent:            "claude",
+			StartedAt:        new(startedAt.Format(time.RFC3339Nano)),
+			EndedAt:          new(endedAt.Format(time.RFC3339Nano)),
+			MessageCount:     2,
+			UserMessageCount: 1,
+			FirstMessage:     new(firstMessage),
+			Cwd:              item.cwd,
+		}
+		if err := database.UpsertSession(session); err != nil {
+			return fmt.Errorf(
+				"upserting project-reclassification session: %w", err,
+			)
+		}
+		if err := database.InsertMessages(generateMessages(
+			sessionID, session.MessageCount, startedAt, model,
+		)); err != nil {
+			return fmt.Errorf(
+				"inserting project-reclassification messages: %w", err,
+			)
+		}
+		if err := database.UpsertProjectIdentityObservation(
+			ctx,
+			export.ProjectIdentityObservation{
+				SessionID:            sessionID,
+				Project:              project,
+				Machine:              machine,
+				RootPath:             worktreeRoot,
+				RepositoryPath:       "/srv/worktrees/github.com/example-org/sample-service",
+				WorktreeName:         "example-worktree",
+				WorktreeRootPath:     worktreeRoot,
+				WorktreeRelationship: export.WorktreeLinked,
+				CheckoutState:        export.CheckoutBranch,
+				GitBranch:            "example-worktree",
+				ObservedAt:           startedAt,
+			},
+		); err != nil {
+			return fmt.Errorf(
+				"upserting project-reclassification identity: %w", err,
+			)
+		}
+		fmt.Printf(
+			"  %s: %d messages (project reclassification)\n",
+			sessionID, session.MessageCount,
+		)
+	}
+	return nil
+}
+
 func writeDuckDBMirror(database *db.DB, path string) error {
 	if err := os.Remove(path); err != nil &&
 		!errors.Is(err, os.ErrNotExist) {

cmd/testfixture/main_test.go

@@ -0,0 +1,53 @@
+package main
+
+import (
+	"context"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"go.kenn.io/agentsview/internal/db"
+)
+
+func TestCreateProjectReclassificationFixture(t *testing.T) {
+	database, err := db.Open(filepath.Join(t.TempDir(), "sessions.db"))
+	require.NoError(t, err)
+	t.Cleanup(func() { require.NoError(t, database.Close()) })
+
+	base := time.Date(2026, 7, 1, 10, 0, 0, 0, time.UTC)
+	require.NoError(t, createProjectReclassificationFixture(database, base))
+
+	const (
+		machine      = "remote-example-host"
+		wrongProject = "wrong_branch_label"
+		worktreeRoot = "/srv/worktrees/github.com/example-org/sample-service/example-worktree"
+	)
+	wantCwds := map[string]string{
+		"test-session-project-reclassification-root":   worktreeRoot,
+		"test-session-project-reclassification-nested": worktreeRoot + "/cmd/server",
+	}
+	for sessionID, wantCwd := range wantCwds {
+		session, getErr := database.GetSession(context.Background(), sessionID)
+		require.NoError(t, getErr)
+		require.NotNil(t, session)
+		assert.Equal(t, machine, session.Machine)
+		assert.Equal(t, wrongProject, session.Project)
+		assert.Equal(t, wantCwd, session.Cwd)
+	}
+
+	snapshots, err := database.ListSessionProjectIdentitySnapshots(
+		context.Background(),
+	)
+	require.NoError(t, err)
+	require.Len(t, snapshots, 2)
+	for _, snapshot := range snapshots {
+		assert.Equal(t, machine, snapshot.Machine)
+		assert.Equal(t, wrongProject, snapshot.Project)
+		assert.Equal(t, worktreeRoot, snapshot.RootPath)
+		assert.Equal(t, worktreeRoot, snapshot.WorktreeRootPath)
+		assert.NotEmpty(t, snapshot.Key)
+	}
+}

docs/activity.md

@@ -106,6 +106,17 @@ that total across usage rows in proportion to their catalog-price estimates. The
 per-model costs are therefore estimated attributions, not provider-reported
 model charges, but they still sum to the displayed total.
 
+## Create A Project Mapping
+
+Worktree layouts the parser does not recognize can surface a branch or
+worktree directory name as a project. Each row in the **Project** breakdown
+links to that project on the [Data page](/data/), where the mapping editor lists
+the project's observed session folders, previews the full-archive impact of a
+folder-path → project rule, and applies a
+[worktree project mapping](/configuration/#worktree-project-mappings) rule in
+one atomic step. Cleaning always evaluates the complete archive; the current
+Activity range and filters do not carry over.
+
 ## Activity Insight
 
 At the bottom of the page, **Activity Insight** shows an existing global

docs/configuration.md

@@ -720,27 +720,43 @@ manual sync, and the periodic directory scan.
 
 ### Worktree Project Mappings
 
-The parser infers a session's project from its `cwd`, which works for standard
-layouts but not custom worktree conventions like
-`~/code/{project}.worktrees/feat/<branch>/` — those sessions otherwise group
-under `<branch>` rather than `{project}`. As of 0.29.0, you can register manual
-**path-prefix → project** rules from the **Worktree Project Mappings** section
-in Settings:
-
-![Worktree Project Mappings settings section](/assets/generated/screenshots/worktree-mappings.png)
+The parser infers a session's project from its `cwd`. It recognizes common
+worktree manager layouts, including the generic
+`worktrees/github.com/<owner>/<repository>/<worktree>` convention, where the
+repository segment becomes the project. Layouts it does not recognize — such
+as `~/code/{project}.worktrees/feat/<branch>/` — otherwise group sessions
+under `<branch>` rather than `{project}`. For those, register manual
+**path-prefix → project** rules from the **Rules** view on the
+[Data page](/data/#rules), or let the
+[mapping editor](/data/#create-a-project-mapping) create one from a project's
+observed session folders:
+
+![Worktree mapping rules on the Data page](/assets/generated/screenshots/worktree-mappings.png)
 
 - Mappings are explicit; there is no auto-discovery.
+- Each rule is scoped to one machine. The machine selector manages rules for
+  the local machine and for any remotely synced machine. Rules live in the
+  writable archive that ingests that machine's sessions, which may be the
+  source machine's local SQLite archive or a separate collector archive.
 - Each rule applies whenever a session's `cwd` falls under the configured
   prefix, on both new sessions as they sync and (via the **Apply** button)
-  already-imported sessions.
+  already-imported sessions. Prefixes match on directory boundaries, so
+  `/worktrees/service` does not match `/worktrees/service-old`.
+- Enabled mappings run after parser inference, so an explicit rule always wins
+  when the two disagree.
 - The default `explicit` layout maps every matching path to the project name
   stored on the rule. The `repo_dot_worktrees` layout derives the project from
   the first path segment under the prefix when it is named `<repo>.worktrees`,
   so a path like `/code/agentsview.worktrees/feature/frontend` resolves to
   project `agentsview`.
-- Rules are stored in a `worktree_project_mappings` SQLite table scoped to the
-  host machine, so a mapping created on one machine does not leak into another
-  machine's view of synced sessions.
+- Rules created from the Data mapping editor record the mislabeled
+  project they corrected, shown as the rule's **original label**. The value is
+  informational and set once; to manually revert a reclassification, edit the
+  rule's target back to that original label and apply again.
+- Disabling or deleting a rule does not rewrite sessions by itself. Sessions
+  whose source files still exist revert to parser-derived names on a later
+  reparse or full resync, while orphaned sessions keep their stored
+  classification.
 - Excluded, trashed, and skipped session files are left alone.
 
 Mappings only mutate the session's `project` field; the rest of the session

docs/data.md

@@ -0,0 +1,75 @@
+---
+title: Data
+description: Project inventory and worktree mapping rules
+---
+
+The **Data** page is where you inspect and clean project classification across
+the whole archive. It is the home of the worktree mapping rules that previously
+lived in the Settings **Worktree mappings** section.
+
+Open it from the **Data** tab in the header, or follow a project link from the
+[Activity breakdown](/activity/#breakdowns). Deep links are stable:
+`/data?project_key=<key>` selects a project, and `/data?view=rules` opens the
+[Rules view](#rules).
+
+## Project Inventory
+
+The default view lists every project in the archive with its session, machine,
+agent, and working-directory counts plus first and last activity timestamps. A
+summary strip totals the projects, sessions, and the sessions currently governed
+by classification rules.
+
+- The table is sortable by any column and filterable by project name.
+- Projects targeted by enabled rules carry a rule badge; projects recorded as a
+  rule's original label carry an original-label badge.
+- Sessions whose stored project label is empty are grouped under a single
+  "unknown" row.
+- Activity bounds come from session timestamps only; rows without any recorded
+  timestamps show a no-activity state.
+
+Selecting a row opens the project workspace. Unknown `project_key` deep links
+show the full inventory with a non-blocking notice.
+
+## Create A Project Mapping
+
+The workspace creates [worktree project
+mappings](/configuration/#worktree-project-mappings) directly:
+
+- **Observed folders** lists every session folder associated with the selected
+  project. Each folder stays visible instead of being hidden behind a worktree
+  selector.
+- Selecting a folder opens one mapping row: **Folder path → Project**. The
+  suggested folder path covers that group's working directories and remains
+  editable, so it can be shortened to cover sibling folders when appropriate.
+- The **Project** typeahead suggests known projects and accepts a new name.
+  When the server normalizes the name (for example `sample-service` becomes
+  `sample_service`), the editor shows the stored form before you apply.
+- The **full archive impact** preview is live and authoritative: it counts
+  matching sessions across all dates for that machine. A prefix that touches
+  more than one existing project shows a warning with per-project counts —
+  usually a sign the prefix is too broad. A prefix matching zero sessions
+  cannot be applied.
+
+**Save and apply mapping** saves the rule and rewrites the matching sessions in
+one atomic step, then reloads the inventory. If the applied rule renamed the
+selected project, the selection follows the new name. If mappings changed
+between preview and apply, the apply is rejected and a fresh preview is
+required.
+
+## Rules
+
+The **Rules** toggle shows the worktree mapping rules for one machine at a time,
+with the same add, edit, apply, and delete controls that Settings previously
+offered — see
+[Worktree Project Mappings](/configuration/#worktree-project-mappings) for the
+full rule semantics. Each rule row also shows its **governed sessions** count
+(how many sessions the rule currently classifies) and the **original label**
+recorded when the rule was created through the mapping editor. Rule
+targets link back to the corresponding inventory row.
+
+## Read-Only Servers
+
+On a read-only server (`pg serve` or `duckdb serve`) the inventory, the
+candidate evidence, and the Rules table remain fully readable, but the editor
+and rule mutations are replaced by a notice: classification rules are managed
+from the writable archive that ingests the machine's sessions.

docs/internal/session-format-sources.md

@@ -1167,6 +1167,10 @@ Grok section and remove the explicit registry exception in the coverage test.
   context/last-turn/total statistics, without per-message cache or cost data.
   Agentsview emits one aggregate usage event and catalog-prices it when model
   identity is available.
+- **Project identity:** Metadata records `session_id`, `git_branch`, and
+  `environment.working_directory`. Agentsview recovers those independent
+  fields even when another optional metadata field is malformed, so a partial
+  parse cannot replace repository classification with generic fallbacks.
 - **Agentsview:** `internal/parser/vibe.go` and
   `internal/parser/vibe_provider.go`.
 

docs/screenshots/extract-db.sh

@@ -290,6 +290,8 @@ DELETE FROM session_project_identity_snapshots;
 DELETE FROM project_identity_observation_changes;
 DELETE FROM session_project_identity_snapshot_changes;
 DELETE FROM worktree_project_mappings;
+-- The mapping delete above journals tombstones, so clear that journal last.
+DELETE FROM worktree_project_mapping_changes;
 
 -- Keep generated screenshots independent of the source machine's hostname.
 -- The PostgreSQL fixture relabels a subset as work-desktop after push so the