Skip to content

defer line counting from PathIndex build to manifest - #6

Merged
xobotyi merged 5 commits into
mainfrom
dev-master
May 27, 2026
Merged

defer line counting from PathIndex build to manifest#6
xobotyi merged 5 commits into
mainfrom
dev-master

Conversation

@xobotyi

@xobotyi xobotyi commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • PathIndex walk is now stat-only — no file opens, no byte reads during index build
  • Line counting and binary content detection moved to buildManifest(), which runs only for changed files
  • readFile() falls through to direct filesystem read when file is not in the index
  • buildManifest parallelized with fetchDiffs and resolveInstructions
  • Bump version to 0.1.7

Performance

Benchmarked on two Gaijin repos:

Repo Files Before After Speedup
Dagor 172K 22.3s 0.97s 23x
Mairon 245K 32.4s 1.99s 16x

Test plan

  • 271 tests pass (3 new for countFileLines)
  • vp check clean
  • Verify session_start completes within MCP timeout on large repos

xobotyi added 4 commits May 26, 2026 19:26
The inline !echo with ${VAR:-default} syntax triggered
Claude Code permission rejection on some clients.
Replaced with printenv (no expansion) plus TeamCreate
tool availability as a secondary gate.
PathIndex walk now does stat-only -- no file opens, no byte
reads. Line counting and binary content detection move to
buildManifest(), which runs only for changed files (~10-50).

172K-file repo: 22s -> 1s for index build.

Other changes in this commit:
- readFile falls through to direct fs read when file is
  not in the index (supports reading outside indexed paths)
- buildManifest runs in parallel with fetchDiffs and
  resolveInstructions
- Pool class and #countLines removed; standalone
  countFileLines() exported for on-demand use

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves session startup performance by making PathIndex construction stat-only and deferring line counting/binary probing to manifest building for changed files, while also parallelizing startup work and bumping plugin/package versions.

Changes:

  • Make PathIndex.new() stat-only (no file opens/reads); move line counting into buildManifest() via new countFileLines().
  • Parallelize buildManifest() with diff fetching and instruction resolution during Session.start().
  • Allow RepoFs.readFile() to fall back to direct filesystem reads when a path is missing from the index; bump versions to 0.1.7.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
claude/kensai/skills/review/SKILL.md Updates environment check command for agent teams enablement.
claude/kensai/mcp/review/src/session/session.ts Defers line counting to manifest build; parallelizes manifest with other startup tasks.
claude/kensai/mcp/review/src/repofs/repofs.ts Changes readFile() to fall back to filesystem when path isn’t indexed.
claude/kensai/mcp/review/src/repofs/pathindex/pathindex.ts Makes index build stat-only; introduces exported countFileLines().
claude/kensai/mcp/review/src/repofs/pathindex/pathindex.test.ts Updates PathIndex expectations and adds tests for countFileLines().
claude/kensai/mcp/review/package.json Bumps MCP package version to 0.1.7.
claude/kensai/.claude-plugin/plugin.json Bumps plugin version to 0.1.7.
.claude-plugin/marketplace.json Bumps marketplace version to 0.1.7.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread claude/kensai/mcp/review/src/repofs/repofs.ts
Comment thread claude/kensai/mcp/review/src/session/session.ts
Comment thread claude/kensai/mcp/review/src/repofs/pathindex/pathindex.ts
Comment thread claude/kensai/skills/review/SKILL.md
Direct filesystem fallback in readFile bypassed the
WALK_EXCLUDE_DIRS guard, allowing reads into .git/.
Now checks the first path segment against the exclusion
set before falling through to stat.

Also fixes isBinary JSDoc to reflect extension-based
detection.
@xobotyi
xobotyi merged commit 9a5d6be into main May 27, 2026
5 checks passed
@xobotyi
xobotyi deleted the dev-master branch May 27, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants