Skip to content

feat(tools): user-defined right-column panels backed by a markdown-emitting script - #243

Open
frenchie4111 wants to merge 6 commits into
mainfrom
allow-custom-tools
Open

feat(tools): user-defined right-column panels backed by a markdown-emitting script#243
frenchie4111 wants to merge 6 commits into
mainfrom
allow-custom-tools

Conversation

@frenchie4111

Copy link
Copy Markdown
Collaborator

Summary

Adds user-defined tools to the right column. A tool is a directory in the worktree:

.harness/tools/<id>/
  tool.json     # { "title": "PR Comments", "script": "run.sh", "refresh": "auto" }
  run.sh        # executable; stdout is markdown and becomes the panel body

Discovery is worktree-local (not repoRoot, unlike .harness.json) so a branch can iterate on its own tooling and a PR that edits a tool exercises the new version.

Markdown is the entire contract. Rather than rendering document typography into a 280px column, SidebarMarkdown maps the markdown AST onto the vocabulary the built-in panels already use:

markdown renders as
# / ## section header (same styling as "Changed Files")
### / #### lighter section header
- list item a panel row
`code` / **bold** inline chips / emphasis, compacted
[label](harness:send?text=...) sends the text to the agent
[label](harness:file?path=...) opens the file
[label](harness:refresh) re-runs the tool
[label](https://...) opens externally
images, raw HTML dropped / inert

A tool physically cannot express anything outside the design system, so custom panels look native by construction.

Panel keys are namespaced tool:<id>, so they slot into the existing rightPanelOrder / hiddenRightPanels machinery in .harness.json for free — drag-to-reorder and hide both work with no new persistence. effectiveRightPanelOrder now takes the discovered keys, so a new tool appends without a config write and a deleted tool drops out of a saved order automatically.

Env exposed to the script: HARNESS_WORKTREE_PATH, HARNESS_BRANCH, HARNESS_REPO_ROOT, HARNESS_TOOL_DIR, HARNESS_TOOL_ID.

Two findings that changed the implementation

  • Don't spawn through a login shell. The first version copied runWorktreeScript's zsh -ilc. That cost 1–2s per run (a test timed out at 5s; the suite took 10.6s) and — worse — would let rc-file chatter (nvm banners, starship init) leak into stdout, which here is the panel body. Now the script is spawned directly and its own shebang picks the interpreter. path-fix.ts already merged the login-shell PATH at boot, so there was nothing to gain. Suite went to 1.05s. EACCES is translated to "run chmod +x".
  • useWatchedQuery refetches on every git change. A refresh: "manual" tool would still re-run constantly and hammer whatever API it talks to. Added revalidateOnFileChange and made fallbackPollMs: 0 mean "no polling", so manual tools run only on mount and on the refresh button.

Known gaps

Called out deliberately rather than half-implemented:

  • The collapsed icon strip ignores custom tools.
  • There's no trust prompt before running a checked-in script — cloning a repo with a .harness/tools/ directory and opening a worktree runs it.
  • The 20s timeout kills the script but not its grandchildren.

Demo tool (not committed)

.harness/tools/ isn't gitignored, so shipping a demo would give every contributor a live panel invoking gh on mount. Left it out; paste this into a repo to try it:

.harness/tools/pages/tool.json

{ "title": "Pages", "script": "run.sh", "refresh": "auto" }

.harness/tools/pages/run.sh (chmod +x)

#!/usr/bin/env bash
set -uo pipefail
echo "## Top level"
for f in *.html; do
  [ -e "$f" ] || continue
  title=$(sed -n 's/.*<title>\(.*\)<\/title>.*/\1/p' "$f" | head -1)
  echo "- [${title:-(no title)}](harness:file?path=$f) \`$(du -h "$f" | cut -f1)\`"
done
echo
echo "## SEO"
missing=$(grep -Lri '<meta name="description"' --include="*.html" . | wc -l | tr -d ' ')
if [ "$missing" = "0" ]; then echo "- Every page has a **meta description**"
else echo "- **$missing** page(s) missing a meta description"; fi
echo
echo "[Ask Claude to audit SEO](harness:send?text=Audit+the+SEO+of+every+page)"

Test plan

  • npm run typecheck
  • npx electron-vite build
  • npx vitest run — 11 new tests in src/main/tools.test.ts (discovery defaults, malformed manifest, script-path escape rejection, env exposure, non-zero exit, missing/non-executable script) + 3 in repo-configs.test.ts for tool-aware ordering
  • Manual: drop the demo tool into a repo, confirm the panel renders, reorders, hides, and that the action links fire

🤖 Generated with Claude Code

frenchie4111 and others added 6 commits August 20, 2026 07:24
…itting script

A tool is a directory under <worktree>/.harness/tools/<id>/ with a
tool.json (static title, so the panel has a header before the script has
ever run) and an executable script whose stdout is rendered as markdown.
The directory name becomes a `tool:<id>` panel key, so custom tools flow
through the existing per-repo order/visibility config and sit alongside
the built-ins in the gear menu.

Markdown maps onto the built-in panels' vocabulary rather than document
typography — headings become the ChangedFilesPanel section header, list
items become rows, and `harness:` links drive send-to-agent / open-file.
A narrow contract is what keeps custom panels from drifting out of the
design system.

Tools are spawned directly on their shebang rather than through a login
shell: `-ilc` cost 1-2s per run and would let rc-file chatter leak into
stdout, which here is the panel body.

useWatchedQuery grows `revalidateOnFileChange` and treats fallbackPollMs
of 0 as "no polling", so a tool marked refresh:"manual" runs only on
mount and on the refresh button — tool scripts routinely hit the network
and the built-in panels' cadence would hammer an API.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… contract

Discovering that custom tools exist at all required reading the source.
Adds an entry at the bottom of the right column's panel menu that sends
the full authoring contract to the agent in the active worktree, reusing
the same onSendToAgent path the harness:send link verb already uses.

The contract lives in src/shared/tools.ts next to the types it describes,
so the manifest fields, env vars, and markdown mapping have one source of
truth rather than drifting from a copy in the renderer. Exporting it from
shared also leaves the door open to serving it over MCP later without a
second copy.

Costs nothing per session (unlike an MCP tool description, which every
agent would pay for whether or not the user ever writes a tool) and is
discoverable by the user, who is the one who doesn't yet know the feature
exists.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Seeding the active tab's chat was too implicit — the contract landed as a
message in whatever conversation happened to be running there, possibly
mid-task, with no indication that was about to happen. Opening the
new-worktree screen with the branch and kickoff prompt pre-filled shows
the user exactly what's going to run before anything does, and both
fields stay editable.

It also fits how tools resolve: discovery is worktree-local, so the panel
goes live on the new branch while the agent iterates on it, and reaches
everyone else on merge.

NewWorktreeScreen gains initialBranch / initialPrompt, following the
existing initialPRNumber prefill pattern. The prefill is transient
renderer state cleared when the screen closes, alongside newWorktreeRepo
and newWorktreeInitialPRNumber.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
react-markdown sanitizes hrefs whose protocol isn't in its safe list, so
every harness:send / harness:file / harness:refresh link arrived with
href="" and fell through to the inert-text branch. Action rows rendered
as plain, unclickable text — the feature's whole interactive surface was
dead on arrival.

Allow the harness: scheme through and defer to defaultUrlTransform for
everything else, so the javascript: protection stays intact.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The panel body carried py-1 and the list wrapper py-0.5, neither of which
the built-in panels have, so a tool's first section header sat lower than
Changed Files' does and the whole panel looked misaligned against its
neighbours. Match ChangedFilesPanel exactly: a bare scroll container, and
rows as direct siblings of their header.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Main renamed the product, so the config file is now .ness.json with
.harness.json read as a legacy fallback. Custom tools shipped the old
brand throughout their user-facing contract: the .harness/tools
directory users create, the harness: link scheme, and the HARNESS_*
env vars handed to every script.

Unlike .harness.json there's nothing to preserve here — the feature is
unreleased, so no dual-read path is needed and none is added. Renaming
now costs one commit; renaming after people have tool directories on
disk costs a migration.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant