Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/ISSUE_TEMPLATE/guide-usage-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Guide usage report
description: >-
Share an opt-in Guide telemetry summary. The report contains only tool-call
counts, outcomes, durations, and timestamps — never tool arguments, queries,
or repository content. Submitting posts it as a public issue.
title: "Guide usage report"
labels:
- usage-report
body:
- type: markdown
attributes:
value: >-
Thanks for sharing! This report comes from `rac mcp-stats --share` and
helps decide where Guide investment goes. Review the prefilled summary
below before submitting — it should contain only counts and
timestamps, and it will be publicly visible.
- type: textarea
id: report
attributes:
label: Telemetry summary
description: The JSON summary produced by `rac mcp-stats --share`.
render: json
validations:
required: true
- type: textarea
id: context
attributes:
label: Anything to add? (optional)
description: >-
How you use Guide, what worked, what didn't — anything that gives the
numbers context.
validations:
required: false
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
- name: explorer
paths: "tests/test_explorer_adapter.py tests/test_explorer_app.py tests/test_explorer_cli.py tests/test_explorer_commands.py tests/test_explorer_editor.py tests/test_explorer_isolation.py tests/test_explorer_workspace.py"
- name: mcp
paths: "tests/test_mcp_server.py tests/test_mcp_tools.py tests/test_mcp_isolation.py"
paths: "tests/test_mcp_server.py tests/test_mcp_tools.py tests/test_mcp_isolation.py tests/test_mcp_telemetry.py"
- name: relationships
paths: "tests/test_relationships.py tests/test_relationships_cmd.py tests/test_relationship_validation.py"
- name: resolve
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ details, release history over commit history.

### Added

- Opt-in Guide telemetry (v0.10.4): `rac mcp --telemetry` records tool-call
counts and metadata — never arguments or repository content — to a local
log under `$XDG_STATE_HOME/rac/`; off by default and announced on stderr
when on. Tool responses are byte-identical with telemetry on and off. A
new `rac mcp-stats` command summarizes the log (`--json` is the shareable
export; `--share` prints a prefilled GitHub usage-report issue URL you
review and submit yourself — RAC contains no network code).

- Review impact and the first-run editor (v0.8.11): every `rac review`
finding now carries an `impact` sentence — why it matters — owned by Core
and present in the JSON contract (additive field; `schema_version`
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ to the corpus artifact and they load through the imports below.

## Working corpus

- Current series: `rac/roadmaps/v0.10.x-guide/` (next up: v0.10.3)
- Current series: `rac/roadmaps/v0.10.x-guide/` (next up: v0.10.4)
- Previous series: `rac/roadmaps/v0.8.x-explorer/` (complete through v0.8.10)
- Decisions (ADRs): `rac/decisions/`
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ Lore asks you to trust it with your product knowledge, so it holds itself to the
- **No AI in the core.** Retrieval is deterministic: the same repo state and the same query always return the same result. The reasoning is your agent's job; Lore's job is to hand it the facts.
- **It dogfoods itself.** Lore's own planning corpus under [`rac/`](https://github.com/tcballard/requirements-as-code/tree/main/rac) is validated by RAC in CI — if the tool's rules break the tool's own artifacts, the build fails.
- **Output is a contract.** Golden tests pin CLI and MCP output; any change to what the tools return is reviewed as a product change.
- **Telemetry is opt-in, local-only, and content-free.** Nothing is recorded without an explicit flag, events never include your arguments or repository content, and nothing leaves your machine unless you submit a report yourself — Lore contains no network code.

## Documentation

Expand Down
45 changes: 44 additions & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CLI Reference

RAC ships a single command, `rac`, with eighteen subcommands. This page documents each
RAC ships a single command, `rac`, with twenty subcommands. This page documents each
one: its purpose, inputs, outputs, and exit codes.

```bash
Expand Down Expand Up @@ -464,6 +464,49 @@ Without the extra, `rac explorer` prints the install hint above and exits `2`.

---

## mcp

Serve RAC repository knowledge to coding agents over MCP (stdio). The four
read-only tools, client configuration, and team setup are documented in the
[MCP server guide](mcp.md).

```bash
rac mcp --root /path/to/repo
rac mcp --root /path/to/repo --telemetry
```

- **`--root PATH`** — repository root to serve (default: current directory)
- **`--telemetry`** — record tool-call counts and metadata (never arguments
or content) to a local log under `$XDG_STATE_HOME/rac/` (default
`~/.local/state/rac/guide-telemetry.jsonl`); off by default, announced on
stderr when on
- **Exit codes:** `0` server shutdown on client disconnect · `2` `--root` is
not a directory

---

## mcp-stats

Summarize the local Guide telemetry log: events, sessions, first and last
timestamps, and per-tool calls, errors, truncation, and average duration.
An empty or missing log is a valid answer — telemetry is opt-in and off by
default.

```bash
rac mcp-stats # human summary
rac mcp-stats --json # the same summary as JSON (the shareable export)
rac mcp-stats --share # prefilled GitHub usage-report issue URL
```

`--share` prints a URL that opens a prefilled usage-report issue containing
only counts and timestamps; you review and submit it in your own browser —
RAC sends nothing itself. `--json` and `--share` are mutually exclusive.

- **Exit codes:** `0` summary produced (including from an empty or missing
log) · `2` usage error

---

## new

Create a new artifact from its canonical bundled template, with a
Expand Down
52 changes: 51 additions & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,57 @@ Two caveats:
carry the same pointer in their native convention (for example
`.cursor/rules`); the prompt artifact remains the single source of truth.

## 7. Troubleshooting
## 7. Telemetry (opt-in)

Guide records nothing by default. If you want to see whether it is actually
being used — and help decide where Guide investment goes — opt in with an
explicit flag in your client's server configuration:

```json
{
"mcpServers": {
"rac-guide": {
"command": "rac",
"args": ["mcp", "--root", "/path/to/repo", "--telemetry"]
}
}
}
```

When enabled, each tool call appends one JSON line to a local log
(`~/.local/state/rac/guide-telemetry.jsonl`, or under `$XDG_STATE_HOME`):

```json
{"schema_version": "1", "ts": "2026-06-12T14:03:22.512Z", "session": "a3f29c1b",
"tool": "search_artifacts", "outcome": "ok", "duration_ms": 12, "truncated": false}
```

What is recorded: timestamp, a random per-session id, the tool name, whether
the call succeeded, how long it took, and whether the response was truncated.
What is **never** recorded: tool arguments, artifact IDs, search queries,
file paths, or any repository content. The server announces the log path on
stderr at startup, so enablement is never silent.

Read the log back any time:

```bash
rac mcp-stats # human summary: events, sessions, per-tool usage
rac mcp-stats --json # the same summary as JSON — this is the export
```

If you want to share your usage with the project (early reports directly
shape Guide's roadmap):

```bash
rac mcp-stats --share
```

This prints a prefilled GitHub issue URL. Open it, review the report — counts
and timestamps only — and submit it with your own account. RAC never sends
anything anywhere; building a URL is string formatting, and transmission
belongs to you and your browser. Submitted reports are public issues.

## 8. Troubleshooting

### Server not listed in the client

Expand Down
173 changes: 173 additions & 0 deletions rac/decisions/adr-040-guide-local-telemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
---
schema_version: 1
id: RAC-KTY25D945HYK
type: decision
---
# ADR-040: Guide Local Telemetry

## Status

Accepted

## Category

Product

## Context

Nothing today answers whether the Guide is actually used, or which of
its four tools earn their place. Search-quality work (v0.10.3), tool
description revisions, and future surface decisions all want usage
evidence; without it they are guesses.

The obvious industry answer — automatic remote telemetry — collides
with recorded positions. ADR-035 forbids a mandatory RAC cloud
dependency, and the trust-transparency requirement names hosted
infrastructure a non-goal: RAC's credibility rests on being boring,
deterministic, and inspectable. A tool that quietly phones home
forfeits exactly the trust the Guide asks an agent's operator to
extend. ADR-032 adds a technical constraint: tool output is a pure
function of repository bytes and tool input, so observability must
never feed back into a response.

The need is real and the constraints are recorded. The decision is the
shape of telemetry that satisfies both.

## Decision

Guide telemetry is opt-in, default-off, local-only, and content-free.

- Enablement is an explicit `--telemetry` flag on `rac mcp` — visible
in the client's server configuration, never persistent, never implied
by a config file or environment variable. When enabled, the server
announces on stderr what is recorded and where.
- Each tool call appends one JSON line to
`$XDG_STATE_HOME/rac/guide-telemetry.jsonl` (default
`~/.local/state/rac/guide-telemetry.jsonl`). The event schema is
pinned: `schema_version`, `ts` (ISO 8601 UTC), `session` (random
per-process hex), `tool`, `outcome` (`ok` | `error` | `exception`),
`error` (structured error code, only when outcome is `error`),
`duration_ms`, `truncated`. Tool arguments, artifact IDs, query
strings, paths, and repository content are never recorded. Adding a
field is a recorded decision, not a patch.
- Recording is write-only observability outside the request/response
contract: the payload returns unchanged, the log is never an input to
a response, and a recorder that cannot write disables itself silently
— telemetry failure never breaks a tool call.
- Read-back is local: `rac mcp-stats` summarizes the log; its `--json`
output is the export. Sharing is a deliberate act: `--share` prints a
prefilled GitHub issue URL for the repository's usage-report issue
form, and the user reviews and submits it in their own browser.
- RAC contains no network code. Building a URL is string formatting;
transmission belongs to the user.

## Consequences

### Positive

- Usage evidence becomes available without compromising the recorded
trust posture; future surface decisions can cite data.
- Everything is inspectable: the log is plain JSONL on the user's disk,
the export is the same bytes the user reads, and the share payload is
reviewed before submission.
- The determinism contract survives intact and remains testable
byte-for-byte with a recorder attached.

### Negative

- Opt-in means sparse, self-selected data; adoption questions get a
floor, not a census.
- Reports arrive as public GitHub issues, readable by anyone — counts
and timestamps only, but public.
- A second generation of telemetry wants (latency percentiles, query
shapes) is foreclosed until a new decision revisits the schema.

### Risks

- Scope creep toward recording content under diagnostic pressure.
Mitigation: the schema is pinned here and in the contract battery;
the named absent fields are a test, not a comment.
- A future change accidentally routes the log into a response.
Mitigation: payload-stability tests compare responses byte-for-byte
with and without a recorder.

## Alternatives Considered

### Automatic remote telemetry

Send events to a hosted endpoint by default, with opt-out.

#### Advantages

- Real adoption data across the install base.

#### Disadvantages

- Directly contradicts ADR-035 and the trust-transparency non-goal;
requires hosted infrastructure, a privacy policy, and the very
category of trust conversation RAC exists to avoid.

### Environment-variable or config-file enablement

Enable via `RAC_TELEMETRY=1` or a user config file.

#### Advantages

- Survives across sessions without editing client server args.

#### Disadvantages

- Persistent invisible state: telemetry silently on long after the
user forgot setting it. The flag keeps enablement where the server
is configured and visible every time.

### No telemetry at all

Keep guessing from issues and conversations.

#### Advantages

- Zero code, zero trust surface.

#### Disadvantages

- The Guide's product questions stay unanswerable; investment in
search quality and descriptions proceeds on anecdote.

Opt-in local recording with user-driven sharing is selected.

## Relationship to Other Decisions

- ADR-032 (stateless reads): telemetry is write-only observability
outside the request/response contract; tool output remains a pure
function of repository bytes and tool input.
- ADR-035 (user-managed credentials, no RAC cloud): no hosted
endpoint, no transmission by RAC.
- ADR-031 (in-process Core consumption): the recorder lives in the
server layer; Core and services stay telemetry-unaware.
- ADR-034 (structured errors): outcome classification reads the
structured error code the tools already return.
- ADR-013 (Git as the state store): the telemetry log is user-machine
state, not repository state; it never enters the corpus.

## Success Measures

- Driving the four tools with `--telemetry` yields events containing
no arguments or content; without the flag, no file is touched.
- Existing Guide goldens pass unchanged in the implementing change.
- At least one early-user report arrives through the share flow and is
useful for a surface decision.

## Review Date

Review when the first real diagnostic need exceeds the pinned schema,
or one quarter after release if no user has opted in.

## Related Requirements

- rac-agent-context-guide
- rac-trust-transparency

## Related Roadmaps

- v0.10.4-guide-telemetry
Loading
Loading