Phase 17/18: reliability + observability pass#33
Conversation
Rename Phase 17/18 around write-mode reliability and observability, add 17-02/18-02 plans and summaries, and align state/project focus to single-replica pragmatic hardening.
Scan regex secrets against staged additions per file to allow safe secret removals, improve no-change refusal guidance, and normalize evidence/runbook fields for easier deliveryId-based operations.
![kodiai[bot]](https://avatars.githubusercontent.com/in/2832814?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR focuses on improving write-mode reliability (fewer secret-scan false positives) and strengthening observability/operational verification by standardizing evidence log fields and enhancing runbooks/smoke templates.
Changes:
- Secret scanning now evaluates staged additions per file (allowing removal of existing secret-like content without refusals).
- Evidence bundle logs now include consistent repo context fields (
owner,repoName,repo) for easier correlation. - Updated runbooks/smoke docs and added planning artifacts for phases 17/18 execution.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/jobs/workspace.ts | Changes secret-scan implementation to scan staged additions per file; refactors added-line extraction. |
| src/handlers/review.ts | Adds owner/repoName to review evidence bundle logs for better correlation. |
| src/handlers/mention.ts | Adds owner/repoName to write-mode evidence logs; improves refusal UX for write-policy-no-changes. |
| src/handlers/mention.test.ts | Adds regression test ensuring removal of existing secret-like content is allowed. |
| docs/smoke/xbmc-kodiai-write-flow.md | Adds a release evidence capture template for smoke runs. |
| docs/runbooks/mentions.md | Updates evidence bundle fields/outcomes and adds operator query snippets + quick map. |
| .planning/phases/18-observability-verification/18-02-SUMMARY.md | Records Phase 18 Plan 02 execution summary and verification. |
| .planning/phases/18-observability-verification/18-02-PLAN.md | Adds Phase 18 Plan 02 execution plan definition. |
| .planning/phases/18-observability-verification/18-01-SUMMARY.md | Renames Phase 18 naming/summary to observability-verification framing. |
| .planning/phases/18-observability-verification/18-01-PLAN.md | Renames Phase 18 plan metadata and updates dependencies/objective wording. |
| .planning/phases/17-write-mode-reliability/17-02-SUMMARY.md | Records Phase 17 Plan 02 execution summary and verification. |
| .planning/phases/17-write-mode-reliability/17-02-PLAN.md | Adds Phase 17 Plan 02 execution plan definition. |
| .planning/phases/17-write-mode-reliability/17-01-SUMMARY.md | Renames Phase 17 naming/summary to reliability framing. |
| .planning/phases/17-write-mode-reliability/17-01-PLAN.md | Renames Phase 17 plan metadata and updates objective wording. |
| .planning/STATE.md | Updates current phase/progress and completed plan counts to reflect phases 17/18 completion. |
| .planning/PROJECT.md | Updates active milestone wording to emphasize reliability + observability scope. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| function extractAddedLines(patch: string): string[] { | ||
| return patch | ||
| .split("\n") | ||
| .filter((l) => l.startsWith("+") && !l.startsWith("+++")) |
There was a problem hiding this comment.
extractAddedLines filters out any added line starting with +++, which is broader than the diff header it’s trying to exclude (+++ b/<path>). This can drop legitimate added content lines that begin with +++ and potentially let a secret bypass the staged-additions scan. Consider filtering only diff headers (e.g., l.startsWith('+++ ')) or parsing the patch format more robustly.
| .filter((l) => l.startsWith("+") && !l.startsWith("+++")) | |
| .filter((l) => l.startsWith("+") && !l.startsWith("+++ ")) |
| @@ -190,6 +212,22 @@ Common refusal reasons: | |||
| - `write-policy-secret-detected`: suspected secret present in staged diff | |||
There was a problem hiding this comment.
The runbook still says write-policy-secret-detected is for a “suspected secret present in staged diff”, but the implementation now scans staged additions only (and the error message says “staged additions”). Update this description to avoid misleading operator guidance.
| - `write-policy-secret-detected`: suspected secret present in staged diff | |
| - `write-policy-secret-detected`: suspected secret present in staged additions |
Summary
Verification