Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions docs/automations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Always-on automations

Two Devin Automations run against this repository. Both are defined in the Devin
Automations UI; this directory holds their reviewable source of truth: the design,
the guardrails, and the exact prompt each one runs.

| Automation | Trigger | Human checkpoint |
| --- | --- | --- |
| [Weekly drift report](weekly-drift-report.md) | Schedule — Mondays 06:00 UTC | Nobody may act on the queue until a maintainer picks items; the automation never opens remediation PRs |
| [Bug triage](bug-triage.md) | GitHub issue labeled `bug` in this repo | Anything above a trivial fix waits for an explicit "go ahead" comment on the issue |

Both automations run as their creator (creator permissions, visible to the creator
and org admins) and deliver everything to GitHub — no Slack, no email fan-out.

Prompts live in [`prompts/`](prompts/) and are the text pasted into each automation.
Change the prompt file and the automation together, in the same PR, so the repo never
disagrees with what actually runs.

## Scanners

Each repository in scope carries its own read-only scanner that emits the same
normalized JSON, so the weekly report is reproducible rather than hand-assembled:

- `timesheet-app` — [`scripts/drift_scan.py`](../../scripts/drift_scan.py) (npm workspaces, GitHub advisories via `npm audit`)
- `petclinic-microservices` — `scripts/drift_scan.py` (Maven coordinates, Maven Central, OSV.dev)
- `timesheet-infra` — `scripts/drift_scan.py` (Terraform providers, registry modules, `required_version`)

The shared item schema, ranking formula, and sizing rubric are documented in
[weekly-drift-report.md](weekly-drift-report.md#normalized-scanner-output).
90 changes: 90 additions & 0 deletions docs/automations/bug-triage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Automation 2 — Ticket-triggered bug triage

Every new bug report in `timesheet-app` gets a reproduction attempt, a named code path
and a severity before a human reads it — and nothing above a trivial fix happens
without a human saying so.

- **Scope:** `timesheet-app` only (`backend/src/`, `frontend/src/`)
- **Prompt:** [`prompts/bug-triage.md`](prompts/bug-triage.md)
- **Delivery:** a comment on the triggering issue

## Trigger

GitHub issue event on `Cognition-Partner-Workshops/timesheet-app` where the issue
carries the label `bug`:

- `action = opened` with the `bug` label, or
- `action = labeled` with `label.name = bug` (so triage still fires when a maintainer
triages an existing report into the bug queue).

Issues without the `bug` label are ignored; the label is the intake gate, which keeps
questions and feature requests out of the reproduction loop.

## Procedure

1. **Read the report.** Extract expected vs. actual behaviour, steps, environment and
any error text. If the report has no observable symptom at all, say so and ask the
reporter for the missing detail instead of guessing.
2. **Reproduce first.** Install, run `cd backend && npm test` and the relevant flows;
start `backend` (port 3001) and `frontend` (port 5173) when the symptom is
user-visible. Reproduction is attempted *before* any severity claim.
3. **Locate the responsible code path.** Follow the failing behaviour to specific files
and line ranges under `backend/src/` or `frontend/src/`, then state the call chain
(route → handler → query, or component → hook → API call) and the root cause.
4. **Assess severity** with the rubric below.
5. **Post findings** as one comment on the issue: reproduction result, code path,
root cause, severity with justification, and a proposed fix with its size.
6. **Stop and ask** unless the fix is trivial (see below).

## Severity rubric

| Severity | Criteria |
| --- | --- |
| `S1 critical` | data loss or corruption, auth/authorization bypass, secret exposure, or the app unusable for all users |
| `S2 high` | a core timesheet flow (create/edit/submit/approve entries, login) broken with no workaround |
| `S3 medium` | a non-core flow broken, or a core flow degraded but with a workaround |
| `S4 low` | cosmetic, copy, or a rare edge case with no functional impact |
| `unreproducible` | the symptom did not occur — reported with exactly what was tried, never silently downgraded |

Severity is reported with the evidence that justifies it (failing test, HTTP status,
console error, screenshot), so the label is auditable rather than a feeling.

## Trivial vs. everything else

**Trivial** — may be fixed in the same run, in a PR, without waiting:

- a typo, a wrong string/label, a wrong constant or off-by-one in one expression
- a missing null/undefined guard in one function
- confined to a single file, under roughly 10 changed lines, no schema/API/auth change,
and covered by existing tests that pass

**Everything else waits**: schema or migration changes, auth or permission logic,
API contract changes, shared component or state refactors, anything touching more than
one file or needing a new test to prove the fix, and anything the automation is not
certain about. Uncertainty counts as non-trivial.

## Guardrails

- **Reproduce before diagnosing.** No severity, no root cause and no fix without either
a reproduction or an explicit statement that reproduction failed and why.
- **No fix above trivial without confirmation** — see the human checkpoint.
- **Never weaken tests.** No editing, skipping or loosening a test to make a failure go
away; if a test looks wrong, say so and stop.
- **No production behaviour changes as a side effect** of triage: no dependency bumps,
no refactors, no formatting sweeps, no unrelated file edits.
- **No secrets in comments.** No tokens, connection strings, `.env` contents or real
user data; redact anything sensitive pulled from logs.
- **No destructive actions.** Never delete or force-push branches, never close or
relabel the reporter's issue, never touch production or live infrastructure.
- **One comment per triage run,** so the issue stays readable.
- **Stay in `timesheet-app`.** A cause outside `backend/src/` or `frontend/src/`
(infrastructure, upstream dependency) is reported, not fixed here.
- **Time-boxed.** If reproduction does not succeed within the run, post what was tried
and what is missing rather than continuing to guess.

## Human checkpoint

**The confirmation gate on the issue.** For anything above trivial, the automation posts
its findings plus a proposed fix and size, then explicitly asks for confirmation and
stops. It resumes only when a maintainer replies on the issue with an explicit go-ahead.
Trivial fixes still land as a PR for review — nothing is merged by the automation.
91 changes: 91 additions & 0 deletions docs/automations/prompts/bug-triage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Prompt — Bug triage

This is the exact prompt run by the "Bug Triage — timesheet-app" automation (trigger:
GitHub issue labeled `bug` in `Cognition-Partner-Workshops/timesheet-app`). Keep this file
and the automation in sync in one PR. Design and rubric:
[../bug-triage.md](../bug-triage.md).

---

A bug was reported in `Cognition-Partner-Workshops/timesheet-app`. The triggering issue
(number, title, body, labels) is in the trigger event payload. Triage it, post your
findings on that issue, and do not fix anything above trivial without confirmation.

## 1. Understand the report

Read the issue with the built-in git tools, including existing comments. Extract expected
behaviour, actual behaviour, steps, environment and error text. If there is no observable
symptom to chase, post what is missing and ask the reporter — do not invent a repro.

## 2. Reproduce before you diagnose

Work on a clean checkout of `main`.

- Backend: `cd backend && npm ci && npm test`, then `npm run dev` (port 3001) and exercise
the reported request path directly.
- Frontend: `cd frontend && npm ci && npm run lint && npm run build`, then `npm run dev`
(port 5173) and drive the reported UI flow in the browser.
- Capture the evidence: failing test output, HTTP status and response body, console error,
or a screenshot of the broken UI.

If you cannot reproduce it, say so explicitly, list exactly what you tried and what you
would need from the reporter. Never mark an unreproducible report as low severity by
default.

## 3. Locate the responsible code path

Trace the symptom to concrete files and line ranges under `backend/src/` or
`frontend/src/` and state the chain (route → handler → query, or component → hook → API
call) plus the root cause: which expression or missing condition produces the observed
behaviour. Cite `path:line` for every claim. If the cause is outside those two trees
(infrastructure, an upstream dependency), report that and stop there.

## 4. Assess severity

- `S1 critical` — data loss/corruption, auth or authorization bypass, secret exposure, or
the app unusable for everyone
- `S2 high` — a core flow (create/edit/submit/approve timesheet entries, login) broken
with no workaround
- `S3 medium` — a non-core flow broken, or a core flow degraded with a workaround
- `S4 low` — cosmetic/copy or a rare edge case with no functional impact
- `unreproducible` — symptom did not occur; report what was tried

Justify the severity with the evidence from step 2.

## 5. Post findings — one comment on the triggering issue

Include, in this order: reproduction result (with the command/flow used and its output),
the responsible code path with `path:line` references, the root cause, the severity with
its justification, and a proposed fix with its size and risk.

## 6. Fix only what is trivial

**Trivial** (fix it in this run, in a PR — do not merge it): a typo, wrong string, wrong
constant, off-by-one in a single expression, or a missing null/undefined guard — confined
to one file, roughly under 10 changed lines, no schema/API/auth change, with existing
tests passing afterwards. Run `cd frontend && npm run lint`, `cd backend && npm test` and
`cd frontend && npm run build` before opening it, and link the PR in your comment.

**Everything else**: do not change any code. Post the proposed fix, its size, its risk and
the files it would touch, then ask on the issue for explicit confirmation before
proceeding, and stop. This includes schema/migration changes, auth or permission logic,
API contract changes, shared component or state refactors, anything touching more than one
file, anything needing a new test to prove the fix, and anything you are unsure about —
uncertainty counts as non-trivial.

## Guardrails

- Never edit, skip or loosen a test to make a failure disappear. If a test looks wrong,
say so and stop.
- No unrelated changes: no dependency bumps, no refactors, no formatting sweeps.
- No secrets or real user data in the comment; redact anything sensitive from logs.
- Do not close, relabel or edit the reporter's issue, do not force-push or delete
branches, and never touch live infrastructure.
- Exactly one findings comment per run.
- Time-box the reproduction: post what you have with the gaps named rather than guessing
indefinitely.

## Human checkpoint

For anything above trivial, the run ends at "here is the fix I propose — confirm and I'll
do it". Resume only on an explicit go-ahead from a maintainer on the issue.
92 changes: 92 additions & 0 deletions docs/automations/prompts/weekly-drift-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Prompt — Weekly drift report

This is the exact prompt run by the "Weekly Drift Report — 3 repos" automation
(schedule: Mondays 06:00 UTC). Keep this file and the automation in sync in one PR.
Design, ranking and guardrail rationale: [../weekly-drift-report.md](../weekly-drift-report.md).

---

Produce this week's dependency drift report across three repositories and publish it,
with a ranked and sized remediation queue, as a single GitHub issue.

Repositories (clone each with the built-in git tools if it is not already on the box):

- `Cognition-Partner-Workshops/timesheet-app`
- `Cognition-Partner-Workshops/petclinic-microservices`
- `Cognition-Partner-Workshops/timesheet-infra`

## 1. Scan

Run each repository's own read-only scanner on a clean checkout of `main` and keep the
JSON output:

- `timesheet-app`: `python3 scripts/drift_scan.py --out drift-timesheet-app.json`
(run `npm ci` in `backend/` and `frontend/` first so `npm outdated`/`npm audit` see an
installed tree; if `npm ci` fails, record it as a scan error and continue)
- `petclinic-microservices`: `python3 scripts/drift_scan.py --out drift-petclinic.json`
(add `--skip-maven` if `./mvnw dependency:list` cannot resolve; that fallback parses the
poms directly and is recorded in `errors`)
- `timesheet-infra`: `python3 scripts/drift_scan.py --out drift-infra.json`

Do not modify anything: no installs that write to `package.json`, no lockfile updates, no
`pom.xml` edits, no `terraform init`/`plan`/`apply`, no `.tf` or `.terraform.lock.hcl`
edits, no source changes. Do not create branches or PRs in any of the three repositories.

If a scanner exits non-zero or its `errors` array is non-empty, the report is **partial**.
Say so explicitly and list every error. Never present an unreachable registry or advisory
API as "no findings".

## 2. Report

Open one issue in `Cognition-Partner-Workshops/timesheet-app`:

- Title: `Drift report — <YYYY-MM-DD>`
- Label: `drift-report` (create the label if it does not exist)

Body sections, in this order:

1. **Scan health** — per repository: scanner status, fallbacks used, and every `errors`
entry verbatim. State plainly whether the report is complete or partial.
2. **Summary** — totals by severity, by repository, and by size (`XS`/`S`/`M`/`L`).
3. **Advisories** — every item with `advisory_count > 0`, worst severity first, with
advisory IDs, links, affected component, location, current and target version.
4. **Version drift** — dependency/plugin items grouped by repository, `current → latest`
with the major/minor/patch gap.
5. **Terraform** — for each root under `timesheet-infra/terraform/`: provider and module
constraint, lock-file pin, newest release. Call out where the declared constraint
itself blocks the newest release (`constraint_allows_latest: false`) and where no
`.terraform.lock.hcl` pins the version.
6. **Remediation queue** — see below.
7. **Raw data** — the three JSON documents in fenced blocks (truncate the `advisories`
arrays if the body would exceed GitHub's size limit, and say that you truncated).

## 3. Remediation queue (input to the Track 2 fan-out)

A single table, ordered by `rank_score` descending across all three repositories, with
these columns: `rank`, `repo`, `component`, `location`, `current → target`,
`advisories`, `max severity`, `size`, `blast radius`, `evidence`.

- One row = one unit of work in one repository. Never bundle ("upgrade all Spring").
- `target` is a concrete released version. Never `latest`, never an unbounded range,
never a prerelease.
- `blast radius`: which modules/workspaces/Terraform roots the change touches.
- `evidence`: the scanner location plus advisory IDs — enough for a worker to verify the
row without re-deriving it.
- Keep component/location identity stable week over week so recurring rows are traceable.
- Cap the table at the top 25 rows and state the total count of remaining rows.

## Guardrails

- Read-only across all three repositories. No PRs, no commits, no upgrades, and do not
start the fan-out yourself.
- No secrets in the issue: version metadata only — no tokens, no `.tfvars` values, no
state contents.
- Do not open per-finding issues; exactly one issue per run.
- If a scan cannot be completed, publish the partial report rather than retrying
indefinitely or skipping the report.

## Human checkpoint

Stop after the issue is published. The queue is a proposal: a maintainer decides which
rows get picked up and whether a Track 2 fan-out starts. End your final message with the
issue link and the top three rows.
Loading