Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 149 additions & 1 deletion .agents/skills/process-pr-reviews/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
---
name: process-pr-reviews
description: Use when the user asks to process, triage, fetch, view, count, list, or resolve review feedback in a GitHub PR. The built-in workflow currently focuses on CodeRabbit. In this workflow, “real review feedback” is strictly defined as actionable inline comments that are neither review summaries nor nitpicks.
description: Use when the user asks to process, triage, fetch, view, count, list, or resolve review feedback in a GitHub PR. Supports both CodeRabbit and Codex review workflows. In this workflow, “real review feedback” is strictly defined as actionable inline comments; for CodeRabbit, exclude review summaries and nitpicks, and for Codex, exclude review summary cards and use PR main-thread reactions only as status signals.
---

# Process PR Reviews

This workflow supports both CodeRabbit and Codex PR review signals.

## CodeRabbit Reviews

“Real review feedback” is strictly defined as:
Expand All @@ -13,6 +15,8 @@ description: Use when the user asks to process, triage, fetch, view, count, list
- **not** a review summary
- **not** a nitpick

Nitpick comments from CodeRabbit must always be ignored to avoid unnecessary noise.

There is no need to analyze the comment content itself.

### Data sources
Expand Down Expand Up @@ -95,6 +99,11 @@ The final goal of the CodeRabbit workflow is always:

> **Top-level inline comments left by CodeRabbit in `pulls/<pr_number>/comments` that are neither nitpicks nor summaries**

Important:

- Treat CodeRabbit nitpicks as non-actionable by default.
- Do not include nitpicks in counts, summaries, or resolution queues unless the user explicitly asks for nitpicks.

In practice, do the following:

1. Get CodeRabbit top-level inline comments from `pulls/<pr_number>/comments`
Expand All @@ -114,6 +123,18 @@ If the output of `gh api --paginate ...` is too large and gets truncated:

### Resolving review conversations

### Resolution policy

When triaging review feedback, apply this rule:

- If a comment will **not** be fixed, you may resolve the conversation after triage.
- If a comment **will** be fixed, do **not** resolve it first — make the code change first, then resolve the conversation afterward.

In short:

- **won't fix / no code change** → triage, then resolve
- **will fix / code change required** → fix first, then resolve

If the user asks to resolve a CodeRabbit review conversation:

1. Identify the target inline comment from the actionable comment list.
Expand All @@ -131,3 +152,130 @@ Notes:
- Resolve the **thread**, not the individual comment.
- `pulls/<pr_number>/comments` remains the source of truth for identifying actionable inline comments.
- `reviewThreads` is only for thread-level operations such as resolving conversations.

## Codex Reviews

“Real review feedback” is strictly defined as:

- **inline review comments**
- **not** the review summary card

There is no need to analyze the comment content itself.

### Important behavior differences from CodeRabbit

- Codex review is **silent while running**.
- Unlike CodeRabbit, Codex does **not** expose an in-progress PR check for review status.
- While Codex is reviewing, the PR main conversation thread gets an `eyes` reaction from `chatgpt-codex-connector[bot]`.
- If Codex finds no issues, it may leave **no actionable inline comments** and instead react to the PR main conversation thread with `+1` (thumbs up).
- If Codex finds issues, it may create inline review comments in `pulls/<pr_number>/comments` and a review summary card in `pulls/<pr_number>/reviews`.

### Data sources

The Codex workflow uses these sources:

1. **PR review comments**

```bash
gh api --paginate repos/<owner>/<repo>/pulls/<pr_number>/comments
```

This is the authoritative source for actionable inline Codex comments.

2. **PR reviews**

```bash
gh api --paginate repos/<owner>/<repo>/pulls/<pr_number>/reviews
```

This is used to identify the Codex review summary card such as `### 💡 Codex Review`. It is not the primary source of actionable inline feedback.

3. **Issue comment reactions on the PR main thread**

First fetch the PR issue node / comments if needed:

```bash
gh api repos/<owner>/<repo>/issues/<pr_number>/comments
gh api repos/<owner>/<repo>/issues/<pr_number>/reactions
```

Use reactions on the PR main conversation thread only to detect Codex review state:

- `eyes` from `chatgpt-codex-connector[bot]` → Codex review appears to be in progress
- `+1` from `chatgpt-codex-connector[bot]` on the PR main thread → Codex reviewed and found no issues

These reactions are **status signals**, not actionable review feedback.

Do not treat these as primary sources for actionable comments:

- `gh pr view ...`
- `gh api repos/<owner>/<repo>/issues/<pr_number>/comments`
- `gh api repos/<owner>/<repo>/issues/<pr_number>/reactions`

Reason: actionable Codex review feedback still lives in PR review comments, not in the PR issue timeline.

### Workflow

#### 1. Fetch inline comments

```bash
gh api --paginate repos/<owner>/<repo>/pulls/<pr_number>/comments
```

Only keep records that satisfy all of the following:

- `user.login` is `chatgpt-codex-connector[bot]`
- `in_reply_to_id == null` (only top-level inline comments, not replies)

This is the candidate set of actionable Codex comments.

#### 2. Fetch reviews to identify the summary card

```bash
gh api --paginate repos/<owner>/<repo>/pulls/<pr_number>/reviews
```

Identify Codex review summaries. Common characteristics include:

- `### 💡 Codex Review`
- explanatory text such as `Here are some automated review suggestions for this pull request.`
- “About Codex in GitHub” help text

These review-level contents are **not the final result**. They are only used to understand whether Codex posted a review summary.

#### 3. Optionally inspect PR main-thread reactions for status

If the user asks whether Codex is still reviewing, or whether Codex finished with no findings, inspect reactions on the PR main thread.

Interpret them as follows:

- `eyes` by `chatgpt-codex-connector[bot]` → likely still reviewing / review in progress
- `+1` by `chatgpt-codex-connector[bot]` with no Codex inline comments → likely completed with no findings

Do not count these reactions as review comments.

### Filtering rule

The final goal of the Codex workflow is always:

> **Top-level inline comments left by Codex in `pulls/<pr_number>/comments`**

In practice, do the following:

1. Get Codex top-level inline comments from `pulls/<pr_number>/comments`
2. Use `pulls/<pr_number>/reviews` only to recognize the summary card
3. If there are no Codex inline comments, optionally inspect PR main-thread reactions to distinguish:
- still reviewing (`eyes`)
- reviewed with no findings (`+1`)
- no observable Codex activity

### Large output handling

If any `gh api --paginate ...` output is too large and gets truncated:

1. Record the tool output file path
2. Do not manually read through the entire large JSON blob
3. Hand it off to `@explorer` to extract:
- Codex-authored inline comments
- the number of top-level inline comments
- each comment’s `path` / `line` / `body`
52 changes: 52 additions & 0 deletions apps/desktop/main/desktop-diagnostics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type {
RuntimeEventQueryResult,
RuntimeLogEntry,
RuntimeState,
StartupProbePayload,
} from "../shared/host";
import type { RuntimeOrchestrator } from "./runtime/daemon-supervisor";
import {
Expand Down Expand Up @@ -54,6 +55,17 @@ type DesktopDiagnosticsSnapshot = {
updatedAt: string;
isPackaged: boolean;
coldStart: DesktopColdStartSnapshot;
startupProbe: {
preloadSeen: boolean;
rendererSeen: boolean;
entries: Array<{
source: StartupProbePayload["source"];
stage: string;
status: StartupProbePayload["status"];
detail: string | null;
at: string;
}>;
};
sleepGuard: SleepGuardSnapshot;
renderer: DesktopRendererSnapshot;
embeddedContents: DesktopEmbeddedContentSnapshot[];
Expand All @@ -68,6 +80,8 @@ function nowIso(): string {
return new Date().toISOString();
}

const MAX_STARTUP_PROBE_ENTRIES = 200;

export function getDesktopDiagnosticsFilePath(): string {
return resolve(app.getPath("userData"), "logs", "desktop-diagnostics.json");
}
Expand All @@ -83,6 +97,12 @@ export class DesktopDiagnosticsReporter {
error: null,
};

private readonly startupProbe: DesktopDiagnosticsSnapshot["startupProbe"] = {
preloadSeen: false,
rendererSeen: false,
entries: [],
};

private sleepGuard: SleepGuardSnapshot = createInitialSleepGuardSnapshot();

private readonly renderer: DesktopRendererSnapshot = {
Expand Down Expand Up @@ -152,6 +172,33 @@ export class DesktopDiagnosticsReporter {
this.scheduleFlush();
}

recordStartupProbe(payload: StartupProbePayload): void {
if (payload.source === "preload") {
this.startupProbe.preloadSeen = true;
}

if (payload.source === "renderer") {
this.startupProbe.rendererSeen = true;
}

this.startupProbe.entries.push({
source: payload.source,
stage: payload.stage,
status: payload.status,
detail: payload.detail ?? null,
at: nowIso(),
});

if (this.startupProbe.entries.length > MAX_STARTUP_PROBE_ENTRIES) {
this.startupProbe.entries.splice(
0,
this.startupProbe.entries.length - MAX_STARTUP_PROBE_ENTRIES,
);
}

this.scheduleFlush();
}

recordRendererDidFinishLoad(url: string): void {
this.renderer.didFinishLoad = true;
this.renderer.lastUrl = url;
Expand Down Expand Up @@ -277,6 +324,11 @@ export class DesktopDiagnosticsReporter {
updatedAt: nowIso(),
isPackaged: app.isPackaged,
coldStart: { ...this.coldStart },
startupProbe: {
preloadSeen: this.startupProbe.preloadSeen,
rendererSeen: this.startupProbe.rendererSeen,
entries: this.startupProbe.entries.map((entry) => ({ ...entry })),
},
sleepGuard: {
...this.sleepGuard,
counters: { ...this.sleepGuard.counters },
Expand Down
Loading
Loading