Skip to content

feat: PraisonAI PR Reviewer Integration #1332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MervinPraison wants to merge 4 commits into from
Closed

feat: PraisonAI PR Reviewer Integration #1332

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c9e5023
feat: PraisonAI PR Reviewer Integration setup
MervinPraison Apr 9, 2026
3929d4f
Release v4.5.127
MervinPraison Apr 9, 2026
dcec82c
fix: isolated PraisonAI GitHub App secrets
MervinPraison Apr 9, 2026
399c6dc
chore: move setup guide to examples directory
MervinPraison Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/praisonai-reviewer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
framework: praisonai
topic: Pull Request Review
roles:
lead_reviewer:
role: Lead Code Reviewer
goal: Provide a comprehensive and constructive review of the pull request
backstory: You are an expert software engineer with deep knowledge of best practices, security, and performance. You meticulously review code for bugs and maintainability.
tasks:
code_review:
description: Review the code changes in the pull request. Look for logic errors, formatting issues, security flaws, and performance bottlenecks. Output the review as a detailed markdown comment.
expected_output: A detailed PR review in markdown format.
Comment on lines +9 to +11
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Agent has no tools to fetch PR diffs — reviews will be hallucinated

The code_review task instructs the agent to "review the code changes in the pull request," but no GitHub API tools or any file-access tools are configured for the agent. In the workflow, the repo is checked out, but the agent is given no mechanism to:

  • Determine the PR number or base/head SHAs
  • Run git diff or read changed files
  • Post a comment back to the PR

Without a tool like gh pr diff or a GitHub REST tool, the LLM will produce a generic, fabricated review that does not reflect the actual code changes. At minimum, the task description should provide the diff inline (e.g., via a shell step that captures git diff output and passes it to the agent), or the agent configuration should include GitHub tools.

40 changes: 40 additions & 0 deletions .github/workflows/praisonai-pr-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: PraisonAI PR Reviewer

on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
issue_comment:
types: [created]
Comment on lines +6 to +7
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security No permission check on issue_comment commenter

The issue_comment trigger fires for any comment by any user (including external contributors, bots, or bad actors) who can comment on the repo. Commenting @praisonai will invoke the full workflow with repo secrets. Consider adding a collaborator/org-member permission check, e.g.:

    if: |
      (github.event_name == 'pull_request' && github.event.pull_request.draft == false) ||
      (github.event_name == 'issue_comment' &&
       contains(github.event.comment.body, '@praisonai') &&
       (github.event.comment.author_association == 'OWNER' ||
        github.event.comment.author_association == 'MEMBER' ||
        github.event.comment.author_association == 'COLLABORATOR'))


jobs:
review:
runs-on: ubuntu-latest
if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai'))
Copy link
Copy Markdown

@qodo-code-review qodo-code-review Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Manual trigger never runs 🐞 Bug ≡ Correctness

The job if: condition references github.event.pull_request.draft, which is not present for
issue_comment events, so @praisonai comments will not trigger the reviewer job.
Agent Prompt 
### Issue description
`issue_comment` events don’t have `github.event.pull_request.draft`, so the job-level `if:` prevents comment-triggered runs.

### Issue Context
The repo already has a working pattern for comment triggers that checks `github.event.issue.pull_request` and gates by association.

### Fix Focus Areas
- .github/workflows/praisonai-pr-review.yml[3-13]

### What to change
- Rewrite the `if:` as a multi-line expression that handles each event type separately, e.g.:
  - For PR events: require `github.event.pull_request.draft == false`
  - For comment events: require `github.event.issue.pull_request` and the trigger phrase in `github.event.comment.body`
- Ensure the `contains()` call only evaluates for `issue_comment` events to avoid null dereferences.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 issue_comment trigger is dead code

The job-level if condition evaluates github.event.pull_request.draft == false first. When the event is issue_comment, github.event.pull_request does not exist and returns null. In GitHub Actions expression evaluation, null == false is false, so the entire condition short-circuits to false — the job will never run for issue comments, making the @praisonai trigger non-functional.

The fix is to guard each branch independently:

Suggested change
if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai'))
if: (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@praisonai'))


Comment on lines +11 to +13
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job-level if: condition references github.event.pull_request.draft, which is not present for issue_comment events. As written, manual triggers via @praisonai will be skipped (expression evaluates false/null) and it may also behave unexpectedly on PR events because contains(github.event.comment.body, ...) is undefined there. Consider splitting the condition by event type (pull_request vs issue_comment) and, for issue_comment, optionally gating by github.event.comment.author_association (e.g., MEMBER/OWNER/COLLABORATOR) to avoid untrusted users triggering runs/costs in public repos.

Copilot uses AI. Check for mistakes.
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Generate GitHub App Token
id: generate_token
uses: tibdex/github-app-token@v2
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security Third-party action not pinned to a commit SHA

tibdex/github-app-token@v2 references a mutable tag. If the upstream repository is compromised or the tag is force-pushed, the action could run arbitrary code with your PRAISONAI_APP_PRIVATE_KEY secret. The GitHub security hardening guide and Actions best practices both recommend pinning third-party actions to a full commit SHA:

Suggested change
uses: tibdex/github-app-token@v2
uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2

You can find the current SHA for the v2 tag in the tibdex/github-app-token releases page.

with:
app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.PRIVATE_KEY }}

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install PraisonAI
run: pip install praisonaiagents[all]
Comment on lines +32 to +33
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow installs praisonaiagents[all], but the CLI invoked later is praisonai .... The praisonai console script is provided by the praisonai package (see src/praisonai/pyproject.toml), so this will fail with praisonai: command not found unless praisonai is installed. Install praisonai (optionally with the needed extras) instead of, or in addition to, praisonaiagents.

Copilot uses AI. Check for mistakes.

- name: Run PraisonAI PR Review
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
praisonai agents --file .github/praisonai-reviewer.yaml
Comment on lines +35 to +40
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praisonai agents --file ... does not match the CLI in this repo. The supported way to execute a YAML file is praisonai run <file> (see src/praisonai/praisonai/cli/commands/run.py). Update the command accordingly so the workflow actually runs.

Copilot uses AI. Check for mistakes.
Comment on lines +35 to +40
Copy link
Copy Markdown

@qodo-code-review qodo-code-review Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Invalid cli invocation 🐞 Bug ≡ Correctness

The workflow runs praisonai agents --file ..., but the agents subcommand only supports
run/list and does not accept --file, so the job will fail immediately and never produce a
review.
Agent Prompt 
### Issue description
The workflow calls `praisonai agents --file .github/praisonai-reviewer.yaml`, but the `agents` subcommand doesn’t support `--file`, so the workflow fails.

### Issue Context
This repo’s documented YAML execution mode is `praisonai <path-to-yaml>` (e.g., `praisonai agents.yaml`).

### Fix Focus Areas
- .github/workflows/praisonai-pr-review.yml[32-40]
- examples/yaml/praisonai-pr-review.yml.template[32-41]

### What to change
- Replace the run command with the positional YAML invocation, e.g.:
  - `praisonai .github/praisonai-reviewer.yaml`
- Keep the rest of the workflow the same unless additional context is needed by PraisonAI.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

25 changes: 25 additions & 0 deletions PRAISONAI_PR_REVIEWER_SETUP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# PraisonAI PR Reviewer Setup Guide

This guide explains how to set up PraisonAI as an automated pull request reviewer using GitHub Actions and GitHub Apps.

## Prerequisites
1. A GitHub App created within your organization or account.
2. The App must have the following permissions:
- Pull Requests: Read & Write
- Issues: Read & Write
- Contents: Read
3. Generate a Private Key for the GitHub App.

## Setup Steps

1. Configure GitHub Secrets for your repository:
- `APP_ID`: The App ID of your GitHub App.
- `PRIVATE_KEY`: The generated Private Key (PEM format).
- `OPENAI_API_KEY` (or other LLM key) for PraisonAI to use.

2. Ensure `.github/workflows/praisonai-pr-review.yml` is present in your default branch.

3. Customize `.github/praisonai-reviewer.yaml` to configure the reviewing agents with specific roles.

## Triggering the Review
The review will run automatically upon PR creation and synchronization. You can also trigger it manually by commenting `@praisonai` on any pull request or issue.
41 changes: 41 additions & 0 deletions examples/yaml/praisonai-pr-review.yml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: PraisonAI PR Reviewer

on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
issue_comment:
types: [created]

jobs:
review:
runs-on: ubuntu-latest
if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai'))
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current if condition will cause the workflow to be skipped when triggered by an issue_comment (e.g., via @praisonai). This is because github.event.pull_request is null for such events, making the expression github.event.pull_request.draft == false evaluate to false. The condition should be updated to handle both event types explicitly and ensure the comment is on a pull request.

    if: (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '@praisonai'))


Comment on lines +10 to +13
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template uses the same job-level if: that references github.event.pull_request.draft, which won’t exist for issue_comment events. Manual @praisonai triggers will be skipped unless you rewrite the condition to handle pull_request vs issue_comment payloads separately (and consider gating by commenter association to prevent untrusted triggers).

Copilot uses AI. Check for mistakes.
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Generate GitHub App Token
id: generate_token
uses: tibdex/github-app-token@v2
with:
app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.PRIVATE_KEY }}

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install PraisonAI
run: pip install praisonaiagents[all]
Comment on lines +32 to +33
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template installs praisonaiagents[all] but then runs the praisonai CLI. praisonai is a different package and provides the console script; without installing it, the workflow will fail. Update the install step to install praisonai (and any required extras).

Copilot uses AI. Check for mistakes.

- name: Run PraisonAI PR Review
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
# Use PraisonAI to review the PR
praisonai agents --file .github/praisonai-reviewer.yaml
Comment on lines +35 to +41
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template runs praisonai agents --file ..., but the CLI’s file execution entrypoint is praisonai run <yaml> in this repo. Update the command so the template is copy/paste runnable.

Copilot uses AI. Check for mistakes.
Loading