WIP Reviewing large projects with AI: contributing in a meaningful way - #1240
Draft
dutow wants to merge 1 commit into
Draft
WIP Reviewing large projects with AI: contributing in a meaningful way#1240dutow wants to merge 1 commit into
dutow wants to merge 1 commit into
Conversation
ImTheKai
reviewed
Jul 22, 2026
|
|
||
| As the saying goes, we can do it fast or cheap or good -- you can choose two. | ||
| The issue in the above examples is that people tend to choose fast and cheap: | ||
| the minimal effort, most comfortable combination, and that results in a local maximum, not global. |
Contributor
There was a problem hiding this comment.
reads better ..."not a global one"
| 3. I possibly spam my colleagues with slop, even if at this point my autoreviewer rarely makes bad change requests. | ||
|
|
||
| For high throughput projects, such as the PostgreSQL mailing list, I don't want to review everything: | ||
| that takes CPU time (build and test execution on my PC), I would also quickly hit my session limits, and even if I would process everything, I do not have the time to personally look through all that. |
Contributor
There was a problem hiding this comment.
"even if I did process everything"
|
|
||
| ## What about reviewing with AI? | ||
|
|
||
| Normally, when you join a new large project you don't start by submitting huge pull requests making changes to multiple distinct parts of the code because you lack the understanding to do so. |
Contributor
There was a problem hiding this comment.
missing comma - "when you join a new large project, you don't start by..."
| As I already have subscriptions, and I'm not using them all the time to their limits, this is basically free. | ||
| 2. Since I have a good desktop PC, I want things to run in parallel: | ||
| if I enqueue two postgres patch reviews at the same time, I would prefer them completing sooner in parallel than later in sequence. | ||
| 3. All this parallelization has to be easily user visible, for example I want to be able to see and access the claude sessions. |
| Back when I started it, programmatic use of `claude` was still allowed with subscription pricing, but there were already talks about it being restricted to token based usage for 3rd party tools. | ||
|
|
||
| Do we have to go there to accomplish our goal? | ||
| Can't we simply ask claude to create a specific empty file on completion, and then `kill` the process when we detect it? Or add a stop hook that accomplishes the same in an even more reliable way? |
|
|
||
| As I said in the beginning, at this point we are using this tool in most of our PostgreSQL-related projects, and that evolved into two different patterns: | ||
|
|
||
| For our github repos, with less traffic, it automatically watches the open pull requests: |
| ## Disclaimer on LLM automation and security | ||
|
|
||
| Running LLMs unsupervised has its own dangers, I wrote more about this [previously](https://percona.community/blog/2026/05/05/how-i-stopped-babysitting-my-coding-agent-with-dotfiles/). | ||
| Autoreviewer runs claude code with `--dangerously-skip-permissions` in a docker/podman sandbox. |
|
|
||
| Running LLMs unsupervised has its own dangers, I wrote more about this [previously](https://percona.community/blog/2026/05/05/how-i-stopped-babysitting-my-coding-agent-with-dotfiles/). | ||
| Autoreviewer runs claude code with `--dangerously-skip-permissions` in a docker/podman sandbox. | ||
| As long as the separation holds, it shouldn't have access to anything outside the review directory on your PC, but it does have full network access, and it holds a github token and possibly a deploy key, so keep that in mind, and keep those as restrictive as possible. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.