Skip to content

Support binary files in PR review workflow#18

Merged
bnavetta merged 1 commit intomainfrom
ben/pr-review-binary-files
Mar 25, 2026
Merged

Support binary files in PR review workflow#18
bnavetta merged 1 commit intomainfrom
ben/pr-review-binary-files

Conversation

@bnavetta
Copy link
Copy Markdown
Contributor

This amends the comment submission and diff formatting logic in the PR review workflow to handle binary files.

Today, if the agent tries to comment on a binary file, we'll generate a malformed request and GitHub will reject the entire review.

As far as I can tell from GitHub's API docs, they support file-level comments on binary files, but not line-level comments.

@bnavetta bnavetta force-pushed the ben/pr-review-binary-files branch from 860d413 to d93d11b Compare March 24, 2026 08:49
Copy link
Copy Markdown
Contributor Author

@bnavetta bnavetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested in warpdotdev/warp-internal#23397 - the agent was able to leave both a line-level comment on the modified text file and a file-level comment on the modified image.

} else {
payload.body = 'Automated review by Oz Agent';
}
if (hasSummary) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all indentation changes

}

console.log('Review posted successfully.');
// Post binary-file comments separately as file-level review comments.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File-level comments can't be posted as part of the createReview request unfortunately. Instead, we have to post each file-level comment separately afterwards.

We could extend this to let the agent post file-level comments on text files, but so far letting it post on the first line works well enough.

/^index / { print; next }
/^---/ { print; next }
/^\+\+\+/ { print; next }
/^Binary files / { print; next }
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures Binary files xyz and abc differ lines don't get an incorrect line number prefix.

@bnavetta bnavetta marked this pull request as ready for review March 24, 2026 09:05
@bnavetta bnavetta requested a review from captainsafia March 24, 2026 09:05

// Binary files lack a patch property in the listFiles response.
const binaryPaths = new Set(
prFiles.filter(f => !f.patch).map(f => f.filename)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little LLM told me that the !f.patch check might be too broad and catch things like submodule changes and file deletions. File deletions are probably less of a concern here but I wonder if there's hardening we can do here. GitHub's docs don't do a good job covering what the response schema looks like in different cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yeah it didn't seem like there's an obvious way to detect binary files reliably in their response 😕

I wonder if we want to treat other files without a patch as binary files for commenting purposes though - presumably, the API doesn't allow line-level comments on submodule changes or deleted files either, in which case we ought to leave file-level comments for those too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to reflect that we're using this approach for any file without a patch intentionally

@bnavetta bnavetta force-pushed the ben/pr-review-binary-files branch from d93d11b to 941d4dd Compare March 25, 2026 16:58
@bnavetta bnavetta merged commit b291b7e into main Mar 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants