ci(workflows): add commit message and PR title quality checks#26581
ci(workflows): add commit message and PR title quality checks#26581
Conversation
|
fyi @hamishwillee |
090c377 to
c13f937
Compare
|
Thanks @mrpollo ! May I propose to change the commit and pr title format to conventional commit combined with px4 subystem? Examples:
I'm just a fan of conventional commit and I like to be able to spot immediatly if I'm looking at a feature or a fix. |
|
I agree with @beniaminopozzan, being able to see whether something is a feature/fix/refactor/etc would be nice. Whatever we decide though we need to enforce consistency. I'd prefer we enforce conventional commits on PR titles only, not individual commits. Here's why:
To enforce this we can add the semantic-pull-request GitHub Action which validates PR titles against conventional commit format. We should also set the repo default squash commit message to "Pull request title" under Settings --> General --> Pull Requests. |
|
@dakejahl would we enforce squashing? I was usually against that because sometimes individual commits make sense in case part of a change needs reverting, but I can be convinced otherwise. |
f0de44a to
f198d02
Compare
|
Thanks for the feedback everyone! Conventional commits: done, I agree with this approach, it provides much more visibility. @beniaminopozzan I've adopted the type(scope): description format as suggested. The latest push migrates everything to conventional commits. You can see it in action right now: the bot already flagged this PR's own title (suggesting feat(ci): add commit message and PR title quality checks) and both commit messages. The second commit ("addressing pr review comments") is intentionally non-conforming so you can all see what the CI feedback looks like in practice. It'll get squashed before the merge. One note: the spec uses feat, not feature, so the examples would be feat(ekf2): new feature, fix(mavlink): command_int frames handling, etc. Squash policy: @dakejahl @julianoes I agree with Julian here. We shouldn't enforce squash-merge across the board. Individual commits are valuable when they're atomic and revertable. This way:
Open question: Should non-conforming commit messages block CI? Right now, missing conventional commit format on individual commits is just an advisory warning. The only blocking commit checks are for things that should never land on main (fixup!, squash!, WIP, throwaway messages like "oops", tmate leftovers). Should we also make non-conforming commit messages a blocking failure? The tradeoff is a cleaner history on main vs. more friction for external contributors who may not be familiar with the format. Interested to hear what you all think. Why not just semantic-pull-request? @dakejahl, that action only validates PR titles. Since we want to preserve individual commits when they're clean, we need commit-level advisory checks as well (catching fixup!, WIP, and throwaway messages like "oops"). The custom scripts also provide PX4-specific scope suggestions and auto-commenting with fix instructions, which the off-the-shelf action doesn't. What changed in this push:
|
|
No broken links found in changed files. |
|
The choice between squash-merge and multi-commit-merge is on a case by case basis. 99% of PRs are squash-merge. The PR title should be formatted correctly regardless of whether the reviewer chooses to squash-merge or multi-commit-merge. I think we should enforce it to get people in the habit of doing it. Maybe the CI failure can comment on the PR and suggest a title, this way a reviewer can easily copy/paste the suggestion and update the PR description if the author is ignorant. |
|
@dakejahl Cool I think thats aligned with my latest change, which as you can see above added a comment, which explains the error and also proposes a fixed title, plus it failed the checks so we can't merge. |
The drop-down with explanation is fine, but the above message is overly verbose. LLMs in-general annoy me with their verbosity. We don't need to explain the justification for the formatting, or at least move that to the drop down. Also we shouldn't suggest people to change commit messages, I want to avoid force-pushing since it erases history forcing a full re-review. Also for noobs it can cause them to accidentally blow away their changes. Since the 99% case is squash-merge and the commit message comes from the PR title, only suggesting/enforcing PR titles seems like the best solution. |
f09a1f5 to
ec7e60c
Compare
|
@dakejahl this is ready i made the changes we discussed, here's the preview for the new comments w/o the AI slop https://gist.github.com/mrpollo/31449b3c9aca89bdbc839d2c5c424464 |
I don't think force pushing after review is a good idea because changes can slip in undetected with a force push from the author. Either unintentionally or maliciously regressions can be introduced, especially when we ask for a rebase against main. Other than that I think it's okay to enforce commits to follow the convention. Also by using the squash-merge strategy you can merge main into your branch without polluting the state of review. It all goes in as one commit at the end of the day with squash-merge anyway (unless the author has a good reason for separate commits, which we handle on a case by case basis) |
ec7e60c to
5e4e214
Compare
Add CI enforcement of conventional commit format for PR titles and commit messages. Includes three Python scripts under Tools/ci/: - conventional_commits.py: shared parsing/validation library - check_pr_title.py: validates PR title format, suggests fixes - check_commit_messages.py: checks commits for blocking errors (fixup/squash/WIP leftovers) and advisory warnings (review-response, formatter-only commits) The workflow (.github/workflows/commit_checks.yml) posts concise GitHub PR comments with actionable suggestions and auto-removes them once issues are resolved. Also updates CONTRIBUTING.md and docs with the conventional commits convention. Signed-off-by: Ramon Roche <mrpollo@gmail.com>
5e4e214 to
30836c9
Compare
Adds CI checks for PR titles and commit messages using conventional commits format (
type(scope): description).Three jobs run on every pull request:
Bot comments are concise: issue, suggested fix, link to CONTRIBUTING.md. Comments auto-remove once resolved. Fork PRs get the same checks in CI logs only.
Comment previews: https://gist.github.com/mrpollo/31449b3c9aca89bdbc839d2c5c424464