Code reviews are way too verbose

[deboer-tim]

Checklist

• This is a new feature, not a refactoring suggestion
• I have checked for this feature in the Sourcery documentation
• I have checked there are no open requests referencing the same feature

Description

Hey @sourcery-ai - I've reviewed your changes - here's some feedback:

Overall Comments:

• Notice all the boilerplate? Find it hard to scan to this comment? We have had to turn off most features of Sourcery and it is still by far the most chatty member of our team. It's like having a new hire that rarely has useful information but feels compelled to comment on everything, in depth.
• As a result, our team response is mixed: some developers want to keep it on because sometimes there's a useful nugget. Others want to disable sourcery completely because it gets in the way of productive work and they have already tuned it out.

Net:

• I'd really appreciate some options to drastically reduce the verboseness: removing all boilerplate text from the comments and making all messages more succinct.
• Options to only comment above a 'threshold of certainty' would also be helpful - i.e. only add a comment when it is more sure that it's actually useful.

Here's what I looked at during the review
Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

[Hellebore]

Ha - great feedback thanks!

Just so I can hone in on the issues here:

• which features have you already turned off, and could we tweak those to make them more useful?
• could you give some examples of the most offensive boilerplate? Definitely open to trimming things down or adding options to try and make things more succinct
• on the certainty that's a tough one. We have multiple layers of validation, but could consider an option to filter even harder. Are there classes of comment you find more or less useful, or that you would especially miss if filtered out? (For example we always let typo comments through since they're useful even if not super important)

[deboer-tim]

• which features have you already turned off, and could we tweak those to make them more useful?

We scaled back pretty quickly to just code reviews - we were hoping that would be a place to start with the least impact / highest benefit.

• could you give some examples of the most offensive boilerplate? Definitely open to trimming things down or adding options to try and make things more succinct

'Hey - I've reviewed your changes - here's some feedback:'
'Overall Comments:'
'Here's what I looked at during the review'
'divider line'

• These are identical in every PR and provide zero value. Personally, I'd prefer not to have any of them.
• It seems like these kind of comments were added to make Sourcery feel more human and explain what it's doing, but for me it does the opposite.

'Sourcery is free for open source - if you like our reviews please consider sharing them ✨'

• I understand you need to advertise, but I don't need to read that 10 times a day.

'Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.'

• Ditto. I only really need to see this one or twice (or in the docs), then could omit.

Beyond this, I'd rather have the quiet coworker that only speaks up when they have something useful to say and gets to the point vs a loud repetitive one.

• on the certainty that's a tough one. We have multiple layers of validation, but could consider an option to filter even harder. Are there classes of comment you find more or less useful, or that you would especially miss if filtered out? (For example we always let typo comments through since they're useful even if not super important)

FWIW I took a quick look through my last 3 PRs. Sorcery added more than 40 lines of output + 30 lines of code snippets / diffs. There were 5 'Consider x' that weren't really useful, a suggestion to extract a function from a test (meh), so I almost missed the one really useful case where it noticed the timeout in a comment didn't match the code. Even that took 4 lines to explain vs what a human would have written, e.g. 'woah - comment doesn't match code here'.

Nobody would read a 70 line email in the hopes that there's one useful point, they'd just file the next email they see from that person. I know this is hard and I don't know what kind of control you have, but if I was prompt tuning it would be something like 'you are a concise code assistant that gets to the point, and only comment when you are very sure of the suggestion'...