How to deal with AI PRs

[niccokunzmann]

Having a lot of open pull requests is nice. However, we struggle to review them all.
See #1470

My assessment - constraints, motivation and values (please add yours in your reply):

• first-timers and new contributors come to icalendar. We invite that for a learning space for humans.
• The way programming has changed for people who are new is that it is AI assisted from the start.
• GitHub does not allow us to moderate this well, see
  • https://www.theregister.com/software/2026/02/03/github-ponders-kill-switch-for-pull-requests-to-stop-ai-slop/4334869
  • https://github.com/orgs/community/discussions/185387
• Our intention is to open a learning space for people and a way to contribute:
  • We find joy in working with people and raising them up to contribute again and better.
  • Interacting with an AI agent, undisclosed or not, is not our intention behind this as we want to train people, not AI.
  • We can also just instruct an AI to solve an issue and are more efficient at that than a new contributor.

Questions I have:

• How can we protect our volunteers' time?
• How can be assess quickly if a PR is worth reviewing before another?

Please answer with:

• Your motivation to contribute to icalendar, to review pull requests.
• Your ideas of how to keep that motivation going and protected through a high load of PRs.
• Ways in which you would like to go forward, boundaries, ways which are not acceptable to you.

---

My motivation:

• I want to know that people who actively contribute already and over a long time feel comfortable doing so, on the base of free giving.
• I want to work on a space that is welcoming and inclusive.
• I want to know that contributors (people) receive the attention they need to continue contributing to open-source and icalendar and the ecosystem around it.
• I would like icalendar to increase in quality.

Possible solutions:

• [Quick PR closing] I notice that many AI assisted PRs do not follow the PR template. I suggest closing the PRs automatically when it comes from external contributors that we do not have a relationship with when it does not follow the PR template, with a note on a docs page that says how to contribute. There is a white-list for people who the team trusts that they know what they are doing. That should be quick and less painful.
  • Considerations:
    • I know the PRs are probably contributing ok code a lot of the time. However without a community of reviewers, they are useless. As such, reviewer & maintainer protection has higher priority to me.
    • It might be good to enforce an open issue to be linked too.
    • These requirements can be automated.
    • People can open them when they actually read what they need to do or ask questions as a human. AI does not ask questions at this point.
• [Office Hours] It is much nicer to know who is working with us. I propose to be online at times in a chat & video call so that people can ask questions about their PR and how to properly contribute. That feedback can flow into documentation.
  Inspiration: NLNet: https://nlnet.nl/officehour/

Less favourite solutions:

• [Improve AI Quality] Another approach is to instruct all those people contributing to automate their AI to follow the form that we provide, e.g. PR templates & AI disclosure. However, automating this does not solve the problem that care is not taken by a person.
• Moving off GitHub as it belongs to Microsoft and as such will be enshittified anyway. As a leading platform, it does not have the motivation to protect the FOSS maintainers in the first order. That is only a required side effect of its marketing. As such, it cannot offer the same protection as e.g. a private GitLab or a moderated GitLab instance.

---

I know that we are not the only ones struggling with spam and higher load. It is not our responsibility to shoulder that load. It is the contributors' responsibility to show quality. So, there are solutions out there and I invite others who see them to also answer.

Specifically asking you, the team: How would you like to go forward with this?
@stevepiercy @SashankBhamidi @abe-101 @angatha @jacadzaca @ennamarie19 @geier @lcampanella98

[stevepiercy]

"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

~ Frank Herbert, Dune

I spent about a month reviewing a flood of AI-assisted pull requests for icalendar. It became clear to me that most of the contributors did not understand the code. I became the prompt to an AI agent to correct its mistakes. Many of the pull requests were for esoteric use cases that would never be experienced in practice, and were difficult for me to understand. The burden to review such pull requests was beyond what I can sustain as a volunteer. At this point, I've stopped reviewing any pull request that smells like AI was involved in its use.

I enjoy contributing to open source software to experience learning, understanding, and discovery, then sharing that experience with other people. AI gets in the way of that experience. It has been sold as a way to be more productive, but producing more slop is unwanted, erodes trust, and distracts core collaborators from being productive themselves. I want to have a significant and positive impact in both icalendar and the community that surrounds it.

I don't understand the motivation behind creating 50-100 pull requests in 3 weeks to external repositories. It's impossible that these contributors could understand all that code. Clearly they don't share the same values that I have for contributing to open source. It seems that they're gamifying contributions to get the highest score, and have no concern whatsoever to the negative impact that they cause.

I think it's time to establish a gateway to contribute, ensuring that we have quality contributions, not quantity. Such a gateway would include the follow points. Only trusted contributors are excepted from these requirements.

1. All PRs must comply with icalendar's requirements as specified in its PR template and Contribute guide.
2. We need some ways to block AI slop PRs. So far we've implemented https://github.com/ehmatthes/gh-profiler to gauge the trustworthiness of a profile. We're looking at other tools.
3. We need some ways to verify real people and create an allow list of them.
   a. I like the idea of regular office hours. The Python documentation team does this, and it's a fun experience. I would volunteer an hour per week to this effort.
   b. Anyone can triage pull requests by reviewing and providing feedback on them. This establishes credibility and trust faster than PR quantity.
   c. We could also assign the GitHub role of Triage to individuals who demonstrate that they share our values.
   d. People who make any other type of contribution to icalendar—whether it's creating a topic in Discussions, reposting our Mastodon toots, writing blog posts about icalendar, or even making a donation to icalendar—could be indicators that they're humans worthy of trust.

[tobixen]

Potentially contributing to the "AI arms race", I asked Claude to have a look into all the open pull requests. A lot of pull requests can probably be closed right away as "AI-slop", but Claude Opus warned that it may be "socially expensive" should any false positives be flagged as "AI-slop". Claude Opus also insisted on "softening the message" before going public with it. Here is the public version:

(post-edit: Any observant reader would be able to see that the "full disclosure" given by Opus below is not a "full disclosure" - I did give some few additional follow-up prompts and answers to questions, including the wish to publish it in this discussion).

⚠️ This comment is AI-generated (Claude Opus 4.8 via Claude Code) on behalf of tobixen ⚠️

This is a direct attempt at @niccokunzmann's question "How can we assess quickly if a PR is worth reviewing before another?" — a worked triage of the current 32 open PRs, produced by exactly the kind of tool the discussion is worried about. I'm posting it in that spirit: as a labour-saving data-point for the human reviewers, not as a verdict. Treat every "close"/"AI" call below as a suggestion to be checked by a person.

Full disclosure of method. This was generated by pointing Claude Code at the repo with this verbatim prompt (typos and all):

The maintainers of this repo complains it's too many open pull requests and too much work to go through all of them. Could you create me an overview of outstanding pull requests, and a short-list of pull requests that ought to be merged, closed or sent back for review, based on criterias like if the contributors guidelines (found in the docs folder) has been followed, the quality of the contribution, security issues found, if there exists an issue linked to the pull request, and weather the issue actually adresses a real-world problem. Also try to assess human contributions vs AI-controbutions in each pull request, real human beings that spends time and effort creating a pull request deserves feedback and attention.

It then read every PR's diff, fetched the linked issues, and applied the project's docs/contribute rules + AI policy. Caveats that matter: the human-vs-AI guess is guesswork (confidence tags below) — please don't treat it as accusation; I could not run any PR, so "quality" is a static read; and a public list naming contributors is inherently harsher than a private one. The irony of using AI to triage AI is not lost on me.

---

The big picture

The backlog is not mostly spam — it's mostly good-first-issue / docstring churn plus a wave of AI-assisted PRs, most honest, a few not. The bottleneck is reviewer attention, not malice. Three patterns explain the pile:

1. A docstring/doc-cleanup swarm off maintainer-filed checklists (apply Google Style to a function documentation [beginner, Python] #1072, Replace copy-paste of RFCs in docstrings with Pythonic docstrings #1244, Clarify auto-generated properties for new class methods #1473, Write tests for structure and quality of docstrings #1481) — ~13 PRs, individually fine, collectively a review tax. Several hit the same files and even collide on news-fragment filenames.
2. AI-disclosure compliance is the single best sorting axis. The policy permits AI use when disclosed (model+version in commits + a changelog note). The good contributors do this; the closeable ones are AI agents committing with no disclosure, usually also missing a changelog and red on CI. This directly supports the "[Quick PR closing]" idea — the signal is already machine-detectable.
3. A few genuinely valuable bug fixes are buried (Fix binary data corruption in vBinary for non-UTF-8 byte input #1356, Preserve RFC 7265 unknown property values verbatim #1450, Resolve globally unique TZIDs with a vendor prefix (#313) #1479, Fix line folding across escape sequences #1502, escape lone carriage return in _escape_char #1462/validate property, parameter and recur names in from_jcal #1466) and deserve to jump the queue.

Tally: ~7 merge-ready or nearly so, 3 close candidates, ~22 need a round-trip (several only a rebase/approval away).

---

⭐ A — Merge (or merge after one approval / rebase)

PR	Author	What	Why it's ready
#1356	uwezkhan	Fix vBinary corruption for non-UTF-8 bytes	Real bug (#1289/#1299), strong tests, 2 approvals, valid breaking-change changelog
#1450	lcampanella98	Preserve RFC 7265 unknown/X- values verbatim	Real bug (#1445), approved, CI green, exemplary AI disclosure
#1479	vjsai	Resolve vendor-prefixed "globally unique" TZIDs (#313)	Real long-standing bug, strong dual-backend tests, disclosed. Highest-quality PR in the backlog.
#1392	gjabell	Restore lost pprint() in tz-equivalence script	Surgical human regression fix, valid changelog
#1488	Labib-Bin-Salam	Reject non-ASCII digits in vWeekday ordwk	Correct, minimal, mirrors merged #1465, model AI disclosure
#1498	alhudz	Docstring polish (:rfc: role + Returns)	Trivial, correct, clean
#1503	andycheng2018	Fix FreeBusy.new() docstring (copy-pasted from Alarm)	Obviously-correct one-liner, disclosed

---

🛑 B — Close candidates (with a courteous reason)

PR	Author	Why
#1432	aniketaggarwal	Merge-conflicting and self-contradictory — claims to document Calendar.from_ical but points the doc links away from it. Author unresponsive to the request to explain. Real issue (#1430), better redone fresh.
#1455	sangamithraashok244-ship-it	Solves a non-problem — removes Sphinx ~ claiming "broken links," but the links work and ~ is intentional. No changelog, conflicts, unanswered pushback.
#1491	HrachShah	ai-suspicion-labelled, three unrelated changes stacked, ruff fails, no linked issue, no changelog, undisclosed AI. Ask for one focused, disclosed PR.

---

🔁 C — Send back for review (priority order)

Real fixes, nearly there — need a maintainer decision:

• Fix line folding across escape sequences #1502 (kingrubic) — line-folding across escape sequences, real bug ([Bug] Description wrapping can create an invalid event #1501), tested, AI now disclosed. Re-review the _foldline rewrite; clear the stale change-requests + ai-suspicion label.
• escape lone carriage return in _escape_char #1462 / validate property, parameter and recur names in from_jcal #1466 (alhudz) — security-relevant (content-line / name injection via lone \r in jCal), solid tests. Need AI disclosure folded into the changelog + conflict resolution.
• Add Alarm.new_display(), new_audio(), and new_email() factory methods #1417 (klouds27) — strong VALARM factory feature (VALARM custom new() methods #865), well-tested. Needs branch update + an explicit answer to the AI-use question (asked twice).
• docs: avoid false section headings in public docstrings #1482 / Add initial automated docstring structure tests #1483 (kingrubic / KartavyaDikshit) — both attack Write tests for structure and quality of docstrings #1481; docs: avoid false section headings in public docstrings #1482 is the better attempt, Add initial automated docstring structure tests #1483's test is logically broken (verified not to catch the bad headings). Pick one, close the other.

On hold by your own decision: #1468 (alhudz) — sound security fix, but you deferred all lone-\r PRs pending the #1471 design. Keep open, blocked.

Legit but incomplete: #1318, #1336, #1385, #1387 (finishing touches), #1438, #1492 (changelog + the RFC context the issue asked for), #1342 (broken RST, no changelog — lean close), #1362 (uwezkhan — undiscussed breaking change, should be breaking not bugfix), #1467, #1478, #1489 (undisclosed-AI commits, missing changelogs — lean close).

The #1504–#1507 burst (checkzhao8888): four same-day docstring PRs, all on #1473. Not spam — content is correct (a couple fix real doc bugs), AI use disclosed. But handle as one coordinated review: they collide on news-fragment names (1473.documentation, .1, .2, .3) and all falsely tick the "added tests" box.

---

🤖 Human vs AI (soft signal — guesswork, confidence noted)

This is the part to take with most salt. It is offered only to help you spend attention where a human is on the other end.

• Clearly human, genuine effort: gjabell (Fix equivalent_timezone_ids script #1392, high), uwezkhan (Fix binary data corruption in vBinary for non-UTF-8 byte input #1356/Avoid implicit file reads from string calendar input #1362, med), alhudz (reject CR and LF in vUri, vCalAddress and vInline values #1468/polish value_separator_index docstring (rfc role + Returns section) #1498, med) — deep engagement, root-cause precision, opened tracking issues.
• AI-assisted but honestly disclosed (policy-compliant — judge on merit): lcampanella98 Preserve RFC 7265 unknown property values verbatim #1450, vjsai Resolve globally unique TZIDs with a vendor prefix (#313) #1479, Labib-Bin-Salam Reject non-ASCII digits in vWeekday ordwk (BYDAY/BYWEEKDAY/WKST) #1488, kingrubic docs: avoid false section headings in public docstrings #1482/Fix line folding across escape sequences #1502, andycheng2018 Fix FreeBusy.new() docstring referencing alarm instead of free/busy #1503, checkzhao8888 docs: clarify Event.new() optional and auto-generated properties #1504–1507. This is the behaviour the policy asks for — model named in commits + changelog, real understanding shown.
• Appears AI-generated and undisclosed (the cases the policy targets): the commits authored by the Zo Bot automation identity (github-automationzo.computer) on validate input type in vDDDTypes.from_ical before calling .upper() #1467, make vGeo.to_ical() return bytes like every other vDataType #1478, drop redundant journal.start assignment in Journal.new #1489, skip None values in Parameters.to_ical instead of crashing #1491 — no model disclosure, scope creep, frequently red CI. This is where "[Quick PR closing]" would do the most good with the least collateral damage.
• Ambiguous (asked about AI, never answered): Add Alarm.new_display(), new_audio(), and new_email() factory methods #1417, escape lone carriage return in _escape_char #1462/validate property, parameter and recur names in from_jcal #1466 — polished, but real understanding shown; a direct question should resolve it.

---

Two concrete ideas feeding @niccokunzmann's proposals

1. The "[Quick PR closing]" criteria you proposed are already detectable. Across this backlog the high-signal, low-false-positive filter was: commit authored by a bot/automation identity AND missing news fragment AND red CI. That set (the Zo Bot cluster) maps almost perfectly onto the PRs a human would also want to bounce. Linking a required open issue, as you suggest, would catch most of the rest.
2. Triage by author, not just by PR. The same two contributors account for ~8 PRs. One templated message each (AI disclosure + one-change-per-PR for the bot cluster; squash/fragment-naming for the docs: clarify Event.new() optional and auto-generated properties #1504–1507 burst) clears more backlog than eight separate review threads.

And one for the welcoming half of your motivation: the disclosed-AI contributors above are doing exactly what the policy asks. If they get bounced together with the undisclosed bots, the lesson learned is "don't disclose," which is the opposite of what we want. Worth rewarding the disclosure explicitly.

---

Methodology, raw per-PR notes, and the diffs behind each call are available — happy to share if useful. Errors are mine (well, the model's); please correct anything I got wrong about your PR.

⚠️ This comment is AI-generated (Claude Opus 4.8 via Claude Code) on behalf of tobixen ⚠️

[angatha]

I found this project because I needed some framework to parse ical files. Somehow, I got stuck here and after some long dormant time, I found joy (and mostly time) in reviewing Code again. From my work background, I like to teach people how to write good code and checking PRs is somehow practice for me.
I sometimes struggle with design decisions made for ical especially regarding strict validation and robustness.

Currently I only react to PRs that mention me in some way. With those, I have no trouble keeping up. I ignore the ones about documentation, mainly because I do not know what we expect.

Reviewing some PRs where you added the AI suspicion flag, I noticed that I do not spot that as reliable. Mainly because I focus on the code and if it works, it is usually fine for me. Since I do not trust any contribution to work, I will not have more work with code that is not understood by the “author”.

It may be possible to be stricter with closing PRs without activity, especially when some reviewer already requested something. Giving a few days’ notice to give them a last chance to respond. We may even say that they can recreate a PR with addressed concerns.
We may include a check for the AI policy within the PR template. Then we may get immediate AI disclosure.

A few months from now, my available time for foss will be quite restricted. So just be warned about that. But I have no special boundaries for reviewing. I do it when I have some time and will so in the future. If that are just agents, sad for the community and programming culture but personally I had my fun. Don’t worry about loosing me because of AI :)

My working hours (or minutes scattered around the day) do not fit well with calls where someone can join to ask questions. And please do not take this personally but I don’t care who you are. @niccokunzmann I would not participate in the [Office Hours] suggestion as a maintainer, but I see potential for faster reviews if I can join a call to ask some question.

[niccokunzmann]

I closed #1507, #1506 and #1505 as AI spam because there is already a PR open #1504 that requires feedback.

A first-timer cannot expect to open several PRs at once and get it right. It is a waste of time to duplicate feedback across PRs that are AI generated. As such, multiple open PRs by one first-timer - from my view - can be closed until the first one of them is merged.

[stevepiercy]

I wasted a couple of hours trying to automate the detection of bots in each commit in #1514. I gave up. There's probably an easier way of doing it by checking the username and email.

[stevepiercy]

I opted for the GitHub Action that checks whether the user's last commit appears to be a bot or AI agent in #1514. That could be a problem, as we want to check all commits, not just the last one. That's why I tried to use the command line version instead of the action.

As configured, it inserts a comment into the PR only on branches in this repo, not from forks. See https://github.com/commit-check/commit-check-action#fork-pr-comments for explanation. We could enable it from forks. They provide sample workflows at https://github.com/commit-check/commit-check-action/blob/main/docs/fork-pr-comments.md.

[angatha]

Forks seem to be relevant: https://github.com/collective/icalendar/actions/runs/28471153620/job/84383840052?pr=1516
Commits where created by "Zo Bot" but the check did not display that (if he would have caught that at all since the github user name was fine)

Especially that user has many open pull requests with ai suspicion. It would be easy to spot a new one if we would show:

1. amount of open PRs (first time contributor was misleading since all five PRs where open)
2. amount of open PRs flagged as ai suspicion
3. total amount of PRs

[stevepiercy]

Do we want to change the configuration to allow it to run on forks, per my previous comment? There are security concerns with the GitHub Action (GHA) that would need to be addressed, but not so much with the command line version.

The GHA as configured is effectively useless other than seeing what it does now on branches in this repo, because we only grant write access on the repo to trusted individuals.

[angatha]

I guess that was just a warning, because the result could not be written but I expected the job to fail, if he found something? So was there no finding? Then there is no need to take the risk.

[stevepiercy]

I didn't understand what you were talking about from the link you provided, so I dug deeper at https://github.com/collective/icalendar/actions/runs/28471153620/job/84383840052?pr=1516#step:3:165. Here's the warning so other people don't wonder either:

Warning: Skipping PR comment: pull requests from forked repositories cannot write comments via the pull_request event (GITHUB_TOKEN is read-only for forks). See https://github.com/commit-check/commit-check-action/blob/main/docs/fork-pr-comments.md for how to enable PR comments on fork PRs.

As currently configured, the GHA will always succeed on forks.

[stevepiercy]

For the previous user that I blocked and reported, I cloned their fork to my local machine in case they deleted theirs on GitHub. It's sitting on my machine if I ever get around to picking up where they left off.

[stevepiercy]

That didn't take long. Same AI slop from @HrachShah and no response #1524

Ban permanently?

Report to GitHub?

[angatha]

Saw this to late. As I did, I would close PRs from him immedeately if they did not conform with the contribution guide with a hint about the suspension.

[angatha]

Reviewed the profile and I'm thinking about a report to github. The account just spams PR but none was ever accepted except ones on his own repositories.

[stevepiercy]

Blocked @HrachShah permanently and reported to GitHub.

[SashankBhamidi]

I've been lurking on this thread while doing PR reviews, and I think it's time I added my two cents since I'm the one who actually wired up the gh-profiler workflow.

Angatha's suggestion about open PR counts is something I can act on directly. Right now the profiler comment shows account age and contribution history, but it says nothing about how many PRs from this person are currently open. If I had seen "5 open PRs, 4 flagged ai-suspicion" on the first Zo Bot PR, I would have closed it the same day. I'll look into whether gh-profiler supports that signal or whether we need to pull it from the GitHub API ourselves in the workflow and prepend a line. Either way, that's a concrete and low-risk improvement.

On the commit-check-action question Steve raised: I went back and looked at the security picture for running it on forks. The concern with pull_request_target is always that you end up executing untrusted code with write permissions, but the command-line version doesn't have that problem, it just can't write back to the PR. Given that angatha's example with Zo Bot showed the action didn't even fire on the fork PR, I think Steve's read is right: as configured it's effectively decorative. If the command-line path can surface the finding in the workflow logs even without commenting, that's still useful signal for whoever's triaging. But I wouldn't prioritize it over the gh-profiler improvements, because username-level detection (what gh-profiler does) catches the problem at the profile level, not per-commit, which is harder to spoof.

On HrachShah specifically: I'd echo angatha on watching what happens after the suspension lifts. A temporary block + a clear message about what needs to change is the right first step. If the pattern continues unchanged, permanent makes sense. I wouldn't report to GitHub yet; that's a higher-stakes action and the situation doesn't feel like it's crossed that line yet, but I defer to Steve and Nicco here since they've been in the weeds on this longer than I have.

On the broader gatekeeping question: the thing I keep coming back to from this thread is that Nicco's "quick PR closing" idea and the allow list are actually complementary, not alternatives. Closing fast when the template isn't followed raises the floor. The allow list rewards people who clear that floor and stick around. You need both. Without the closing side, the allow list is just a different flavor of the same bottleneck. And given that the broader ecosystem (tldraw, Ghostty, curl's bug bounty) is moving toward much more aggressive measures, closing a PR for missing a changelog entry feels pretty mild by comparison.

Office hours: I like the idea in principle, but I can't commit to a fixed slot. My schedule is unpredictable enough that I'd miss more sessions than I'd make, which would be worse than not signing up at all. That said, I think it's worth doing even without me, and pointing contributors toward it explicitly in the auto-close message would make a real difference, so there's a clear "here's how to actually engage" path when we close a template-incomplete PR.

[SashankBhamidi]

See:

• https://www.coderabbit.ai/blog/ai-is-burning-out-the-people-who-keep-open-source-alive
• https://medium.com/@signadot/open-source-maintainers-are-drowning-in-ai-generated-pull-requests-enterprise-teams-are-next-a598b61b5fbc
• https://www.cncf.io/blog/2026/03/10/sustaining-open-source-in-the-age-of-generative-ai/

[stevepiercy]

As another option for AI slop detection, I just stumbled across https://github.com/marketplace/actions/anti-slop in the GitHub Marketplace after reading the entire article, but jumping to the section What You Can Do Right Now. It appears to check all the boxes for what we want. If y'all think it looks promising, I can make a PR to add it.

I think that commit-check or its GHA are a dead end, so I'd remove it in a separate PR.

I like what gh-profiler offers, and I think it would be good to compare it against anti-slop with some actual experience.

---

Side note, I checked the author of anti-slop, and they're a core contributor to coolify, an OSS project that I've never heard about until today. I read a few docs, and watched parts of some third party video tutorials, and now I'm eager to try it myself.

[niccokunzmann]

I would say go ahead with changes that you would like to see on the project. You can ask if people have concerns but generally, I would like to see initiative rewarded and not blocked by concerns mixed with uncertainty. Moving ahead with this brings clarity. Right now, we need to experiment and expand to solve the issues. We can select later what works when we have the data.

[stevepiercy]

Opened PR #1527.

[stevepiercy]

I just read a post and ensuing replies on Mastodon that clearly expresses why I don't like to review PRs where AI was used.

https://fosstodon.org/@jwildeboer@social.wildeboer.net/116860158248942940

[angatha]

I was reviewing #1536 and noticed that the new PR tempalte check did not marked anything on the PR. (I saw the problems with closing the PR automatically)

The action log shows the faillure but it is not shown on the PR. Action even succeeded:

https://github.com/collective/icalendar/actions/runs/29100015955/job/86386185744?pr=1536

PR is even marked as "PR quality: good" despite having these problems as well as no changelog. When is this lable added?

[stevepiercy]

The configuration has this setting https://github.com/marketplace/actions/anti-slop#max-failures.

          max-failures: 4

I lowered it to 1, which should be a little more helpful to reviewers.

I also lowered min-profile-completeness from its default of 4 to 1. I could drop it to 0, but I think every GitHub user should have a human name. Also gh-profiler handles this in a different way.

See PR #1537

[stevepiercy]

Two AI PRs came through to "fix" #1532.

• Replace parser assertions with runtime validation #1542 is in draft, and I assume the author did that deliberately.
• fix: replace parser assert guards with explicit raises (#1532) #1543 didn't follow our PR template and didn't disclose AI usage.

The anti-slop bot flagged them both. Good bot!

I'd suggest we all wait to review, or even comment on, such PRs until they respond to or comply with the comment we configured in the anti-slop bot, specifically, "Please review Contribute and make any necessary amendments." I prefer that they prove that they're human. If they don't respond within 7 days, close the PR.

However, should I adjust any of the bot settings, per the failed checks? After expanding the checks at the end of each PR, click "PR Quality / anti-slop (pull_request_target)", then the "Summary" tab in the upper left to get a nice view. For convenience:

• https://github.com/collective/icalendar/actions/runs/29189610336?pr=1542
• https://github.com/collective/icalendar/actions/runs/29196858436?pr=1543

[angatha]

Who is a good bot? You are :D

The summary is nice and I'm fine with what they report. "detect-spam-usernames" and "account-age" may fire falsely and we do not give guidance that thouse exist. A failed action would give a hint that contributors can look there but without that, its not a big deal either.

[stevepiercy]

We've recently had a serial AI commenter: https://github.com/collective/icalendar/issues?q=is%3Aissue%20state%3Aopen%20commenter%3Asaisharan0103 I'm inclined to delete their comments as AI slop, as they merely repeat what has already been described, adding no value or pull request.

[stevepiercy]

saisharan0103 has been blocked from the collective organization. Their comments have been hidden as abuse.

[stevepiercy]

I also reported the disruptive behavior to GitHub.

[niccokunzmann]

I would say the commenter can be deleted, blocked and reported. You wrote to them. They did not take care of what their account was doing but instead it looked to me like they would confuse other people. That behaviour is not welcome, not personal, damaging for newcomers, takes up our time. I would welcome not reading another such message.

[stevepiercy]

It looks like comments are the AI slop of the day.

#1543 (comment)

[stevepiercy]

#1432 @aniketaggarwal has been blocked from the collective organization. Their comments have been hidden as low-quality.

[stevepiercy]

#1455 @sangamithraashok244-ship-it has been blocked from the collective organization. Their comments have been hidden as low-quality.

[stevepiercy]

Blocked and reported @santhreal for opening and closing hundreds of low-quality PRs, including:

• Fix AttributeError when VTIMEZONE has duplicate TZID properties #1564
• Fix AttributeError when VALUE parameter is a comma-separated list #1565
• fix: return empty flat map when container appstruct is null Pylons/colander#372

[stevepiercy]

This AI slop comment is exactly the type of comment I'd want to block. It adds nothing to the discussion, a big nothing sandwich. It includes 2-bit words—heuristic, deterministic, distribution, provenance—that no human, other than professional statisticians, would use, and would never intersect with the icalendar community.

It also is self-promotional, with a gist to their personal repo. Blocking and reporting.

[angatha]

Maybe we need to increase description-max-length in the anti slop action. The initial description of #1601 contained some code references but did not seem to be that long. It still required 1435 characters while the maximum allowed was 1000.

[angatha]

@stevepiercy already created an issue #1602 (missed that)

[stevepiercy]

Also a PR #1604

[vibecoder666-max]

Maybe we should do this?

[SashankBhamidi]

Hi @vibecoder666-max, what are you suggesting?

[stevepiercy]

@vibecoder666-max in addition to all that has been suggested and is already being done in this discussion, what specific action should we do? You can also upvote or add reactions to comments, as that seems to be your intent here.

[vibecoder666-max]

From my strictly nihilistic and lion mentality paradigm, don't give up on humanity just yet. Epistemologically speaking, enforcing strict automated templates before humans manually review seems like the most resilient path forward.

[stevepiercy]

Methinks it's a troll.

[SemTiOne]

Worth noting about PR #1730

the author also opened a PR on a repo I maintain: RonaldHensbergen/composable-data-stack#473. I checked the account, they've opened 90+ PRs in 20+ days against 61 external repos. This looks like an unsupervised AI agent.

Worth noting to save time.

[SashankBhamidi]

Thanks for the report, @SemTiOne!

@stevepiercy Off to you.

[stevepiercy]

Closed, blocked, and reported. Thanks for the heads up!

[angatha]

Should we take some actions on the creator of #1738? He didn't even read the document he should be aware of to add the one sentence.

[angatha]

Its so sad that the other contribution also used AI for this.

[stevepiercy]

#1738 is correct. Did you mean #1739?

[angatha]

Nope, I ment #1738. A second delayed commit was required to state AI was use to edit the document that states that they need to disclose AI use.

[stevepiercy]

Its so sad that the other contribution also used AI for this.

Indeed, and the latter didn't bother to check that work had already been done. I guess kids these days don't care about burning down their planet.

[stevepiercy]

There's a spectrum of disciplinary action. IMO, the main purpose of our automated tools and documentation is to help contributors do better.

Those tools in this case took the necessary disciplinary action of warning the contributor of what they did wrong. Then they corrected it. Success! 🎉

Now if the contributor didn't care at all about causing harm—as demonstrated by any of those who I blocked and reported in this topic—then 🚫.

I think one more thing we can do is encourage contributions to add more models to https://github.com/pycalendar/ai-prompt-auto-commit#supported-ai-models. In fact, I just noticed that this is broken in the documentation at https://icalendar.readthedocs.io/en/stable/contribute/index.html#responsible-ai-use. I'll fix that, as I don't want a contributor to try. Edit: nm, it's fine.

[stevepiercy]

I've been getting increasingly annoyed by the verbosity of pull requests and comments generated by AI. Here are the issues I see. Please add as you see fit, and I'll draft a PR to deal with it.

---

GitHub comments and commit messages

GitHub sends a notification to subscribers on every push, comment, and failed CI check.
Additionally GitHub stores a record of these details in its user interface.

There is absolutely no need to repeat any of this information in your comments or commit messages.
It wastes reviewers' time, sends noisy email notifications, and wastes resources.

AI is not helpful in this regard.
AI is verbose and often merely repeats obvious details.

Contributors should be concise when writing their comments and commit messages.
Contributors who aren't concise will delay the merge of their pull requests.
Please don't be that person.

The following is a list of things to exclude.

• Test result counts. They're in the CI checks.
• References to every file or line of code that you changed. They're in the git diff view.
• Anything that the automated GitHub profile check, pull request quality review, documentation pull request preview, or CI check already shows in GitHub's user interface.
• Every single change you make along the way. They're in the git commit message history.

[SashankBhamidi]

I'm thinking to remove me from codeowners file and every Friday I manually review the PRs. I miss a lot of important emails because of the spam.

Our own CI spams PRs with new comments for every new PR that an author opens.

[stevepiercy]

It's almost as if we should entirely ban AI-assisted contributions. hmmm...

Alternatively, you can adjust your subscription not to receive notifications for Issues and Pull requests, leaving the two default settings for @mentions and participations, and checking Discussions, releases, and security.

[angatha]

I went so far that I delete github emails for PRs of this repo automatically and use the notifications page instead.