New RFC: nf-core should have a policy about use of LLMs/AI

[jfy133]

Have you read the RFC docs?

• Yes, I have read and understood the RFC docs

Summary

We should have a clear policy on when it is OK/not OK, and how to use LLMS for generating new code and PRs.

Champion

No response

Background & Motivation

Derived from the proposal here: #61, some members of the core team and community were not comfortable to immediately start 'blindly' allowing LLM associated code within the nf-core ecosystem, primarily due to:

• Quality of code in some PRs
• Concerns about the legality/lack of attribution
• Adding a lot of extra cruft to the template purely to try and make sure AI agents 'do the right thing' being a bad thing

with particularly the first point ending up putting more work on community coordinators/managers.

Generally there was an agreement that LLMs are here to stay, and we cannot avoid them.

But there was a feeling we need to at least make a statement on our stance on them and possibly a policy document.

Goals

• Make clear when LLM code or related things are (generally) acceptable within nf-core
• Make clear what the consequences of abusing these guidelines are

Non-Goals

• To enforce AI usage on the community
• To ban AI usage
• To implement mechanisms for the use of AI

References

#61

[nf-core-bot]

Approval status: 🕐 Pending

RFC has approvals from 4/7 required @nf-core/core quorum.

Review Status	Core Team members
✅ Approved	@maxulysse, @mribeirodantas, @mirpedrol, @mashehu
🕐 Pending	@ewels, @JoseEspinosa, @FriederikeHanssen, @kenibrewer, @jfy133, @edmundmiller, @sateeshperi, @FranBonath, @christopher-hakkaart, @nvnieuwk

[maxulysse]

/approve

[jfy133]

To copy from #61 this is what I original proposed:

I also agree the general sentiment that we basically won't be able to prevent it AI contributions, even though my personal experience of LLMs remains to be pretty poor except for extremely specific cases, so I'm still relatively skeptical

Regardless of crappy chatbots and the social implication of thise, I think coding is at least one the lesser controversial applications of AI and can sometimes work well. Also given all of our code is under a very explicitly open license, it is reasonasble to allow AI to have been trained and given suggestions for our own code base (based on our own code base, which I presume is some of some of the biggest sources of data for nextflow parts of models)

So to be more practical here

I think I would start first of setting up a specific policy, i.e., something along the lines of:

1. We do allow it to assist in writing and submitting PRs including code changes
2. All use of AI-assisted code generation must have the AI tagged as a co-author (note: this might be cool for stats ;) )
3. The human author however is entirely responsible for their PRs - repeated spam or low-quality slop will result in a ban
4. AI-assisted code generation should use AGENTS.md where possible

Number 4 can be implemented in piece meal fashion, allowing AI-friendly devs to add it themselves if they wish from a template we provide, and if it is truly helpful and not abused, can slowly roll it out.

Or something like this. Thoughts @nf-core/core @nf-core/maintainers (and of course @ammaraziz )

Point 3 is the trickest one maybe, so we will need have a process for this (most likley: report to core team, core team warns with author, if continues removal from nf-core organisation or something like that)

[pinin4fjords]

I remain to be convinced that we need anything more than current guidelines on testing/ code quality/ linting etc. I would treat code the same if it's crap due to LLMs as I would if it was crap due to humans.

I also worry that the lines are ill-defined. Most of my code from the last year or so has some component of LLM input, but it's highly variable how useful that input has been and it's hard to remember how much input there as been.

[prototaxites]

I am not sure it is that useful to have a set of strict rules on how and where LLM-generated code can be used. I do think we need to accept that this kind of code is going to come - but I would rather write a some kind of document, if we are going to do it, that focuses on /how/ to get the best out of them - a list of recommended good practice that we can point folk at.

From my experience trying LLMs, that could be stuff like:

• LLMs are great at solving small problems and prompting better ways to structure your code
• LLMs (often!) make mistakes and you should always seek to understand the code it writes for you
• LLM agents, if allowed, will try and do big drastic things to the codebase - this is not good in the kind of environment where we want PRs to be minimal for easy review

But beyond that, how the code gets written is not so relevant, so long as what comes to us is a) good, and b) not in such a deluge that we can't actively deal with it.

edit: and it would be good to require a policy of openness and honesty about where and how LLMs have been used. Not necessarily up-front (I don't think we need to credit LLMs as authors, they're machines), but if asked

[pinin4fjords]

Heavy +1 for "Here's how to use LLMs well for contributions" rather than "here's a bunch of rules we want you to follow when using LLMs"

I don't believe anyone will be writing this stuff completely manually in a year or two, lots of rules won't age well.

[ewels]

Agree with the above comments 👍🏻 A brief guide, mention of the fact that we will close PRs without proper review if they are too big and don't look like they have had enough human intervention, etc. Anything remotely detailed will be out of date by the time the PR is approved, so doesn't need to be at all long I think, just something to link to when we close PRs that are AI-slop.

I think it amounts to the same thing as proposed by @jfy133 but the way its framed comes across nicer and more helpful imho.

[FriederikeHanssen]

It will probably help if we start enforcing single-feature PRs more strongly as well. Whether these are written by an LLM or my sad artisanal attempts won't matter. Same rules apply: author is responsible, PR description should explain why and what, changes should affect single features, tests should be added and running, etc.....

[adamrtalbot]

l probably help if we start enforcing single-feature PRs more strongly as well. Whether these are written by an LLM or my sad artisanal attempts won't matter. Same rules apply: author is responsible, PR description should explain why and what, changes should affect single features, tests should be added and running, etc.....

Bingo. The same rules apply to AI as they do to humans. Keep PRs small and focused, with a good description of why you are introducing changes and the impact in the PR description. The onus is on the submitter to make sure the code works before they ask for a review. In some ways the use of LLMs has uncovered the shortcomings in the process that have always been there.

If we tighten up the acceptance guidelines we should cover human written code and sloppy AI usage:

• PRs should only change the relevant files
• A PR should introduce a single change and one change only
• A PR should accurately describe why it is introducing the change and what it affects
• A PR should be affect as little code as possible

Any violation of these should be grounds for rejection without asking for more reviewers time.

Come to think of it, do we have guidance for reviewers? I see guidance for authors.

[FloWuenne]

I also agree with everyone's comments regarding rather guiding people into how best to use LLMs to help write and maintain good code and coding practices, rather than setting some imaginative barriers where and how LLMs can be used within nf-core. If used correctly, LLMs can definitely reduce the time to implement features or changes and help provide proper documentation (which otherwise is often missing to start with).

While I generally agree with @adamrtalbot idea of very targeted PRs with very small changes, the reality is that it's hard enough to start contributing to nf-core. By imposing extremely strict acceptance criteria just to open a PR, I feel that would set this bar to first contribution even higher. Rather than enforcing strict rules to ourselves, we should encourage best practices and potentially use the help of LLMs to break down large PRs into digestible chunks. That after all is what LLMs are extremely good at, consuming large amounts of information to help us better understand the context.