Skip to content

Require disclosure of tool (e.g. LLM) usage to generate code #8625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

abadams
Copy link
Member

@abadams abadams commented May 9, 2025

Vibe coding is a hot topic. In our experience, it doesn't produce acceptably-good code right now, but it is steadily improving, so we probably need a policy on it. Following a discussion in the dev meeting, we feel that it's fine to use tools like LLMs to generate Halide PRs. They will be held to the same standard as any other PR. The human that opens the PR will be treated as responsible for the code. In fact, we already have a history of accepting tool-generated code (some simplifier rules were discovered by superoptimization). However, we decided contributors must disclose the tools used in any significantly-tool-generated code, for a few reasons.

  1. You shouldn't claim credit for something you didn't do.
  2. It affects how a reviewer will review the code (e.g. what kind of bugs they will look for).
  3. In a variety of possible future scenarios, it may be useful to know which commits were substantially authored by these tools.

We decided the commit message was the right place to put the disclosure.

The credit issue makes it worthy of insertion into the code of conduct.

To set an example: The sentence added to the code of conduct below was suggested by Gemini 2.0 Flash.

Vibe coding is a hot topic. In our experience, it doesn't produce acceptably-good code right now, but it is steadily improving, so we probably need a policy on it. Following a discussion in the dev meeting, we feel that it's fine to use tools like LLMs to generate Halide PRs. They will be held to the same standard as any other PR. The human that opens the PR will be treated as responsible for the code. In fact, we already have a history of accepting tool-generated code (some simplifier rules were discovered by superoptimization). However, we decided contributors must disclose the tools used in any significantly-tool-generated code, for a few reasons. 1) You shouldn't claim credit for something you didn't do. 2) It affects how a reviewer will review the code (e.g. what kind of bugs they will look for). 3) In a variety of possible future scenarios, it may be useful to know which commits were substantially authored by these tools.

The credit issue makes it worthy of insertion into the code of conduct.
@alexreinking
Copy link
Member

To set an example: The sentence added to the code of conduct below was suggested by Gemini 2.0 Flash.

It doesn't look like this made it into the commit message.

it may be useful to know which commits were substantially authored by these tools

To that end, we should probably use a standard format. Something like

Co-authored-by: Google Gemini 2.0 Flash

Not sure how GitHub will respond to a missing email from this trailer, but it should remain searchable via git log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants