ci: add mandatory LLM-involvement category disclosure#2343
Conversation
Fixes: 20349ce ("doc: add GitHub issue templates (nix-community#921)")
Fixes: 20349ce ("doc: add GitHub issue templates (nix-community#921)")
Fixes: 20349ce ("doc: add GitHub issue templates (nix-community#921)")
| - [ ] `llm: 0`: Manual: The absence of LLMs would result in equal results and quality | ||
| - [ ] `llm: 1`: Syntactic: Includes non-negligible LLM content |
There was a problem hiding this comment.
llm: 0 is very open to interpretation, where as the other three are significantly more clear. I would prefer having llm: 0 be non-existent LLM usage and llm: 1 be any LLM usage,
| - [ ] `llm: 0`: Manual: The absence of LLMs would result in equal results and quality | |
| - [ ] `llm: 1`: Syntactic: Includes non-negligible LLM content | |
| - [ ] `llm: 0`: Manual: LLMs were absent in the creation of this PR | |
| - [ ] `llm: 1`: Syntactic: Includes LLM generated content |
There was a problem hiding this comment.
llm: 0is very open to interpretation, where as the other three are significantly more clear. I would prefer havingllm: 0be non-existent LLM usage
It intentionally has a faithfull loophole inspired by:
These rules apply only to outside contributions to Ghostty. Maintainers are exempt from these rules and may use AI tools at their discretion; they've proven themselves trustworthy to apply good judgment.
I suggest initially going with this, allowing competent contributors to faithfully disclose their LLM dependency because it seems well-regarded Nix maintainers use LLMs for minor things. Regardless of the label description, it is suspect to dishonesty that I suggest ignoring for now:
Considering the inherent LLM discrimination introduced by the
13.llm: nlabels, malicious or dishonest contributors might intentionally miscategorize their contributions towards13.llm: 0to potentially avoid being ignored by LLM skeptics. Unless it becomes obvious that a non-negligible amount of contributions are intentionally miscategorized, I suggest faithfully assuming contributors to be honest, before designing a LLM ban procedure.
and
llm: 1be any LLM usage
This also intentionally has a smaller loophole, allowing trivial use cases like LLMs fixing trivial syntax issues or suggesting the most idiomatic function to use. I suppose what this really entails will be clear once we see how people interpret this.
I see your point of more explicitly disclosing LLM contributions, but I suggest going with this for now and later determining how much people abuse these faithfull definitions.
| - [ ] `llm: 0`: Manual: The absence of LLMs would result in equal results and quality | ||
| - [ ] `llm: 1`: Syntactic: Includes non-negligible LLM content | ||
| - [ ] `llm: 2`: Semantic: LLM involved in design decisions or core logic | ||
| - [ ] `llm: 3`: Unaudited: LLM output with little to no human verification |
There was a problem hiding this comment.
We should make it clear that is level of LLM involvement is not welcome in Stylix, as was agreed on in #2178.
There was a problem hiding this comment.
We should make it clear that is level of LLM involvement is not welcome in Stylix, as was agreed on in #2178.
IMHO, the categorization is already fairly discriminatory towards LLM contributions, making it somewhat obvious that unaudited contributions are unwelcome. These are also rude submissions, but this is irrelevant to the LLM categorization.
I wonder whether we will even receive submissions categorized as unaudited. Keep in mind that you are free to ignore such submissions or only very superficially interact with them.
This PR should not close #2178 because the automatic labelling has not been implemented.
The
[PATCH 4/5] ci: pull_request_template: add mandatory LLM-involvement disclosure[PATCH 5/5] ci: issue_template: add mandatory LLM-involvement category disclosurepatches have been tested in trueNAHO/stylix-playground#30 and trueNAHO/stylix-playground#29 respectively.
CC: @0xda157
llm: 0: Manual: The absence of LLMs would result in equal results and qualityllm: 1: Syntactic: Includes non-negligible LLM contentllm: 2: Semantic: LLM involved in design decisions or core logicllm: 3: Unaudited: LLM output with little to no human verification