Improving GH Issue Triaging

[digantdesai]

Hello ExecuTorch,

During my oncall experience, I encountered several issues that I believe are worth discussing here.

We have multiple categorization systems on Github for Issues:

• Issue Labels: bug, enhancement, feature, rfc, and help wanted.
• Issue Types: Bug, Task, and Feature.
• Github sections: Discussion, Bugs, and Project.

I would like to discuss the following points:

1. We have overlapping Labels and Types for Issues, such as bug and feature. Should we consider replacing those Labels with Types?
2. The guidance for applying issue Types can be improved. For example, it's not clear to me when to use the Task type versus the Feature type.
3. Establishing clear guidelines with examples would be beneficial in determining what constitutes a Discussion, a question, or an RFC.
4. What is the primary purpose of the Bugs section on Github? How do we intend to differentiate between bugs and issues?
5. Should assigning a Project be a qualifying criterion for marking an issue as Triaged?

cc @manuelcandales

[mergennachin]

cc @dbort

[dbort]

We have overlapping Labels and Types for Issues, such as bug and feature. Should we consider replacing those Labels with Types?

I think that seems reasonable; better to use built-in github concepts when we can. We'd need to do a one-time migration and update the docs/runbook to match.

The guidance for applying issue Types can be improved. For example, it's not clear to me when to use the Task type versus the Feature type.

I see Feature as adding a new concept or option to the user-facing product. Task is like "do this internal refactoring", or "add a CI job to test X", things that users won't notice directly.

Should assigning a Project be a qualifying criterion for marking an issue as Triaged?

I was thinking along the same lines. Maybe we should retire "module:" labels in favor of a Project per module. Oncall would be responsible for moving an issue to the right Project, and then their job would be over. Project owners would tier-2 triage issues that are added to their Project, getting extra info from the author if necessary, assigning priority, scheduling etc.

Or we could have one super-Project and module owners would filter by their "module:" label. Oncall would just add the label and add it to the super-Project, and then module owners take care of it from there.

[byjlw]

Yeah I've never liked the module:x terminology. The prefix just adds extra text

[byjlw]

Thanks for spinning this up. I have some ideas, but won't have a chance to fully comment until Friday

[byjlw]

Here are my thoughts.
Follow-up to this would include: Defining the set of labels and RFC template

Issue Categorization

1. Use of Issue Types:
   • Bug: For confirmed defects or errors in the system.
   • Task: For work items that involve maintenance, refactoring, or documentation updates.
   • Feature: This should be reviewed and in most cases converted to a discussion item in the idea category
2. Labels:
   • Use labels for additional context, such as user experience, QNN, needs-discussion.
   • Avoid using labels that overlap with issue types (e.g., do not use bug or feature as labels).

Discussions, Questions, and RFCs

1. Discussion:
   • Use for open-ended topics that require community input or brainstorming.
   • Example: "Exploring new design patterns for the UI."
   • Often it's helpful to have an open ended discussion before creating a formal RFC
2. Question:
   • Use for inquiries that seek specific answers or clarifications.
   • Example: "How does the authentication module handle token expiration?"
3. Ideas:
   • Idea: feature request, idea or discussion that may result in an RFC in the future. It may not have a solution and there is no formal template
   • RFC (Request for Comments)
     • Use for formal proposals that may require feedback before implementation.
     • Example: "Proposal to migrate the database to a new schema."
     • These are well-formed, and include both a problem and a solution described in enough detail that the someone other than the author can implement

Bugs Section

• Purpose: The Bugs section is dedicated to tracking and managing confirmed defects.
• Process:
  • Only issues verified as defects should be moved to the Bugs section.
  • Use status labels such as confirmed, in progress, and resolved to track the bug's lifecycle.

Project Assignment and Triaging

1. Project Assignment:
   • Assigning an item to a project is mandatory for an issue to be marked as Triaged.
   • Projects help in organizing and prioritizing work effectively.
2. Triaging Criteria:
   • An issue is considered Triaged when it has:
     • A clear and detailed description.
     • An assigned priority label.
     • A designated project.

[dbort]

@mergennachin and I are putting together a proposal for how we can use Projects and Milestones in the mix here, and it complements a lot of the suggestions from this thread. We're also working on some recommendations for clarifying ownership and oncall responsibilities.

[byjlw]

excellent

[mergennachin]

@digantdesai @manuelcandales @jackzhxng @SS-JIA @mcr229 (recent oncalls)

@cbilgin @byjlw @iseeyuan @kimishpatel @ali-khosh @orionr (leads)

As @dbort mentioned above, here's our current proposal that we would like to standardize. It is related to @digantdesai 's original discussion.

https://docs.google.com/presentation/d/1fPHxaadzoeVZiuw8iY7m08qXnWRwp9AJHWQNKrcTLis/edit#slide=id.p

[kimishpatel]

Are slides reflective of the other changes proposed here. e.g. deprecate 'module' prefix from labels etc. But rest look like good change to me

[jackzhxng]

Agree with 1, if we don't have any other workflows that break because of this (@kirklandsign) I'd prefer to get rid of feature/task/bug labels and use type instead

[mergennachin]

Okay, let's do "migration" then for existing issues and delete those labels altogether.