Shadow Reviewers

[xmulligan]

We propose adding a Shadow Reviewers program within Cilium to help new contributors grow their skills and provide more opportunities for them to become active reviewers in the project. The goal is to provide a clearer pathway for contributors to move from code submissions to actively participating in code reviews and becoming trusted reviewers over time.

Problem Statement:

Currently, new contributors might find it challenging to take on the reviewer role, especially without prior experience. While there are guidelines for reviewing contributions, contributors might lack the opportunity to learn the review process in a supportive way. Adding a Shadow Reviewers program would help these contributors build the skills and confidence to eventually become full reviewers.

Next steps:

There is not a great way to do this right now with exiting github tooling. It might make sense to explore how K8s does this with reviewers and approvers. Once we look at that, we should define clear criteria and steps for contributors to transition from shadow reviewer to a full reviewer role. Finally, we should create and maintain documentation that outlines expectations, processes, and best practices for shadow reviewers.

Benefits:

Helps new contributors grow into the role of trusted reviewers.

Reduces the burden on core maintainers by gradually increasing the number of reviewers.

[joestringer]

I'll provide some general context for readers & participants in this discussion about how Pull Request (PR) review works today in most Cilium repositories. This doesn't speak directly to the shadow reviewer proposal, but it may help in reasoning about how such a system may work.

Cilium currently relies heavily on GitHub codeowners to assign review for specific files to specific teams as defined in this repository under ladder/teams. These teams must have write permissions to the repository for the GitHub Code Owners feature to work, and for feedback from team members to count as approval on PRs. A typical team will rely on GitHub load balancing to automatically select a reviewer from each relevant team to review a PR. This has a benefit of distributing and assigning clear owners when someone proposes a change to a specific file. We also don't need to teach all new contributors how to navigate the project structure to find a relevant reviewer. However, due to the elevated permissions, this mechanism is inappropriate for usage by newer untrusted reviewers.

I should note that while the above is the current typical system, we could explore options that are not native to the platform such as Prow that K8s uses.

One idea I had been proposing is to make links for reviews more visible, for example: https://github.com/pulls?q=is%3Aopen+is%3Apr+team-review-requested%3Acilium%2Fdocs-structure+archived%3Afalse+org%3Acilium+ . If someone wants to become a reviewer, they can monitor a list like this (or replace the "team-review-requested" value field with any other team) and directly leave feedback on PRs. This does not require any special privileges, but the reviewers for that specific area will begin to see the new contributor's feedback, and can help steer that feedback (for instance by encouraging those reviews, discussing the nuances with the new reviewer, or providing guidance about the quality bar). I would imagine that for a reviewer onboarding process to work well, we need some combination of buy-in from both the person who is looking to become a reviewer as well as the existing reviewers to build the trust, expectations for the quality bar in the relevant area, and so on.

[lizrice]

Does this relate to the "mentors" nomenclature that you're proposing in #235, @joestringer?

[joestringer]

The mentors nomenclature is documented in this PR from Jef: https://github.com/cilium/community/pull/198/files#diff-150b220b667c94edd471d6e650db25fac0dc724a226eeea450336be7faf9500cR18

Current text:

With membership comes both privileges and responsibilities. One of the responsibilities of being on an active Cilium GitHub team member is being available for automatic pull request review assignment for files owned by that GitHub team.

Over time as contributors transition from one GitHub team to another inside the project, while not active on a team any longer, their historic experience may be needed to help review proposed changes with functionality they have insight into. In these cases these individuals can be added to the team's mentor list in the team configuration file here. Mentors are no longer part of the auto-assigned PR review rotation, but they can still keep team approval access. Active Mentors of a team are expected to provide guidance to others in the team and seek reviews they should participate in for the code owner area.

The thinking above is intended to allow seasoned contributors to lessen the demands on their time to focus on areas where they can have the most impact, while still providing a mechanism for them to contribute their expertise as needed by the project. I didn't have shadow reviewers in mind while considering the above with Jef.