[opentelemetry.io] Fine-grained organization token with Copilot access

[vitorvasc]

Description

In the opentelemetry.io repo, we've been working on an AI content detection workflow that uses the Copilot CLI to analyze PR diffs and detect potentially AI-generated content. See the related issue on open-telemetry/opentelemetry.io#8520 and the implementation on open-telemetry/opentelemetry.io#8637.

The current workflow is ready to be deployed and tested, but we've hit an authentication limitation (see the workflow run) that didn't show up while testing it in private repos (and also outside an organization).

The current autentication setup uses the otelbot GitHub App, which works great for standard GitHub API operations like reading PRs, posting comments, or adding labels.

The current issue with the workflow is that the Copilot CLI requires user-level permissions, and the GitHub Apps tokens apparently cannot be granted these permissions to access the Copilot CLI. To access the Copilot CLI within a workflow, we would need to create an organization secret containing a fine-grained Personal Access Token (PAT) with the "Copilot Requests" permission. The workflow would then use:

• The otelbot token for all repository operations (continues as-is).
• Organization fine-grained access token for Copilot CLI authentication only.

Given the requirement for a user-level PAT, and based on my current understanding that the token creator must individually possess a Copilot license, I have some questions regarding the setup for this token:

1. Can we take advantage of the organization's Copilot license instead?
2. Is it possible to issue this fine-grained token without tying its usage and licensing to an individual user?

/cc @open-telemetry/docs-maintainers

[svrnm]

@trask I suspect we can drive this through the admin repo? Would you be able to help @vitorvasc with that?

[trask]

I suspect we can drive this through the admin repo?

probably not, the GitHub App stuff isn't supported through terraform at all

@vitorvasc I'm not clear on the steps needed here, in particular I'm not finding any way to do this:

Organization fine-grained access token for Copilot CLI authentication only.

I think tokens still need a user or bot account.