Skip to content

Conversation

@JairoTM-es
Copy link

Description (required)

Fixes #6588

What changes did you make and why?

This PR improves the reliability of the Check Pull Request Scope GitHub Actions workflow.

  1. Fixed 403 error when posting out-of-scope comments

    • The workflow was failing with HttpError: Resource not accessible by integration (403) when the github-script step attempted to create a comment on a pull request.
    • This happened because the workflow GITHUB_TOKEN did not have sufficient permissions.
    • Updated the workflow permissions to include:
      contents: read
      issues: write
      pull-requests: write
      models: read
      This allows the workflow to create PR comments successfully.
  2. Made the AI-based scope detection more deterministic and consistent

    • The same pull request diff was sometimes classified differently across runs.
    • To reduce variability and make outputs stable:
      • Set temperature: 0
      • Limited output using max-tokens: 10
      • Updated the prompt to clearly define:
        • what counts as related vs unrelated changes
        • several examples for both categories
        • a strict requirement for the model to return exactly "yes" or "no" in lowercase, with no extra text
    • These adjustments help ensure that the condition
      if: steps.analyze.outputs.response == 'yes'
      behaves consistently.

Tests performed (required)

This PR modifies only a GitHub Actions workflow and does not affect the Android application code or UI.

  • Verified locally that the YAML syntax is valid.
  • Relying on CI to execute the updated workflow on this pull request.
  • No device or emulator testing is required for this type of change.

Screenshots (for UI changes only)

Not applicable — this PR does not introduce UI changes.


Note: I have reviewed CONTRIBUTING.md as recommended for first-time contributors.

- Fixed workflow permissions so github-script can post PR comments (403 error)
- Set temperature to 0 and limited max-tokens to 10 for deterministic yes/no outputs
- Updated the prompt with clear related/unrelated definitions and strict yes/no requirements
- Fixed workflow permissions so github-script can post PR comments (403 error)
- Set temperature to 0 and limited max-tokens to 10 for deterministic yes/no outputs
- Updated the prompt with clear related/unrelated definitions and strict yes/no requirements
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check_scope workflow is not reliable enough

1 participant