Presubmit Workflow Fails in Forked Repositories Due to LLM Model Errors #3392
Description
Description
The presubmit workflow (.github/workflows/presubmit.yml) fails to run successfully in forked repositories due to LLM model API errors.
Even after updating the LLM_MODEL field to gemini-1.0-pro or gemini-1.5-flash, the job throws a 404 or “model not found” error.
Attempts to switch to OpenAI models also result in similar workflow failures.
Error Message
Us [AI_APICallError]: models/gemini-1.5-flash is not found for API version v1beta, or is not supported for generateContent.
Call ListModels to see the list of available models and their supported methods.
status: NOT_FOUND
Steps to Reproduce
Fork the repository
Create a new branch and make a small change (e.g., localization or README update)
Push the branch and open a Pull Request
Observe the Presubmit.ai GitHub Action — it fails with the above error
Troubleshooting Done
Tested with both gemini-1.0-pro and gpt-4-turbo
Added valid API keys under LLM_API_KEY
Tried both pull_request_target and pull_request triggers
Re-ran after clearing cache and re-opening PR
Possible Cause
The issue seems to stem from the presubmit/ai-reviewer@latest GitHub Action.
It might be using a hardcoded or deprecated API endpoint for Gemini models and may not handle forked repository PRs correctly.
Environment
Ubuntu 22.04 (local)
Node.js 20.19.4
GitHub Actions runner: ubuntu-latest
Suggested Fix
Update the presubmit workflow to use a currently supported model from the Gemini or OpenAI API list.
Alternatively, disable or skip the AI review step for forked PRs to avoid unnecessary failures.