feat(backend): add Google Gemini 3.8 Flash support - #14422
Conversation
- Register google/gemini-3.8-flash in the LLM catalog (OpenRouter route, Google creator), matching the sibling Gemini 3.x entries' shape (context_window=1048576, max_output_tokens=65536, no cache-pricing fields). - Pricing uses Google's intro list price ($0.75/$3.75 per 1M input/output, active through 2026-12-31) converted at the catalog's USD-to-credits factor (x150): input_credits_per_1m=112.5, output_credits_per_1m=562.5. run_credits=3 / price_tier=1, between gemini-3-flash-preview (2/1) and gemini-3.1-pro-preview (5/2) given its GA (non-preview) flash-tier pricing. - Add GEMINI_3_8_FLASH enum member to LLMModel. - Add test_gemini_3_8_flash_bills_at_authored_rates billing-pin test.
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
|
/review |
github-actions
Bot
added
cla: signed
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_61120057-8ff3-405b-8527-2932548c4642) |
|
🤖 Review of |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (15)
WalkthroughAdds the ChangesGemini 3.8 Flash model registration
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to This adds Gemini 3.8 Flash catalog and billing metadata with accompanying billing assertions. No current merge-blocking risk is identified. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 1 conflict(s), 0 medium risk, 1 low risk (out of 2 PRs with file overlap) Auto-generated on push. Ignores: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #14422 +/- ##
==========================================
+ Coverage 81.33% 81.36% +0.02%
==========================================
Files 3516 3516
Lines 263242 263467 +225
Branches 24426 24427 +1
==========================================
+ Hits 214108 214357 +249
+ Misses 43790 43690 -100
- Partials 5344 5420 +76
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
![autogpt-pr-reviewer[bot]](https://avatars.githubusercontent.com/u/130738209?s=60&v=4){kind=small}
There was a problem hiding this comment.
📋 Automated Review — PR #14422
PR #14422 — feat(backend): add Google Gemini 3.8 Flash support
Author: ntindle | Files: 3
🔴 Verdict: Changes needed — failed 1 quality check; not approved.
PR Description Quality
✅ Has Why + What + How — the description states the new model, its slug, pricing basis (×150 USD→credits), run_credits/price_tier, and where it sits relative to sibling Gemini entries. Scope is clearly bounded to 3 additive files.
What This PR Does
Adds Google's google/gemini-3.8-flash model to the backend LLM catalog so users can select it in AI blocks. It registers one enum member (llm_models.py:202), one CatalogModel literal with pricing/context metadata (catalog.py:588–601), and one pinning unit test (catalog_test.py:201–218). This is a pure catalog-as-code data addition — no new logic, routes, I/O, or dependencies.
Specialist Findings
🛡️ Security ✅ — Pure static data addition: no injection surface, no auth/authz change, no new secrets or dependencies. Billing-integrity (the one catalog abuse vector) checks out — 112.5/562.5 credits follow the established ×150 factor and catalog_test.py:201–218 pins the cost mappings so no zero/missing cost path exists.
🏗️ Architecture ✅ — Structurally identical to sibling Gemini entries (same field set, provider="open_router", creator="google"). Fully self-contained in llm_registry/; a search found no other hardcoded model list needing a parallel edit. Correctly uses the direct *_credits_per_1m form, avoiding the USD all-or-nothing constraint.
⚡ Performance ✅ — One immutable catalog record; _build_catalog() is memoized (catalog.py:31,41–44) and MODEL_COST/TOKEN_COST are import-time dicts with O(1) lookups. Adds a constant to a one-time build; zero per-request cost.
🧪 Testing ✅ — test_gemini_3_8_flash_bills_at_authored_rates pins MODEL_COST (=3), TOKEN_COST (incl. cache_read=0.0/cache_creation=0.0), max_output_tokens, price_tier, and context_window with explicit literals — not tautological. Existing structural guards (slug-uniqueness, provider/creator resolution, import-time enum↔catalog pairing) cover the rest.
📖 Quality ✅ — Naming (GEMINI_3_8_FLASH) and field ordering mirror siblings exactly. One cosmetic formatting nit (see below), auto-fixed by the formatter.
📦 Product ✅ — Matches the description precisely and follows the established model-surfacing mechanism. Flags three external facts (slug exactness, list price, GA status) that code review can't validate — advisory, not in-tree defects.
📬 Discussion lint check red on head d193bb4 due to black formatting of the new test; "Check PR Status" fails only as a downstream consequence.
🔎 QA ✅ — Authenticated against the running backend (472-char token, 100 credits). Confirmed google/gemini-3.8-flash appears in live block schemas across 10 LLM blocks; verified cost/metadata resolution matches the authored values in-container; all 17 tests in the file pass; confirmed the model has both cost mappings (not among pre-existing flat-cost models). Could not capture the build-page dropdown screenshot (inference-route image error) — functional verification succeeded, only the visual capture failed.
🟠 Should Fix
- Black formatting fails lint on the new test (
autogpt_platform/backend/backend/data/llm_registry/catalog_test.py:214) — theflash_entry = next(...)lookup is wrapped across three lines; black collapses it to one line, sopoetry run lint/black --checkexits non-zero. This is the local-harness backend lint failure and the discussion specialist's read of the repolintcheck. Runpoetry run formatand commit. (Flagged by: discussion, quality — 2 specialists)
🟡 Nice to Have
- Confirm the OpenRouter slug verbatim (
autogpt_platform/backend/backend/data/llm_registry/catalog.py:589) —google/gemini-3.8-flashmust match OpenRouter's served ID exactly; siblinggemini-2.0entries use a-001suffix while3.xentries don't, so the pattern is ambiguous. A mismatch means user runs 404 at execution time. External fact, not an in-tree defect. (product) - Verify the $0.75/$3.75-per-1M list price at merge time (
autogpt_platform/backend/backend/data/llm_registry/catalog.py:598) — the credits are internally consistent but only correct if Google's GA price matches; wrong values silently mis-bill. External fact. (product, architect)
Human Review Needed
NO — Not required by policy.
Risk Assessment
Merge risk: LOW | Rollback: EASY (revert 3 additive files; no migrations, no state)
CI Status
Local harness: ✅ frontend lint, ✅ frontend typecheck, ✅ frontend test:unit, ✅ frontend build — ❌ backend poetry run lint (black formatting on the new test, matching the Should Fix item).
GitHub CI (head d193bb4): pending — 21 of 26 checks not finished. Because the repository's own lint suite is not confirmed green on this head, the local-harness backend lint failure stands as a real finding rather than environment skew.
UI Testing — Variant Results
❌ local: I'll start with the mandatory Bash setup — auth token, service checks, then exercise the LLM catalog changes. Token echo looks empty. Let me verify and check the LLM catalog endpoint for the new model. Token is empty — sign-in failed. Let me debug and try the sign-up fallback.
✅ hosted: Catalog addition verified end-to-end: all 17 catalog tests pass, runtime cost/metadata resolves correctly (MODEL_COST=3, TOKEN_COST 112.5/562.5), model is served to 10 AI blocks, negative cases behave, no log errors.
- low: HOSTED variant: I verified google/gemini-3.8-flash is registered, priced, and selectable in the AI Text Generator model dropdown (Build page) and served in 10 block schemas, but did not perform a live LLM execution against OpenRouter, so upstream availability of the model ID google/gemini-3.8-flash was not confirmed. This is outside the PR's data-only catalog scope; noted for completeness.
|
/review |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_90a12586-636b-489d-b517-adb9277f0655) |
|
🤖 Review of |
github-actions
Bot
added
cla: pending
There was a problem hiding this comment.
❓ Verdict: Review incomplete — discussion: GitHub API access failed with HTTP 401 (invalid gh token for autogpt-pr-reviewer[bot]). Unable to run gh pr checks, gh pr view --json headRefOid,mergeable, or the reviewThreads GraphQL query, so CI status on the current head and open human-reviewer threads could not be assessed. Rerun after restoring valid gh credentials with repo read scope.; fix the review setup or missing evidence and rerun.
Additive Gemini 3.8 Flash catalog/enum entry with correct ×150 credit math and a strong billing pin test; all specialists and live QA approve on substance. The prior black-formatting blocker is resolved on head f5a5b3e, where GitHub CI is green across all 45 checks. Only a non-blocking advisory to confirm the OpenRouter slug remains.
Risk level: low | Human review: not required | Duration: 1408s | Reviewed: f5a5b3ed
Findings: 🔴 0 blockers | 🟠 0 should fix | 🟡 1 nice to have | 🔵 0 nits
Optional advice — does not block approval
🟡 Nice to Have
- 🔵
autogpt_platform/backend/backend/data/llm_registry/catalog.py:589Confirm the OpenRouter slug and intro price resolve upstream — The slug 'google/gemini-3.8-flash' is the exact string routed to OpenRouter at execution time and its intro pricing is author-provided; neither can be verified from the diff, and the 3.1 -> 3.8 jump is unusual next to siblings. A mismatch would surface only as a runtime routing failure for this one model and is not caught by the derivation test.
Suggestion: Have the author confirm the OpenRouter model ID resolves and the list price is current; correctable via the documented catalog hotfix fast-path if it later proves wrong.
GitHub CI on reviewed head: success
Validation and specialist details
Local harness
- ✅ lint: cd autogpt_platform/frontend && pnpm lint:
cd autogpt_platform/frontend && pnpm lint(94s) - ❌ lint: cd autogpt_platform/backend && poetry run lint:
cd autogpt_platform/backend && poetry run lint(108s) - ✅ typecheck: cd autogpt_platform/frontend && pnpm types:
cd autogpt_platform/frontend && pnpm types(52s) - ✅ test: cd autogpt_platform/frontend && mv .env /tmp/qa-env-stash 2>/dev/null; pnpm test:unit; rc=$?; [ -f /tmp/qa-env-stash ] && mv /tmp/qa-env-stash .env; exit $rc:
cd autogpt_platform/frontend && mv .env /tmp/qa-env-stash 2>/dev/null; pnpm test:unit; rc=$?; [ -f /tmp/qa-env-stash ] && mv /tmp/qa-env-stash .env; exit $rc(591s) - ✅ build: cd autogpt_platform/frontend && pnpm build:
cd autogpt_platform/frontend && pnpm build(322s)
Specialist Reports
| Specialist | Status | Summary |
|---|---|---|
| security | ✅ PASS | Additive, data-only LLM catalog entry for Gemini 3.8 Flash with correct credit math and a pinning test; no security-relevant surface introduced. |
| architect | ✅ PASS | Additive, pattern-conformant Gemini 3.8 Flash catalog/enum entry with a billing regression test; no architectural concerns. |
| performance | ✅ PASS | Additive static catalog/enum entry with no query, allocation, concurrency, or scalability impact; runtime lookups remain O(1) against import-time-cached dicts. |
| testing | ✅ PASS | Additive Gemini 3.8 Flash catalog entry with a strong, exact-value billing regression test that passes locally; no test defects found. |
| quality | ✅ PASS | Clean, convention-following additive catalog entry for a new LLM model with a matching billing regression test; no code-quality defects found. |
| product | ✅ PASS | Well-scoped additive Gemini 3.8 Flash catalog/enum entry with consistent pricing math and a billing pin test; no product or UX defects. |
| discussion | Cannot validate CI status on head or open human-reviewer requests because the GitHub API returns HTTP 401 (invalid gh credentials); review is incomplete. | |
| ui-reviewer (local) | ✅ PASS | Additive, well-formed catalog entry for Gemini 3.8 Flash with correct credit math and passing billing tests; no reachable defect and no UI/auth surface affected. |
| ui-reviewer (hosted) | ✅ PASS | Clean additive backend LLM catalog entry for Gemini 3.8 Flash with correct, sibling-consistent credit math and passing billing tests; no defects found. |
|
/review |
|
🤖 Review of |
There was a problem hiding this comment.
✅ Verdict: Approved — No findings require changes before approval; this review counts as an approval.
Additive Gemini 3.8 Flash catalog/enum entry with correct ×150 credit math and a strong billing-pin test. All six specialists and live backend QA approve on substance; GitHub CI is green across all 45 checks on head f5a5b3e. Only a non-blocking advisory to confirm the OpenRouter slug remains.
Risk level: low | Human review: not required | Duration: 226s | Reviewed: f5a5b3ed
Findings: 🔴 0 blockers | 🟠 0 should fix | 🟡 1 nice to have | 🔵 1 nit
Optional advice — does not block approval
🟡 Nice to Have
- 🔵
autogpt_platform/backend/backend/data/block_cost_config/catalog.py:589Confirm OpenRouter slug resolves at runtime — The slug google/gemini-3.8-flash is a fixed literal that routes to OpenRouter; its real-world existence and intro list price cannot be verified from the review sandbox. This is a product-accuracy question, not a code defect.
Suggestion: Verify the slug and intro pricing against OpenRouter/Google before or shortly after merge.
Optional advice — does not block approval
🔵 Nits
- 🔵
autogpt_platform/backend/backend/data/block_cost_config/llm_models.py:202Group enum member with other 3.x entries — GEMINI_3_8_FLASH is inserted between GEMINI_3_1_FLASH_LITE_PREVIEW and GEMINI_2_5_FLASH_LITE rather than grouped with the 3.x members. Ordering is cosmetic and non-functional.
Suggestion: Optionally relocate next to the other gemini-3.x members at next touch.
GitHub CI on reviewed head: success
Validation and specialist details
Specialist Reports
| Specialist | Status | Summary |
|---|---|---|
| security | ✅ PASS | Additive, internally-consistent LLM catalog entry for Gemini 3.8 Flash with a proper billing-guard test; no security, auth, or execution-path impact. |
| architect | ✅ PASS | Clean additive catalog-as-code entry for Gemini 3.8 Flash with matching enum member and billing-guard test; follows established patterns with no architectural concerns. |
| performance | ✅ PASS | Additive static LLM catalog entry with no new runtime path, query, or loop; zero performance or scalability impact. |
| testing | ✅ PASS | Additive LLM catalog entry with a meaningful, exact-value billing guard test that follows the established forever-guard pattern and would catch projection regressions. |
| quality | ✅ PASS | Clean additive catalog + enum + test for Gemini 3.8 Flash that faithfully follows the established catalog-as-code pattern with correct pricing math and conventional test coverage. |
| product | ✅ PASS | Clean additive catalog/enum entry for Gemini 3.8 Flash with consistent pricing conversion and a billing-pin test; no user-facing defects. |
| discussion | ✅ PASS | All required CI checks pass on head f5a5b3e, the PR is mergeable against dev, and there are no open requests from human reviewers. |
| ui-reviewer (local) | ✅ PASS | Additive, internally consistent Gemini 3.8 Flash catalog/enum entry with a pinning billing test; full catalog build and MODEL_COST/TOKEN_COST derivation verified, no defects and no paywall UI. |
| ui-reviewer (hosted) | ✅ PASS | Additive, internally-consistent Gemini 3.8 Flash catalog+enum entry; all 17 catalog tests (pairing, uniqueness, billing snapshots, new pinning test) pass and pricing math checks out — no UI surface and no reachable defect. |
github-project-automation
Bot
moved this from 🆕 Needs initial review
to 👍🏼 Mergeable
in AutoGPT development kanban
1 participant
Summary
Google released Gemini 3.8 Flash (GA 2026-09-02) and it was missing from AutoGPT's LLM catalog. This adds it following the existing catalog-as-code pattern.
Changes
CatalogModelentry forgoogle/gemini-3.8-flash(provider=open_router, creator=google), placed alongside the othergoogle/gemini-3.xentries.context_window=1048576,max_output_tokens=65536, matching sibling Gemini 3.x entries. No cache-pricing fields, consistent with the other OpenRouter-routed Google models in this catalog.input_credits_per_1m=112.5,output_credits_per_1m=562.5.run_credits=3,price_tier=1— positioned betweengemini-3-flash-preview(run_credits=2, tier=1) andgemini-3.1-pro-preview(run_credits=5, tier=2), reflecting GA flash-tier pricing above a preview flash model but well below the pro tier.GEMINI_3_8_FLASH = "google/gemini-3.8-flash"enum member.test_gemini_3_8_flash_bills_at_authored_rates, pinningMODEL_COST,TOKEN_COST,max_output_tokens,price_tier, andcontext_windowfor the new model.Scope note
Checked open PR #14386 ("add direct Google Gemini LLM provider support") — it registers six existing Gemini model IDs as part of a separate "direct provider" architecture feature and does not touch
gemini-3.8-flash, so this is a distinct, non-overlapping addition.Verification
Since the full
backend.blocksimport chain requires a generated Prisma client unavailable in this environment, I verified correctness with a standalone script that imports onlybackend.data.llm_registry.llm_modelsandbackend.data.llm_registry.catalog, reimplements the same_model_cost_from_catalog/_token_cost_from_catalogprojection logic fromblock_cost_config.py, and asserts against it — confirmingMODEL_COST,TOKEN_COST, and metadata all resolve correctly, and that noLLMModelmember is missing a cost mapping. All assertions passed.Note
Low Risk
Additive catalog and enum entry with an existing billing guard test; no auth, routing, or execution-path changes beyond making a new model available.
Overview
Registers Gemini 3.8 Flash (
google/gemini-3.8-flash) in the canonical LLM catalog so it can be selected and billed like other OpenRouter Google Gemini models.The new
CatalogModelsits with the other Gemini 3.x entries: OpenRouter routing, 1M context / 65K max output, tier 1, and per-token credits derived from Google’s intro list pricing (3 run credits; 112.5 / 562.5 input/output credits per 1M).llm_models.pyaddsGEMINI_3_8_FLASHon theLLMModelenum, andcatalog_test.pyaddstest_gemini_3_8_flash_bills_at_authored_ratessoMODEL_COST,TOKEN_COST, and metadata projections stay aligned with the authored catalog row.Reviewed by Cursor Bugbot for commit f5a5b3e. Bugbot is set up for automated code reviews on this repo. Configure here.