refactor: update Live API configuration, transcription support, and i…#1224
Conversation
…nput handling for model version gemini-3.1-flash-live-preview
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly updates the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the Live API quickstart to use the google-genai SDK (v1.0.0+), involving model and API key updates, enabling audio transcriptions, and refactoring the audio reception logic. The review feedback highlights several necessary corrections for SDK compatibility, such as the proper usage of send_realtime_input and send_client_content. Additionally, improvements were suggested regarding the repository's style guide for model selection, the use of constants for sample rates, and the handling of incremental transcription updates.
* Bump uuid and googleapis in /quickstarts/file-api (#1212) Removes [uuid](https://github.com/uuidjs/uuid). It's no longer used after updating ancestor dependency [googleapis](https://github.com/googleapis/google-api-nodejs-client). These dependencies need to be updated together. Removes `uuid` Updates `googleapis` from 134.0.0 to 171.4.0 - [Release notes](https://github.com/googleapis/google-api-nodejs-client/releases) - [Commits](googleapis/google-api-nodejs-client@googleapis-v134.0.0...googleapis-v171.4.0) --- updated-dependencies: - dependency-name: uuid dependency-version: dependency-type: indirect - dependency-name: googleapis dependency-version: 171.4.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Flash-Lite 3.1 goes GA (#1221) * Create README.md for the prompting_techniques folder (#1230) --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Added new notebook Basic_Code_Review to examples (#689) --------- Co-authored-by: kkorpal <kkorpal@google.com> Co-authored-by: Shipra <Shi-pra-19@users.noreply.github.com> * Add Basic_Code_Review example to README (#1218) * Add Basic_Code_Review example to README * Update examples/prompting/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Added example for gemini api used to extract insights from github #923 #887 (#1150) * added example for gemini api used to extract insights from github --------- Co-authored-by: kkorpal <kkorpal@google.com> * Add example: EU AI Act compliance checking with Gemini function calling + Agent Module (#1167) * Add EU AI Act compliance example with Gemini function calling + Agent Module * style: use triple-quoted string per review feedback * fix: correct API schema + formatting per review - status["ethics"] -> status["ai_compliance"] (v0.9.21 schema rename; was the KeyError that blocked the reviewer run) - Remove unnecessary backslash line-continuations in the compliance_run prompt string; indent body; newline before closing """ - Replace static 22-row module table with dynamic discovery pointer (/api/status, /api/demo, /llms.txt) so the snapshot never goes stale - Branding: "ethics modules" -> "AI Compliance modules" in human-facing markdown; vertical slug documented as ai-compliance with ethics as legacy alias (both still accepted by the API) - Correct ETH_001 EU AI Act mapping (Art. 9 + 10, not 10 + 13) * fix: prevent trial key collision on reruns Follow-up to the schema/formatting fixes. Colab testing surfaced a runtime bug: the Agent Module API returns a masked preview of an existing trial key (e.g. `am_trial_841...f660`) if you request a new one while the old one is still active, and the original notebook wrapper blindly stored the preview — corrupting _trial_key and breaking every downstream retrieve_compliance_logic call. Changes: - Generate a unique per-run agent_id (uuid suffix) so reruns within 24 hours do not collide with a still-active trial on the server - Make get_trial_key idempotent (short-circuit if _trial_key already set in this session) so Gemini tool loops cannot re-trigger it - Detect status=="already_active" in the MCP response and refuse to store the masked preview; surface a clear error instead - Remove get_trial_key from the tools list passed to Gemini. Trial bootstrap is a setup step, not a runtime tool — the model should only see retrieve_compliance_logic and submit_proof_of_value - Reframe the cell 27 preamble to explain the setup/runtime split * fix: pass both token and trial_key to submit_pov MCP tool * fix: print full chat history so cell 30 shows the complete compliance report * chore: save executed notebook with cell outputs * fix: add Colab button + README link for CI compliance - Add Colab badge to notebook title cell (tfo-notebook-buttons CI check) - Add notebook entry to examples/README.md table (new-example-links CI check) * chore: retrigger CI checks * Fix notebook lint (inclusive_language) + nbfmt format - Rewords 'Agent-native' to 'Purpose-built for autonomous agents' in the branding cell to clear google::inclusive_language lint flag - Applies tensorflow_docs.tools.nbfmt formatting pass to clear Notebook format check - No other content changes; all 21 cell outputs preserved * Fix colab.name metadata + cell 30 prompt (third-person) to clear nbfmt + second_person checks --------- Co-authored-by: Guillaume Vernade <Giom-V@users.noreply.github.com> * add gemini-3.5-flash (#1232) * New agent notebook (#1233) * Fixing typos * Adding links to the agents notebooks (#1234) * docs: fix typos and capitalization consistency in README (#1240) * docs: fix typos and capitalization consistency in README * docs: apply gemini suggestion Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Bump qs from 6.15.1 to 6.15.2 in /quickstarts/file-api (#1238) Bumps [qs](https://github.com/ljharb/qs) from 6.15.1 to 6.15.2. - [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md) - [Commits](ljharb/qs@v6.15.1...v6.15.2) --- updated-dependencies: - dependency-name: qs dependency-version: 6.15.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix scale= command in Lyria realtime quickstart (#1236) The scale= parser compared the raw prompt (which still contains the 'scale=' prefix) against enum names like A_FLAT_MAJOR_F_MINOR, so the match never succeeded and the command always printed 'Matching enum not found.' Strip the 'scale=' prefix before comparing, matching how the sibling bpm= and top_k= handlers already work. * update live api ws cookbook (#1223) * Interactions steps (#1237) * Merge origin/main into interaction-steps and resolve conflict * steps * Add environment support * work * cleanup * format * cleanup * Add back execution_count * Standardize Colab buttons and fix URLs across edited notebooks * feat: update visual model names to GA and add video-to-image example (#1244) * feat: update visual model names to GA and add video-to-image example - Updated gemini-3-pro-image-preview to gemini-3-pro-image (Nano Banana Pro GA) - Updated gemini-3.1-flash-image-preview to gemini-3.1-flash-image (Nano Banana 2 GA) - Added a new video-to-image example in Get_Started_Nano_Banana.ipynb - Formatted modified notebooks with nbfmt * feat(batch): add gemini-3.5-flash to model selector * style: format banana notebook to clean up PR diff * style: trim thought signature in Get_Started_Nano_Banana notebook To reduce file size and prevent git/diff hanging on massive cryptographically signed binary thoughts. * refactor: update Live API configuration, transcription support, and input handling for model version gemini-3.1-flash-live-preview (#1224) * Update Get_started_LiveAPI (#1222) * Update Get_started_LiveAPI * remove interactivity. * session resumption update. * run. * nbfmt * Resizing the NB notebook (#1245) * feat: add example for custom FPS in Grounding notebook (#1243) * feat: add example for custom FPS in Grounding notebook * Format custom frame rate explanation in Grounding.ipynb * Fix formatting of YouTube link and response generation * Fix formatting issues in Grounding.ipynb * Fix YouTube link formatting in Grounding.ipynb * docs: fix typographical and spelling errors in styleguide and quickstarts (#1253) * docs: fix typographical and spelling errors in styleguide.md * docs: fix typo in Image_out notebook header * fix: remove leftover google-generativeai imports from two notebooks (#1246) Two notebooks had leftover `import google.generativeai as genai` lines from a prior partial migration. Both notebooks otherwise use the new `google-genai` SDK correctly via `from google import genai`, and neither notebook's `%pip install` cell installs `google-generativeai`. On a clean Colab kernel the dead imports raise `ModuleNotFoundError`; on a kernel where the legacy package happens to be pre-installed, the dead imports silently shadow the working `genai` namespace. - quickstarts/Spatial_understanding.ipynb cell 19: delete the dead `import google.generativeai as genai` line; the rest of the cell (PIL.Image, io, os, requests, BytesIO) is genuinely used downstream - examples/Animated_Story_Video_Generation_gemini.ipynb cell 7: delete the cell entirely. The cell holds only the dead import No behavioural change beyond removing dead code that would otherwise fail or shadow at notebook runtime. * fix(examples): replace deprecated Live API session.send in Animated Story notebook (#1249) Use session.send_client_content(turns=..., turn_complete=True) instead of the deprecated session.send(input=..., end_of_turn=True), which emits a DeprecationWarning in google-genai (removal not before Q3 2025). Co-authored-by: Sri Ujwal <sbeeredd04@users.noreply.github.com> * fix(quickstarts): correct case-sensitive links to Search_Grounding/Code_Execution (#1248) Three cross-notebook links used lowercase filenames (Search_grounding.ipynb, Code_execution.ipynb) that 404 on case-sensitive file systems like GitHub. Point them at the actual files: Search_Grounding.ipynb and Code_Execution.ipynb. Co-authored-by: Sri Ujwal <sbeeredd04@users.noreply.github.com> * fix(quickstarts): correct broken Contributing guide link in Template.ipynb (#1247) The template linked the contributing guide as `Contributing.mg`, which is both a typo (`.mg`) and a path that does not exist next to the notebook. Point it at the repo's CONTRIBUTING.md using the same absolute GitHub URL the notebook already uses elsewhere. Co-authored-by: Sri Ujwal <sbeeredd04@users.noreply.github.com> * fix: align error handling notebook with google-genai retries (#1229) * docs: fix broken/incomplete links and remove duplicate entry in quickstarts README (#1257) * docs: fix broken/incomplete links and remove duplicate entry in quickstarts README * Update quickstarts/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * feat: add Live Translation quickstart tutorial and examples (#1259) * feat: add Live Translation quickstart tutorial and examples * chore: clean up notebook metadata and reorder cell output attributes * refactor: Live Translation notebook and script from cookbook (#1261) * Prohibit yt-dlp usage for YouTube content (#1264) Added a restriction against using yt-dlp for downloading or streaming YouTube content due to terms of service violations. * Sketch2Paint using Gemini (#790) --------- Co-authored-by: Guillaume Vernade <Giom-V@users.noreply.github.com> * Add service account auth section with IAM roles for Gemini, File, and… (#1258) * Add service account auth section with IAM roles for Gemini, File, and Vertex AI APIs * Remove service account section, add link to GCP auth docs instead * Add Vertex AI authentication documentation link * Fix embedding types & casing (#1032) * Fix embedding types & casing (fixes #1030) * Fix query task_type and indentation style violations * fix indentation style * fixed the formatting and linting issue * update model ID * fix formatting * Update styleguide with API key and crypto restrictions Added guidelines for API key usage and restrictions on crypto topics. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kkorpal <kkorpal@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Shipra <ships3040@gmail.com> Co-authored-by: Shipra <Shi-pra-19@users.noreply.github.com> Co-authored-by: Rohan Dwivedi <165134541+ROHANDWIVEDI2005@users.noreply.github.com> Co-authored-by: AgentModuleAdmin <0agentmodule@gmail.com> Co-authored-by: Patrick Loeber <50772274+patrickloeber@users.noreply.github.com> Co-authored-by: Irfaan Mansoori <irfaanmansoori100@gmail.com> Co-authored-by: Aryan Motgi <85900811+aryanmotgi@users.noreply.github.com> Co-authored-by: Thor 雷神 Schaeff <5748289+thorwebdev@users.noreply.github.com> Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: August Cayzer <august@cayzer.me> Co-authored-by: Sri Ujwal <113922053+sbeeredd04@users.noreply.github.com> Co-authored-by: Sri Ujwal <sbeeredd04@users.noreply.github.com> Co-authored-by: qianzhu / Lucien <2729478828@qq.com> Co-authored-by: Naveen Kumar G <naveen13524g@gmail.com> Co-authored-by: Nitin Tiwari <37960279+NSTiwari@users.noreply.github.com> Co-authored-by: Anika Islam <172253577+Surjasa@users.noreply.github.com>
…nput handling for model version gemini-3.1-flash-live-preview
fixes #1108