Skip to content

Bug: Gemini realtime path returns empty success (score=1.0, output=None) on a safety-blocked or refused response #508

Description

@mittalpk

Describe the overall issue and situation

In the Gemini provider's realtime (non-batch) inference path, a safety-blocked prompt, a candidate that stopped for a non-STOP reason (safety, recitation, etc.) with no content, or a response containing only non-text parts is returned as a successful, empty extraction instead of raising.

langextract/providers/gemini.py (_process_single_prompt):

response = self._client.models.generate_content(
    model=self.model_id, contents=prompt, config=call_config
)
return core_types.ScoredOutput(score=1.0, output=response.text)

google.genai's GenerateContentResponse.text (_get_text() in the SDK) returns None, not an exception, when:

  • not self.candidates — the prompt itself was blocked (prompt_feedback.block_reason set),
  • candidates[0].content or .content.parts is empty — the candidate stopped for a non-STOP finish_reason (SAFETY, RECITATION, PROHIBITED_CONTENT, etc.) with no output, or
  • the response contains only non-text parts (e.g. a function call).

ScoredOutput.output is typed str | None, so None is accepted and returned with score=1.0. Downstream, this is reported as a successful empty extraction — the caller gets no signal that anything was blocked, and the block/refusal reason (prompt_feedback.block_reason, candidates[0].finish_reason) is silently discarded.

This is the same class of bug reported for the OpenAI realtime path in #491 (fixed for OpenAI in #496). #491 explicitly noted: "The Gemini realtime path has the same class of gap on a safety-blocked response.text (version-dependent); worth folding the same guard in there too." PR #496 only touched openai.py, so this side was never addressed.

Expected behavior

When response.text is None, _process_single_prompt raises InferenceRuntimeError with the available diagnostic (block reason / finish reason), mirroring the fix already applied to the OpenAI provider.

Actual behavior

_process_single_prompt returns ScoredOutput(score=1.0, output=None). The caller receives a "successful" extraction with no content; the actual block/refusal reason is lost.

Steps to reproduce the issue

  1. Configure GeminiLanguageModel and issue a prompt that trips Gemini's safety filters, or otherwise produces a candidate with a non-STOP finish reason and no text content.
  2. The SDK's response.text evaluates to None (confirmed directly against google-genai 2.11.0's GenerateContentResponse._get_text).
  3. Observe _process_single_prompt returns ScoredOutput(score=1.0, output=None) instead of raising — the extraction is reported as a successful empty result.

Any additional content

  • Verified without live API: mock models.generate_content to return a response with text=None, prompt_feedback.block_reason or candidates[0].finish_reason set, and assert _process_single_prompt raises InferenceRuntimeError surfacing the reason (mirrors the existing OpenAI refusal test added in Handle OpenAI refusals and missing message content #496).
  • Environment: current main at commit b5fe0ba.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions