Skip to content

LLM-enhanced blueprint fails with from_json error when the model wraps output in a ```json code fence #136

Description

@yosef-chai

Summary

The LLM-enhanced local assist blueprint parses the model response with a bare
from_json:

llm_result: '{{ result.response.speech.plain.speech | from_json }}'

The prompt instructs the model to reply with raw JSON ("So also no code tags.
Only the JSON!"), but several models — most notably Google Generative AI
(Gemini)
— frequently ignore this and wrap the payload in a Markdown code
fence:

```json
{"action_data": {"media_id": "...", "media_type": "track"}, "media_description": "..."}
```

from_json then fails on the leading ```json and the automation aborts:

ValueError: Template error: from_json got invalid input '```json
...
'

The automation never reaches the play_media step, so nothing plays. This is
intermittent because whether the model adds the fence varies run-to-run for the
same prompt.

Environment

Component Version
Blueprint llm-enhanced-local-assist-blueprint (all language variants: _en, _fr, _hu, _nl, community translations)
Home Assistant Core 2026.6.0
Conversation agent Google Generative AI (Gemini)

Steps to reproduce

  1. Use the blueprint with a Google Generative AI conversation agent.
  2. Issue a normal music request (e.g. "play ").
  3. Intermittently the agent returns the JSON inside a ```json fence and
    the automation errors at the from_json step; the request silently does
    nothing.

Root cause

The blueprint trusts the prompt's "no code tags" instruction and does no
defensive sanitization before from_json. Prompt instructions are not a
guarantee, and Gemini in particular tends to Markdown-format JSON.

Proposed fix

Strip an optional leading/trailing Markdown code fence before parsing. This is
backward compatible (raw JSON is unaffected) and tolerant of the most common
deviation:

llm_result: >-
  {% set r = result.response.speech.plain.speech | string %}
  {% if r.startswith("```") %}
    {% set r = r.split("\n", 1)[1] %}
    {% if r.rstrip().endswith("```") %}{% set r = r.rstrip()[:-3] %}{% endif %}
  {% endif %}
  {{ r | from_json }}

(A regex-based variant — e.g. r | regex_replace('^```(json)?\\s*|\\s*```$', '')
— works too; the explicit form above avoids edge cases with fences appearing
mid-string.)

Verified locally: with this change, both raw-JSON and ```json-wrapped
responses from Gemini parse correctly and playback proceeds.

Impact

  • Intermittent, hard-to-diagnose failures for users on Google Generative AI
    (one of the most common HA conversation agents).
  • The failure is silent from the user's perspective (assistant says nothing /
    generic response, no music), which makes it look like the voice pipeline is
    broken rather than a parsing issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions