New Providers, Bug Fixes & Improvements

1.5.1 Release Notes

⚠️ Please read the breaking changes before updating!

This update adds native support for OpenRouter and Azure, fixes an issue with Google Gemini, adds support for GPT-5 models and adds Dutch and Polish translations.

Contributors

A huge thank you to our contributors @TheRealFalseReality, @NikNikovsky, @Minionguyjpro, @meceonzo—and of course everyone who helped test and provided feedback for the beta versions!

⚠️ Breaking Changes

• Timeline snapshots have moved from /www to /media. add the following to your configuration.yaml:

  # Add llmvision /media folder
  homeassistant:
    media_dirs:
      media: /media
      llmvision: /config/media/llmvision

  💡 Make sure to create the /media and the /llmvision inside it before restarting!
  💡 Update LLM Vision Card to the latest version! Otherwise snapshots won't show correctly.

• max_tokens is now displayed as a box type number selector rather than a slider. This change accommodates newer models that use thinking parameters counting toward max_tokens. Unlike the limited slider, the box allows for inputting much larger numbers.

  💡If you notice empty responses, consider increasing max_tokens.

Integration

✨ Features

• New Provider: LLM Vision now supports OpenRouter natively. (@valentinfrlch)
• New Provider: LLM Vision now supports Azure natively (#64, #103, #144). (@valentinfrlch)
• Support for Polish language: Added 🇵🇱 Polish translations. Thank you @NikNikovsky!
• Support for Dutch language: Added 🇳🇱 Dutch translations. Thank you @Minionguyjpro!

🔧 Improvements & Fixes

• Google Gemini fix: Fixed an infinite retry loop in the Google Gemini provider and corrected an issue where the fallback provider wasn't being used properly. (#398, #262), (@valentinfrlch)
• Support for GPT-5 models: Remove temperature and top_p from request when a gpt-5 model is used (#437), (@valentinfrlch)
• Moved Timeline snapshots: Snapshots are now stored securely behind Home Assistant authentication in /media. LLM Vision will attempt to migrate previously captured snapshots to the new folder.
• Remove max_tokens limit: max_tokens is now displayed as a box type number selector rather than a slider. This change allows for larger token limits, as thinking tokens count toward the max_tokens limit. (@valentinfrlch)
• Gemini key logging: Fixed a security issue that logged the Google API key in clear text (#334), (@valentinfrlch)
• Missing import: Add aiofile package to imports (#404), (@valentinfrlch)
• Sanitize IP address: Ignore any protocol input for the IP address field for Ollama and Open WebUI providers. Thank you @meceonzo!
• Fallback provider: Fixed a retry logic bug that caused an exception when no fallback provider was configured. (#461)
• Title generation: Fixed an issue that caused an exception when generate_title was not set. (#463)

Blueprint

Update the blueprint by re-importing it from the Blueprint settings page in Home Assistant.

https://github.com/valentinfrlch/ha-llmvision/blob/main/blueprints/event_summary.yaml

A huge thank you to @TheRealFalseReality for maintaining the blueprint!

✨ Features

• See snapshot quick action: Notification now shows button to preview the snapshot in a browser.

🔧 Improvements & Fixes

• Multiple Cameras: Fixed an issue that displayed the wrong camera (#403)
• Live Preview notification (iOS only): Preview mode can now be customized before and after analysis.
• Time format: Remove leading zero from 12-hour format.
• New media folder: Changed snapshot location from /www to /media/llmvision/snapshots (#457)

Improved Setup and Provider Settings

v1.5.0 Release Notes

This update improves the setup process, adds a fallback provider option, support for urls and other path types in video analyzer, introduces advanced settings for Ollama, and fixes an issue with Google Gemini.

Contributors

A huge thank you to our contributors @t3hk0d3, @TheRealFalseReality, @ond000, @natedev, @sb-develop—and of course everyone who helped test and provided feedback for the beta versions!

Integration

⚠️ Breaking Changes

• Timeline and Memory have been merged: The new Settings Config Entry unifies Timeline, Prompt and Memory (Beta) Settings as well as a new Fallback Provider option. If the Timeline provider still exists after this update, you may safely delete it. You may need to change the entity id of timeline entity (now associated with the Settings config entry).
• Temperature has moved: The temperature settings has moved to the provider settings. It can now longer be set individually for each call and is instead set for each provider. You can create multiple instances of the same provider if you need different temperature settings.

✨ Features

• New setup and reconfigure process: It is now easier to configure new providers as well as change settings for existing providers. The setup flow also allows for more provider specific settings. (@valentinfrlch)
• New Fallback provider option: You can now choose a fallback provider if the call fails with the selected provider. The request will be sent to the default model of the selected fallback provider. (@valentinfrlch)
• Support for additional paths: video_analyzer now supports URLs, media-source:// and relative /api/frigate paths. Thank you @t3hk0d3!
• New advanced settings for Ollama: You can now change the Context window length and “Keep alive” settings for Ollama. (#212)(@valentinfrlch)
• Added Top P: You can now control Top P (nucleus sampling) for all providers. This allows for more precise control over the randomness of generated responses. (@valentinfrlch)
• Support for Czech language: Added 🇨🇿 Czech translations. Thank you @ond000!

🔧 Improvements & Fixes

• Improved Image Pipeline efficiency: The image pipeline can now process multiple files or streams at once. Thank you @t3hk0d3!
• Google Gemini fix: Implemented exponential backoff retry logic for Google. Thank you @natedev! (#262)
• Update OpenAI Provider: Rename max_tokens to max_output_tokens for compatibility with newer o-series models. (#383)
• Custom OpenAI Provider: Use specified model for validation (#366)

Blueprint

• Fixed File Path default selector
• Fixed Custom Notification Channel default selector
• Added Time to notification title with the option of 12/24 hour format
• Fixed group/channel components with multiple camera entities

Thank you @TheRealFalseReality!

Improved Blueprint & Bug fixes

1.4.3 Release Notes

Warning

You may need to reconfigure certain fields of the blueprint after updating to v1.4.3.

Tip

The blueprint requires a separate update. You can update it by re-importing
(Settings → Automations & Scenes → Blueprints → three-dot menu → Re-import blueprint)

Blueprint

@TheRealFalseReality has completely rewritten the blueprint and enhanced it with lots of additional features:

• Added collapsible sections to the blueprint to make it easier to set up. Thank you @TheRealFalseReality!
• Added notification settings. Thank you @TheRealFalseReality!
  • notify_condition A list of conditions that need to be met to send the notification. The blueprint can still run and save events to the timeline but skips sending the notification. (Different from run_conditions which prevent the blueprint from running all-together).
  • delay_notification A separate cooldown for notifications (blueprint can still run, but doesn’t send a notification again. Separate from cooldown).
  • notification_sticky (Android only) make the notification stay active until dismissed.
  • notification_color (Android only) Customize the color of the notification
  • notification_icon (Android only) Customize the icon of the notification
  • notification_channel (Android only) Custom notification channel for custom notification sounds and vibration patterns. Allows overriding “Do Not Disturb”.
  • tts_notifications (Android only) Use text-to-speech to speak the notification.
• Added additional_actions (Experimental) Run custom actions after the blueprint.

Integration

• Added 🇸🇰 Slovak translation. Thank you @jose1711!
• Updated Groq default model (previous model is deprecated) (#311)
• Fixed a bug in the timeline that deleted events when retention_time was set to 0. (#309)

Performance Improvements

1.4.2 Release Notes

Tip

The blueprint requires a separate update. You can update it by reimporting.

Breaking Changes

Warning

The cleanup action is deprecated. The database now runs an automatic cleanup every time Home Assistant starts.

Fixes & Changes

• Significantly improved image pipeline performance. Thank you @sb-develop!
• Improved frame selection by @sb-develop and @lich2000117
  • The first captured frame is now always considered the key frame for stream_analyzer (blueprint)
• Images are now stored in the timeline in full resolution. (@sb-develop)
• Adds 🇨🇳 Chinese translations. Thank you @lich2000117!
• Adds 🇸🇪 Swedish translations. Thank you @adnansarajlic!
• Fixed a bug that prevented events from being properly deleted after retention_time. (@valentinfrlch)
• An automatic cleanup will run when Home Assistant starts to remove any unused images from /www/llmvision/. (@valentinfrlch)
• Fixed a bug that prevented images to be stored correctly on docker installations. (@valentinfrlch)

Bug fixes & Improvements

1.4.1 Release Notes

Tip

The blueprint requires a separate update. You can update it by reimporting.

Integration

• Memory now supports images with transparency (such as png). Thank you @lich2000117!
• Data Analyzer now supports the remember parameter and can contribute events to the timeline.
• Titles now support characters from the extended ASCII charset such as “Ø”, “Ӑ” and many others. Thank you, @ttopholm!
• Improved database performance and stability
• Fixed a bug where images would not appear in the timeline when using image_input or a frigate event_id. (#266, #255)

Blueprint

• Updated notification message. Thanks @lich2000117!

Memory & Timeline

1.4.0 Release Notes

This is a major update. Please read through the release notes and breaking changes especially. See the updated docs if you have any questions.

A huge thank you to everyone who helped test this version and shared their feedback!

Warning

The blueprint has been significantly restructured, which means you will have to set up some parts again. Due to many issues with the Frigate mode of this blueprint it has been removed. However, the blueprint is still compatible with Frigate: You will have to use the camera entities exposed by the Frigate integration as well as one of the binary_sensors for each camera.

Tip

The blueprint requires a separate update. You can update it by reimporting.

⚠️ Breaking Changes

• input_mode (Blueprint) has been deprecated. The blueprint now only supports what was previously the camera mode. See above for how you can still use Frigate with this blueprint.
• detail has been deprecated. To control the quality of images use the target_width parameter which downscales images that are larger than the set width.
• expose_images_persist has been deprecated. expose_images now saves the image to /www/llmvision/<uid>-<key_frame>.jpg. This path is returned with the response variable in the key_frame key. Use the new cleanup action to remove unneeded images from time to time.

Features

Integration

• Timeline replaces Event Calendar: LLM Vision now has a frontend card for your dashboard to display events along with an image and a summary of the event. Get the card here: https://github.com/valentinfrlch/llmvision-card
• Added Memory: You can now store reference images along with a description in memory. This provides additional context to the model and can be used e.g. to provide the model with pictures of members of your household to identify them. Content in memory syncs across all providers (except Groq as it can only process one image per call). With this there is now also a use_memory toggle in all actions to control whether content stored in memory should be used or not.
  Disclaimer: This is not a facial recognition model. AI can make mistakes. Do not use for unlocking doors or similarly critical actions!
• Added customizable system prompt: In the memory settings, a system prompt has been added which can be customized as well. The default system prompt greatly improves accuracy compared to before. The system prompt applies to all providers and is sent even when use_memory is off.
• Added customizable title prompt: The prompt used to generate titles for events can no be customized in the Memory settings.
• Added a new action remember: Use this action to store custom events to the timeline (and the LLM Vision Card).
• Added a new action cleanup: Remove images in /www/llmvision/ that are not associated with an event in the timeline.

Blueprint

Warning

See the breaking changes above before updating!

• Added a run_condition: You can now add custom conditions to the blueprint. All conditions have to be true in order for the blueprint to run. Use this to only run the automation when no one is home or when the alarm is enabled.
• Added notification_delivery: Choose between ‘Dynamic’ and ‘Consolidated’ delivery modes. Dynamic will immediately send a notification with a live camera preview and update the notification silently when the summary is available. Consolidated will only send the notification once the summary is available. Use this if you got two separate notifications before.
• Added use_memory: Enables access to information stored in memory. Memory needs to be set up for this.

Open WebUI

1.3.9 Release Notes

Features

Integration

• Added Open WebUI as a supported provider (#112), @valentinfrlch
  Open WebUI is a self-hosted AI platform with a built-in inference engine for databases so you can add knowledge to your LLM which is also available in LLM Vision.
  https://github.com/open-webui/open-webui

Bug fixes

1.3.8 Release Notes

⚠️ Breaking Changes

Warning

Blueprint: mode has been renamed to input_mode. Just select your input_mode again and save your automation.

Tip

The blueprint requires a separate update. You can update it by reimporting.

Bug Fixes

Integration

• Removed redundant imports

Blueprint

• Fixed a bug where the blueprint wouldn’t trigger for Frigate mode (#185) by @valentinfrlch
• Description improvements

AWS Bedrock, reconfigure providers and bug fixes

1.3.7 Release Notes

⚠️ Breaking Changes

Warning

If you were using any custom OpenAI-compatible providers, you'll need to set them up again.

Tip

The blueprint needs to be updated separately. You can update it by reimporting.
Go to Settings > Automations & scenes > Blueprints > (...) > Re-import blueprint

Features

Integration

• Added AWS Bedrock support by @Smiley73
• Enable provider reconfiguration by @Smiley73

Blueprint

• Add filter for Frigate zones and option to ignore stationary objects by @amcfague

Bug Fixes

Integration

• Fixed an issue preventing proper setup of some custom OpenAI-compatible providers (#173) by @bill9109 and @Smiley73
• Resolved a setup failure that occurred when "Event Calendar" was selected as the first provider (#176) by @valentinfrlch
• Fixed duplicate API calls in the Ollama provider by @Smiley73
• Fixed a bug where stream_analyzerand video_analyzer could fail if stream resolution changed during recording (#158, #142) by @valentinfrlch
• Increased max_tokens for event titles to prevent truncation (#182)
• Translation improvements

Blueprint

• Fixed a bug that caused the blueprint to fail when MQTT was not configured (#155) by @valentinfrlch
• Fixed a bug that where the blueprint would fail when motion sensors contained "camera" in their name (#101) by @valentinfrlch

genAI Titles & Bug fixes

Changelog v1.3.6

Bug fixes

• Improved data analyzer results parsing (#151, #153) by @valentinfrlch

Changelog v1.3.5

Blueprint

Tip

The blueprints needs to be re-imported to update!

• Fixed issues for devices with special characters (#121) by @rappazzo
• Added filter for Frigate labels by @rappazzo, @SleepyMoz
• Fixed a bug that would prevent from running the automation again even though cooldown delay had passed.

LLM Vision Integration

Features

• Added option to generate a title by @valentinfrlch: Used to create better titles for remembered events.
• Added option to configure retries and time between retries by @rsteckler
• Added option to persistently expose analyzed images by @rsteckler. Useful for creating your own automations. Images will be exposed in /www/llmvision/.

Bug fixes

• Fixed a bug where data analyzer would remove an entity's icon (#90) by @valentinfrlch
• Fixed frame extraction from videos (#80) by @rsteckler
• Fixed a bug that would prevent Google and Groq from being setup successfully because of validation errors by @valentinfrlch
• Calendar events.json moved from /custom_components/llmvision/events.json to /llmvision/events.json so it isn't deleted with every update (#95) by @valentinfrlch