Releases: valentinfrlch/ha-llmvision
Release list
Bug Fixes and Performance Improvements
v1.7.1 Beta 1 Release Notes
This update fixes some bugs and improves performance. There are no breaking changes.
Contributors
A huge thank you to our contributors
Integration
🔧 Fixes
- Thinking mode for Anthropic provider: Fixed an issue with the reasoning budget not converted properly by @valentinfrlch in #661 (#660)
- Ollama
keep_alive: Fixed an issue withkeep_alivevalues not converted correctly by @valentinfrlch in #664 (#648) - Timeline cleanup: Increase time to cleanup orphaned snapshots for models that take longer to answer (e.g. reasoning or local models) by @lyonsad in #670 (#676)
- Timeline update: debounce timeline calendar refresh if multiple events are added simultaneously by @jon1012 in #674 (#673)
Full Changelog: v1.7.0...v1.7.1-beta.1
Support for Glimpse-v1, New Action & API Improvements
v1.7.0 Release Notes
We’ve released our own AI model, Glimpse-v1! While general multi-modal LLMs can describe images well, they often miss the contextual intent behind a scene. Sending every motion event to a cloud LLM is slow, costly, and exposes private footage to third parties.
We solved this by post-training Gemma 3 with reinforcement learning on thousands of labeled samples. Through this process, our model has learned that, for example “a person wearing a blue uniform carrying a package, with a gray van parked near the driveway”, is actually an “Amazon delivery”. Glimpse also understands the difference between a false positive (triggered by motion), and an actual event.
Additionally, the model was trained to generate event title and description in a single call. This significantly reduces inference time, so notifications can be sent more quickly.
We believe privacy is fundamental to smart homes. As such, we encourage everyone to run the AI models needed for LLM Vision on a local machine. By building specialized, compact models that can run locally on hardware with limited memory and compute resources, we aim to make local AI accessible to everyone.
While we’re excited what’s possible in the future, this is the first release of our model. As such, there will be limitations. For example, the model can currently only generate English responses.
We need your help! Models like these need a lot of training data to understand different situations well. You can help make Glimpse better for everyone by submitting feedback through the LLM Vision Card.
Check out Glimpse-v1 on our website: https://llmvision.org/glimpse/
Feedback is entirely optional and never happens without your explicit permission. We do not collect personal data. See our privacy policy.
Contributors
A huge thank you to our contributors
Integration
✨ Features
- Support for Glimpse-v1: This release adds optimizations for our own model which generates title and description in a single call. (by @valentinfrlch)
- New action: In 1.6.0 we introduced the new timeline API. Now, in addition to the API, you can fetch events from your timeline based on filters via a new Home Assistant action: get_events. (by @valentinfrlch)
- Timeline API: Now supports optional
startandendparameters to query the timeline with a precise range. The limit has also been increased to 10’000. (by @wysie) - Configurable reasoning effort: Some providers now support configurable thinking/reasoning effort. (by @valentinfrlch) (#609)
🔧 Fixes
- Bring back calendar attributes: The calendar extra attributes have been deprecated in favor of the new timeline API in 1.6.0. For ease of use we're bringing back some calendar extra attributes (for the most recent event). To fetch events based on filters, see the new get_events action. (by @valentinfrlch)
- Timeline refresh: The timeline now refreshes correctly. (by @CamSoper), (#600, #613)
- Timeline migration: Migration could cause some snapshots not be cleaned up correctly for some users. (by @valentinfrlch) (#640)
- Event classification: improved label and category classification for events. (by @valentinfrlch)
- Default Google model: The default model for the Google provider has been updated to
gemini-3.1-flash-lite(by @valentinfrlch) (#622) - Default Anthropic model: The default model for the Google provider has been updated to
claude-haiku-4-5(by @valentinfrlch) (#475) - Custom OpenAI: Now only requires base url (without
/chat/completionsendpoint) (by @haocn-ops) (#649) - Test coverage: We've drastically increased test coverage from <10% to >80% in an effort to make llm vision more stable and dependable. (by @valentinfrlch)
🌐 Languages
Support for Glimpse-v1, New Action & API Improvements
v1.7.0 RC 1 Release Notes
We’ve released our own AI model, Glimpse-v1! While general multi-modal LLMs can describe images well, they often miss the contextual intent behind a scene. Sending every motion event to a cloud LLM is slow, costly, and exposes private footage to third parties.
We solved this by post-training Gemma 3 with reinforcement learning on thousands of reward-labeled samples. Through this process, our model has learned that “a person wearing a blue uniform carrying a package, with a gray van parked near the driveway”, is actually an “Amazon delivery”. Glimpse also understands the difference between a false positive (triggered by motion), and an actual event.
Additionally, the model was trained to generate event title and description in a single call. This significantly reduces inference time, so notifications can be sent more quickly.
We believe privacy is fundamental to smart homes. As such, we encourage everyone to run the AI models needed for LLM Vision on a local machine. By building specialized, compact models that can run locally on hardware with limited memory and compute resources, we aim to make local AI accessible to everyone.
While we’re excited what’s possible in the future, this is the first release of our model. As such, there will be limitations. For example, the model can currently only generate English responses.
We need your help! Models like these need a lot of training data to understand different situations well. You can help make Glimpse better for everyone by submitting feedback through the LLM Vision Card.
Check out Glimpse-v1 on our website: https://llmvision.org/glimpse/
Feedback is entirely optional and never happens without your explicit permission. We do not collect personal data. See our privacy policy.
Contributors
A huge thank you to our contributors
Integration
✨ Features
- Support for Glimpse-v1: This release adds optimizations for our own model which generates title and description in a single call. (by @valentinfrlch)
- New action: In 1.6.0 we introduced the new timeline API. Now, in addition to the API, you can fetch events from your timeline based on filters via a new Home Assistant action: get_events. (by @valentinfrlch)
- Timeline API: Now supports optional
startandendparameters to query the timeline with a precise range. The limit has also been increased to 10’000. (by @wysie) - Configurable reasoning effort: Some providers now support configurable thinking/reasoning effort. (by @valentinfrlch) (#609)
🔧 Fixes
- Bring back calendar attributes: The calendar extra attributes have been deprecated in favor of the new timeline API in 1.6.0. For ease of use we're bringing back some calendar extra attributes (for the most recent event). To fetch events based on filters, see the new get_events action. (by @valentinfrlch)
- Timeline refresh: The timeline now refreshes correctly. (by @CamSoper), (#600, #613)
- Timeline migration: Migration could cause some snapshots not be cleaned up correctly for some users. (by @valentinfrlch) (#640)
- Event classification: improved label and category classification for events. (by @valentinfrlch)
- Default Google model: The default model for the Google provider has been updated to
gemini-3.1-flash-lite-preview(by @valentinfrlch) (#622) - Default Anthropic model: The default model for the Google provider has been updated to
claude-haiku-4-5(by @valentinfrlch) (#475) - Custom OpenAI: Now only requires base url (without
/chat/completionsendpoint) (by @haocn-ops) (#649) - Test coverage: We've drastically increased test coverage from <10% to >80% in an effort to make llm vision more stable and dependable. (by @valentinfrlch)
🌐 Languages
Structured Output & New Timeline API
v1.6.0 Release Notes
This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output, a new Timeline API, and fixes various bugs.
⚠️ Breaking Changes
To avoid confusion with LLM Vision Memory:
- The
rememberaction is now calledcreate_event - The
rememberparameter has been renamed tostore_in_timeline.
Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.
If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.
Contributors
A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for more details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
- Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the
create_eventaction. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch) - Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
- Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551, #553) (by @valentinfrlch)
- Special Characters: Improved handling of special characters in event titles (by @radzio)
- Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
- Unit Test Coverage: Added unit tests. (by @Smiley73)
- Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
- Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
- Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
- Pinned
boto3to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
- Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)
Blueprint
If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.
🔧 Improvements & Bug Fixes
- Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
- Fixes the “camera is undefined” error. (by @TheRealFalseReality)
Structured Output & New Timeline API
v1.6.0 Release Candidate 1 Release Notes
This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.
⚠️ Breaking Changes
To avoid confusion with LLM Vision Memory:
- The
rememberaction is now calledcreate_event - The
rememberparameter has been renamed tostore_in_timeline.
Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.
If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.
Contributors
A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for more details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
- Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the
create_eventaction. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch) - Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
- Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551, #553) (by @valentinfrlch)
- Special Characters: Improved handling of special characters in event titles (by @radzio)
- Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
- Unit Test Coverage: Added unit tests. (by @Smiley73)
- Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
- Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
- Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
- Pinned
boto3to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
- Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)
Blueprint
If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.
🔧 Improvements & Bug Fixes
- Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
- Fixes the “camera is undefined” error. (by @TheRealFalseReality)
Structured Output & New Timeline API
v1.6.0 Beta 1 Release Notes
This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.
⚠️ Breaking Changes
To avoid confusion with LLM Vision Memory:
- The
rememberaction is now calledcreate_event - The
rememberparameter has been renamed tostore_in_timeline.
Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.
If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.
Contributors
A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, @TheRealFalseReality, and @dcgrove!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for more details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
- Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the
create_eventaction. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch) - Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
- Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
- Special Characters: Improved handling of special characters in event titles (by @radzio)
- Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
- Unit Test Coverage: Added unit tests. (by @Smiley73)
- Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
- Fixed an issue that caused validation to fail for gpt-5 models when using Azure (#494) (by @valentinfrlch)
- Fixed an issue that caused newer claude models to fail silently (#562) (by @dcgrove)
- Pinned
boto3to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
- Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)
Blueprint
If you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.
🔧 Improvements & Bug Fixes
- Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
- Fixes the “camera is undefined” error. (by @TheRealFalseReality)
Structured Output & New Timeline API
v1.6.0 Alpha 3 Release Notes
This major release contains breaking changes, please read the breaking changes section before upgrading. It adds support for structured output and a new Timeline API.
⚠️ Breaking Changes
To avoid confusion with LLM Vision Memory:
- The
rememberaction is now calledcreate_event - The
rememberparameter has been renamed tostore_in_timeline.
Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version.
If you use LLM Vision Cards, please update to v1.6.0 as well. The card now uses the new Timeline API, and older versions are no longer supported.
Contributors
A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, @boian-ivanov, @ljmerza, @radzio, and @TheRealFalseReality!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for more details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. The LLM Vision Cards use this API, but you can also use it for your own automations or to integrate third-party services. See the API docs for more details. (by @valentinfrlch)
- Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the
create_eventaction. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch) - Request Timeout: The request timeout is now configurable (useful for local LLMs that may need more time to respond). (by @ljmerza)
- Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
- Special Characters: Improved handling of special characters in event titles (by @radzio)
- Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
- Unit Test Coverage: Added unit tests. (by @Smiley73)
- Fixed a bad request issue during inital setup. (#574) (by @valentinfrlch)
- Pinned
boto3to 1.37.1 to avoid conflicts with other integrations (#580) (by @valentinfrlch)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
- Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)
Blueprint
For this alpha, please import the blueprint using this url:
https://github.com/valentinfrlch/ha-llmvision/blob/v1.6.0-beta/blueprints/event_summary_beta.yamlIf you're having this issue on an existing blueprint, set this new option explicitly so it changes from the unknown {{ camera }} variable to a static "LLM Vision Snapshot" group.
🔧 Improvements & Bug Fixes
- Importance Classification: The system prompt to classify important events is now configurable. (by @TheRealFalseReality)
- Fixes the “camera is undefined” error. (by @TheRealFalseReality)
Structured Output & New Timeline API
v1.6.0 Alpha 2 Release Notes
This release adds support for structured output and a new Timeline API.
⚠️ Breaking Changes
To avoid confusion with LLM Vision Memory:
- The
rememberaction is now calledcreate_event - The
rememberparameter has been renamed tostore_in_timeline.
Please make sure to update these in your custom automations. If you use the blueprint, make sure to update to the latest version. For this alpha version please use the following url to import the v1.6.0-alpha blueprint:
https://github.com/valentinfrlch/ha-llmvision/blob/v1.6.0-beta/blueprints/event_summary_beta.yamlIf you use LLM Vision Cards, please update to v1.6.0 as well. The cards now use the new Timeline API, and older versions are no longer supported.
Contributors
A huge thank you to our contributors @rmaher001, @Smiley73, @muminkoykiran, and @boian-ivanov!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for more details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. This is used by the LLM Vision Cards, but you may use the API for your automations as well. See the API docs for more details. (by @valentinfrlch)
- Event classification: Events are now classified into categories and labels. For custom events, these can be assigned manually using the
create_eventaction. For events that are automatically added through an analyzer or the blueprint, category and label is assigned automatically. For a full list of all available categories and labels, see the docs. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category and (new) label for an event manually. (#536) (by @valentinfrlch) - Timeline: Fixed an issue that caused images to be removed before the event was created, resulting in broken notifications and timeline cards. (#551) (by @valentinfrlch)
- Unavailable Cameras: Added checks to detect if a camera is offline. Only raises an error if all cameras are unavailable. (by @rmaher001)
- Unit Test Coverage: Added unit tests. (by @Smiley73)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
- Support for Turkish language: Added 🇹🇷 Turkish translations. (by @muminkoykiran)
Structured Output & New Timeline API
v1.6 Alpha 1 Release Notes
This update adds support for structured output and a new Timeline API.
⚠️ Breaking Changes
- The
rememberaction is now calledcreate_eventto avoid confusion with LLM Vision Memory.
Contributors
A huge thank you to our contributors @rmaher001, and @boian-ivanov!
Integration
✨ Features
- Structured Output: Valid JSON output can now be forced using the
structureparameter. See the docs for details. (by @rmaher001) - Timeline API: Events stored in the timeline can now be fetched through the new Timeline API. This is used by the LLM Vision Cards, but you may use the API for your automations as well. See the docs for details. (by @valentinfrlch)
🔧 Improvements & Bug Fixes
- Create Events: The
create_eventsaction now supports setting the category for an event manually. (#536) (by @valentinfrlch)
🌐 Languages
- Support for Hungarian language: Added 🇧🇬 Bulgarian translations. (by @boian-ivanov)
Bug Fixes & Improvements
v1.5.2 Release Notes
This update includes bug fixes and performance improvements.
Contributors
A huge thank you to our contributors @TheRealFalseReality, @bekdan1988, @alemuro, and everyone who helped test the pre-release versions!
⚠️ Breaking Changes
In our previous release (1.5.1), we moved snapshots to /config/media/llmvision/snapshots. This change caused issues for some users who were using other integrations that depend on the /media directory. As a result, the snapshots directory is moving again to /media/llmvision/snapshots.
What this means for you:
- Existing snapshots will automatically migrate to the new location when you restart after upgrading to 1.5.2.
- Remove the media folder configuration that you may have added to your
configuration.yamlhomeassistant: media_dirs: llmvision: /config/media/llmvision # Remove this line media: /media # Remove this line
- If you have any custom automations that use snapshots, update your template to:
{{ response.key_frame.replace('/media','/media/local') }} - If you use the Timeline Card, make sure to update it as well!
I apologize for any inconvenience this causes!
Integration
🌐 Languages
- Support for Hungarian language: Added 🇭🇺 Hungarian translations. (by @bekdan1988)
- Support for Catalan language: Added Catalan translations. (by @alemuro)
🔧 Improvements & Fixes
- Improved keyframe selection: Keyframe should now show the detected object more reliably. (by @valentinfrlch)
- Timeline events not deleted: Fixed a bug that caused events not being deleted properly. (by @valentinfrlch)
- Anthropic fix: Fixed an issue that caused validation to fail for the Anthropic provider. (#475) (by @valentinfrlch)
- Reduced image pipeline latency: Frames are now stored directly in memory instead of in a temporary directory. This reduces disk wear and reduces latency caused by I/O operations. It also fixes a race condition when running multiple actions in parallel. (#505, #510) (by @valentinfrlch)
- Moved Timeline snapshots: Snapshots are now stored securely behind Home Assistant authentication in
/media. LLM Vision will attempt to migrate your previously captured snapshots to the new folder. (by @valentinfrlch) - System Prompts: System prompts are now injected as actual system messages for every provider, which should improve their effectiveness. Also try out the updated default system and title prompts. (by @valentinfrlch)
max_tokensnow optional:max_tokensis now an optional parameter. Newer models use tokens for reasoning, which could result in empty responses ifmax_tokenswas set too low. (by @valentinfrlch)
Blueprint
Make sure to update the blueprint by re-importing it!
A huge thank you to @TheRealFalseReality for maintaining the blueprint!
🔧 Improvements & Fixes
- New media folder: Updated all
key_framepaths to match new media path. (by @TheRealFalseReality)