0.3.0

What's Changed

• Update codecov/codecov-action action to v5 by @renovate[bot] in #194
• Update peter-evans/create-pull-request action to v7 by @renovate[bot] in #195
• Added results for Qwen3-30B-A3B (GPTQ quantization) by @JohnTheNerd in #190
• Update dependency pytest_homeassistant_custom_component to v0.13.249 by @renovate[bot] in #196
• Reject new updates detected with Cruft by @github-actions[bot] in #198
• chore(deps): update dependency pytest_homeassistant_custom_component to v0.13.259 by @renovate[bot] in #203
• New updates detected with Cruft by @github-actions[bot] in #199
• Add qwen3-4b model and assist benchmark data by @allenporter in #204
• Update google model cards by @allenporter in #205
• Update gemini 2.5 flash model card with correct model nid by @allenporter in #206
• Publish assist reports for qwen3-4b, and latest gemini models by @allenporter in #207
• Update assist-mini and questions datasets by @allenporter in #208
• Update leaderboard by @allenporter in #209
• Update automations dataset reports by @allenporter in #210
• Add qwen3-8b eval reports by @allenporter in #211
• Add qwen3-1.7b results by @allenporter in #212
• Update actions/download-artifact action to v5 by @renovate[bot] in #216
• Update actions/checkout action to v5 by @renovate[bot] in #219
• Update pre-commit hook pre-commit/pre-commit-hooks to v6 by @renovate[bot] in #218
• Update dependency datasets to v4 by @renovate[bot] in #213
• Update actions/upload-pages-artifact action to v4 by @renovate[bot] in #220
• Update actions/setup-python action to v6 by @renovate[bot] in #222
• Update dependency openai to v2 by @renovate[bot] in #223
• Update dependency pdoc to v16 by @renovate[bot] in #226
• Update GitHub Artifact Actions (major) by @renovate[bot] in #225
• Update astral-sh/setup-uv action to v7 by @renovate[bot] in #224
• Update actions/checkout action to v6 by @renovate[bot] in #229
• Drop black and switch to ruff format by @allenporter in #231
• Update ruff lint workflow by @allenporter in #232
• Add AITask support by @allenporter in #230
• Update peter-evans/create-pull-request action to v8 by @renovate[bot] in #234
• Update dependencies and snapshots by @allenporter in #240
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.14.9 by @renovate[bot] in #237
• Remove the dataset dependency from the agent module by @allenporter in #239
• Update GitHub Artifact Actions (major) by @renovate[bot] in #238
• Update dependency openai to v2.12.0 by @renovate[bot] in #235
• Update dependency ruff to v0.14.10 by @renovate[bot] in #236
• Update dependency pytest_homeassistant_custom_component to v0.13.301 by @renovate[bot] in #221
• Test Qwen3-4b-instruct-2507:IQ4_XS by @cibernox in #243

New Contributors

• @github-actions[bot] made their first contribution in #198
• @cibernox made their first contribution in #243

Full Changelog: 0.2.0...0.3.0

0.2.0

What's Changed

• Manage project with scruft by @allenporter in #193

Full Changelog: 0.1.1...0.2.0

0.1.1

Full Changelog: 0.1.0...0.1.1

0.1.0

What's Changed

• Create a basic device action dataset by @allenporter in #3
• Example model evaluation for device actions by @allenporter in #2
• Update dom1-pl dataset by @allenporter in #4
• Evaluation improves for the device action eval by @allenporter in #5
• Restructure evaluations to be self contained by @allenporter in #6
• Update the device actions evaluation by @allenporter in #7
• Update assistant baseline and add gemini evaluations by @allenporter in #8
• Add device action data for gpt-4o by @allenporter in #9
• Update model output for v1 device actions eval by @allenporter in #10
• Update dependency ollama to v0.3.1 by @renovate in #16
• Update dependency mashumaro to v3.13.1 by @renovate in #17
• Update dependency google-generativeai to v0.7.2 by @renovate in #19
• Update dependency openai to v1.38.0 by @renovate in #21
• Update pre-commit hook psf/black to v24.8.0 by @renovate in #25
• Update pre-commit hook codespell-project/codespell to v2.3.0 by @renovate in #24
• Update dependency mypy to v1.11.1 by @renovate in #20
• Update dependency pdoc to v14.6.0 by @renovate in #22
• Update pre-commit hook charliermarsh/ruff-pre-commit to v0.5.6 by @renovate in #23
• Update dependency openai to v1.40.0 by @renovate in #26
• Update dependency pytest_homeassistant_custom_component to v0.13.154 by @renovate in #29
• Update dependency datasets to v2.21.0 by @renovate in #33
• Update dependency openai to v1.41.0 by @renovate in #30
• Update dependency setuptools to v73 by @renovate in #35
• Update pre-commit hook charliermarsh/ruff-pre-commit to v0.6.1 by @renovate in #34
• Update dependency setuptools to v74 by @renovate in #37
• Update dependency setuptools to v74.1.1 by @renovate in #38
• Update dependency pytest_homeassistant_custom_component to v0.13.159 by @renovate in #39
• Update dependency datasets to v3 by @renovate in #41
• Update dependency setuptools to v75 by @renovate in #42
• Add results for qwen2.5 by @tannisroot in #44
• Update dependency pytest_homeassistant_custom_component to v0.13.165 by @renovate in #45
• Update dependency openai to v1.48.0 by @renovate in #46
• Add scores for Llama 3.2 3B and 1B by @tannisroot in #47
• Update pre-commit hook pre-commit/pre-commit-hooks to v5 by @renovate in #50
• fix Llama 3.2 urls in models.yaml by @domdomegg in #48
• Update dependency pdoc to v15 by @renovate in #51
• Update dependency pytest_homeassistant_custom_component to v0.13.177 by @renovate in #53
• Update dependency mypy to v1.13.0 by @renovate in #52
• Migrate renovate config by @renovate in #58
• Update pre-commit hook charliermarsh/ruff-pre-commit to v0.7.4 by @renovate in #56
• Update dependency setuptools to v75.5.0 by @renovate in #55
• Update dependency openai to v1.54.4 by @renovate in #54
• Update dependency pytest_homeassistant_custom_component to v0.13.186 by @renovate in #60
• Update dependency ollama to v0.4.4 by @renovate in #63
• Support to load additional components by @Shulyaka in #65
• mentioned requirements_eval.txt in all READMEs by @JohnTheNerd in #66
• Update dependency openai to v1.59.8 by @renovate in #68
• Update dependency anthropic to v0.43.1 by @renovate in #67
• added evals for llama 3.3 (using Q4_K_M in ollama and AWQ in vLLM) by @JohnTheNerd in #69
• Update pre-commit hook psf/black to v25 by @renovate in #70
• Update dependency pytest_homeassistant_custom_component to v0.13.207 by @renovate in #71
• Update dependency datasets to v3.3.1 by @renovate in #72
• Update dependency openai to v1.63.2 by @renovate in #73
• Update dependency anthropic to v0.46.0 by @renovate in #75
• Update dependency synthetic_home to v4.5.0 by @renovate in #74
• Update dependency anthropic to v0.49.0 by @renovate in #78
• Update dependency openai to v1.65.2 by @renovate in #76
• Update dependency setuptools to v75.8.2 by @renovate in #79
• Update dependency pytest_homeassistant_custom_component to v0.13.218 - autoclosed by @renovate in #80
• Update pre-commit hook charliermarsh/ruff-pre-commit to v0.9.9 by @renovate in #81
• Update dependency synthetic_home to v4.6.2 by @renovate in #77
• Added the new gemini models and update Flash by @Lash-L in #84
• Update dependency setuptools to v76 by @renovate in #82
• Update pre-commit hook adrienverge/yamllint to v1.36.0 by @renovate in #88
• Add count to assist by @Lash-L in #83
• Fix automation eval by adding model_id by @Lash-L in #87
• Update prebuild script to exclude automations by @allenporter in #90
• Update the results of prebuild output and update prebuild tool to generate consistent outputs by @Lash-L in #86
• Add eval results for Gemma3 4b by @allenporter in #91
• Fix automation leaderboard by @allenporter in #92
• Add gemma3-27b results to the leaderboard by @allenporter in #94
• Add results for qwq and more qwen2.5 models by @tannisroot in #93
• Update dependency setuptools to v77 by @renovate in #97
• Add token stats to the eval report by @allenporter in #99
• Updated scores for 2025.4.0b with new GetHomeState API call by @allenporter in #98
• Record latency performance metrics by @allenporter in #100
• Add duration to automation eavl by @allenporter in #101
• Improvements to automation evaluation by @allenporter in #102
• Update the eval command to generate the full report using the new evaluation approach by @allenporter in http...