Skip to content

Simplify source submission: auto-detect icons, centralize translations, add scaffolding#4

Closed
markvp wants to merge 43 commits into
masterfrom
claude/simplify-source-submission-oZUVh
Closed

Simplify source submission: auto-detect icons, centralize translations, add scaffolding#4
markvp wants to merge 43 commits into
masterfrom
claude/simplify-source-submission-oZUVh

Conversation

@markvp

@markvp markvp commented Apr 2, 2026

Copy link
Copy Markdown
Owner

Summary

This PR significantly reduces the boilerplate required to submit new waste collection sources. 544 files changed, ~2,770 lines removed net.

A minimal new source goes from ~100 lines to ~20 lines.

Changes

  • Framework-level icon auto-detection — new service/icons.py with guess_icon() that maps waste type names to MDI icons via keyword matching across 12+ languages (English, German, French, Italian, Dutch, Swedish, Polish, Czech, Norwegian, Danish, Finnish, Spanish). Icons are auto-assigned in Collection.__init__() when none is explicitly passed. Removes or reduces ICON_MAP from 521 source files.

  • Expanded default_translations.py — adds ~35 missing common parameters (street, address, city, house_number, suburb, etc.) across all 4 languages (en/de/it/fr). Removes redundant PARAM_TRANSLATIONS from 77 source files where they duplicated defaults.

  • Source scaffolding CLI (scripts/new_source.py) — interactive + non-interactive tool to generate new source files with correct boilerplate structure.

  • Updated contributing docs (doc/contributing_source.md) reflecting that ICON_MAP and PARAM_TRANSLATIONS are no longer needed for common cases.

How icon auto-detection works

Collection.__init__() now calls guess_icon(waste_type) when no icon= is passed. The function uses keyword-in-string matching against a curated database of ~200 waste type keywords across 12+ languages:

# Before (every source needed this):
ICON_MAP = {
    "Restmüll": "mdi:trash-can",
    "Biotonne": "mdi:leaf",
    "Gelber Sack": "mdi:recycle",
    "Papier": "mdi:package-variant",
}
# ...
Collection(date=d, t=t, icon=ICON_MAP.get(t))

# After (framework handles it):
Collection(date=d, t=t)

Sources can still pass icon= explicitly to override auto-detection. Existing ICON_MAP entries that can't be auto-detected are preserved.

Related upstream PRs (mampfes/hacs_waste_collection_schedule)

Test plan

  • pytest tests/test_source_components.py — 6/6 pass
  • python update_docu_links.py && git diff --exit-code — clean
  • Rebased onto latest upstream master (resolved 3 conflicts: ecoharmonogram_pl, lumire_se, southkesteven_gov_uk)
  • Spot-check icon auto-detection on representative sources across languages
  • Review keyword coverage in service/icons.py for edge cases
  • Verify scaffolding CLI generates valid source files

https://claude.ai/code/session_01Q9WEShLzCkFySi98PUYumA

chex7 and others added 30 commits January 10, 2026 21:23
Lumire API moved from wordpress to REST
- Added `moonee_valley_vic_gov_au.py` source file
- Uses ArcGIS FeatureServer API for address lookup
- Supports three bin types:
  - General Waste (weekly)
  - Recycling (fortnightly)
  - FOGO (fortnightly)
- Added documentation in `doc/source/` include howto setup in HomeAssistant UI.
- Added entry to README.md
- Added unit tests
- Registered in `sources.json`# Please enter the commit message for your changes. Lines starting
Please enter the commit message for your changes. Lines starting
Add Watford Borough Council source
Add Watford Borough Council documentation
- Replace unimported SourceArgumentRequired with SourceArgumentExceptionMultiple
  referencing both 'uprn' and 'address' arguments
- Normalise UPRN (strip leading zeros) once in _resolve_identifiers and reuse
  the normalised value across all subsequent lookups
- Pass datetime.date object to Collection instead of a string
- Defer _fetch_calendar() call until it is actually needed (error path only)
- Remove unreachable duplicate raise SourceArgumentException block
- Change bold heading to proper ## heading for configuration.yaml section
- Add ### subheading for Configuration Variables (consistent with other sources)
- Update uprn description to reflect that address is also accepted
- Add REQUEST_TIMEOUT = 30 constant and apply to all GET/POST requests
  to prevent indefinite hangs if the council endpoint stalls
- Use the argument the user actually configured (uprn or address) when
  raising SourceArgumentException, rather than always reporting 'uprn'
The CCC API can return stale next_planned_date_app values that are in
the past, causing collections to be missing. This mirrors the fix used
on the CCC website itself: advance past dates forward by the collection
interval (1 week for Organic, 2 weeks for others) until they are in
the future. Also skip bins with pick_up_group "Daily" or "Not Collected".

Fixes mampfes#5605

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…chedule/source/ccc_govt_nz.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…r Schadstoffmobil events in AWIDO source

Agent-Logs-Url: https://github.com/mampfes/hacs_waste_collection_schedule/sessions/df12d8ef-4939-4858-9f0f-b2e32e136eb9

Co-authored-by: markvp <6936351+markvp@users.noreply.github.com>
The EcoHarmonogram API supports multiple languages (pl, en, uk, ru) but
the language was hardcoded to "pl". This adds a configurable language
parameter so users can receive waste type names in their preferred language.

Closes mampfes#5639

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After advancing a stale collection date forward by the collection
interval, recheck overrides so that holiday/special-date overrides
are not missed for the corrected date. Includes a safety limit of
52 iterations to prevent infinite loops.

Resolves Copilot review comment on PR mampfes#5645.

https://claude.ai/code/session_01VztFqugBBTcbvDrV6T5Zod
…es-5605

Fix ccc_govt_nz missing collections from stale API dates
…anguage-5639

Add language support to EcoHarmonogram source
markvp and others added 13 commits April 3, 2026 04:03
…ource

Add City of Moonee Valley (Australia) waste collection source
Regenerate sources.json, source_metadata.json, translations,
and info.md for the Watford Borough Council source.

https://claude.ai/code/session_01YM2x5fg1TDwkQevbnt9QwB
Run update_docu_links.py to regenerate doc files
…fservice

Fix southkesteven_gov_uk after SKDC endpoint change
Fix muellmax_de: restore two-step street selection, fix session handling and duplicate key
…ttributes-wastemobil

AWIDO: Add location/time attributes for Schadstoffmobil events
Replace all cloudscraper usage with curl_cffi across 6 sources
(valorlux_lu, chichester_gov_uk, stockton_gov_uk, east_renfrewshire_gov_uk,
awb_es_de, mustankorkea_fi) and migrate 3 additional sources with 403 bugs
(eastleigh_gov_uk, islington_gov_uk, hume_vic_gov_au) from plain requests
to curl_cffi to fix Cloudflare/bot detection blocking.

curl_cffi's TLS fingerprint impersonation is more robust than cloudscraper's
JS challenge solving, as proven in PR mampfes#5706 (gateshead_gov_uk).

Fixes mampfes#5665, mampfes#4949, mampfes#4961, mampfes#5306
Refs mampfes#5714

https://claude.ai/code/session_018M8xvqcZLKmus16xZdaJcB
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…acement-O2dy8

Replace cloudscraper with curl_cffi Chrome impersonation
…s, add scaffolding

- Add framework-level icon auto-detection in Collection.__init__() via guess_icon()
  so sources no longer need to define ICON_MAP. The new service/icons.py module
  maps waste type names to MDI icons using keyword matching across 12+ languages.

- Remove or reduce ICON_MAP from 521 source files (100 fully removed, 399 reduced
  to only override entries that guess_icon can't handle, 22 kept for business logic).

- Expand default_translations.py with ~35 missing common parameters (street, address,
  city, house_number, suburb, etc.) across all 4 languages (en/de/it/fr).

- Remove redundant PARAM_TRANSLATIONS from 77 source files (15 fully removed,
  62 reduced to only source-specific entries).

- Add scripts/new_source.py scaffolding CLI to generate new source files.

- Update contributing docs to reflect that ICON_MAP and PARAM_TRANSLATIONS
  are no longer needed for common cases.

Net result: -2,772 lines across 537 files. New sources require significantly
less boilerplate code.

https://claude.ai/code/session_01Q9WEShLzCkFySi98PUYumA
@markvp
markvp force-pushed the claude/simplify-source-submission-oZUVh branch from 072dc9f to 89cffe6 Compare April 2, 2026 23:41
@markvp markvp closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants