Releases: Yggdrasil-AI-labs/wigle-to-wdgwars
Release list
v1.6.0 — skip already-processed uploads
Stops the daily --from-wigle pull from re-downloading uploads it already pushed. Records each successfully-pushed transid in ~/.config/wigle-to-wdgwars/processed-transids.json and skips re-pulling it (WiGLE regenerates each CSV server-side, minutes per large upload). Recorded only on a real non-dry-run success; new --reprocess flag forces a re-pull. +7 tests.
v1.5.1 — survive slow WiGLE CSV exports
Fix: wigle_download_csv retries the CSV download with a longer read timeout (600s, then 900s) instead of a flat 300s. WiGLE builds the CSV server-side, so large uploads can take minutes; the 300s ceiling crashed the daily run with an uncaught TimeoutError.
v1.5.0 — trailing-window gate (--since)
Add a --since gate (default 7d) so cron jobs stop re-uploading the entire WiGLE history every tick. Row-level FirstSeen filter before chunking; --all-time disables it.
v1.4.0 — HTTP 413 auto-bisect
Auto-bisect a chunk that trips the WDGoWars upload cap: on HTTP 413, split the batch and retry the halves instead of failing the run.
v1.3.0 — family-parity catch-up
Summary
Closes the flag-surface gaps between wigle-to-wdgwars and its sibling feeders Muninn and Heimdall. Each of the five flags below already shipped in the other two uploaders; this brings wigle-to-wdgwars to full parity.
| Flag | Muninn | Heimdall | Wigle before | Wigle after |
|---|---|---|---|---|
--update |
yes | yes | no | yes |
--preview |
yes | yes | no | yes |
-q / --quiet |
yes | yes | no | yes |
--no-version-check |
yes | yes | no | yes |
--api-url |
yes | yes | no | yes |
--watch |
yes | n/a | n/a | n/a (documented) |
What changed
Added
--update:git pull --ff-onlywhen this is a git checkout, otherwise raw-GitHub fetch ofwigle_to_wdgwars.py+requirements.txt. Atomic replace, pip refresh after. Lifted from Heimdall.--preview: prints first 6 WiGLE CSV rows as JSON-lines, decompresses gzip transparently. No network. Matches Heimdall + Muninn semantics.-q/--quiet: suppress informational banners (errors still print). Gates the version-check nudge.--no-version-check: skip the daily GitHub releases probe entirely.--api-url URL: override the CSV upload endpoint. Useful for staging or local mocks. Aircraft JSON uploads still use the signed endpoint unchanged.- Daily-cached version-check nudge in
main(), gated by both flags above. SECURITY.md, refreshed for the v1.3.0 surface.
Not added
--watchis intentionally absent. WiGLE CSV uploads are bulk one-shot; the polling equivalent for this tool is--from-wigle --schedule, which already exists. Documented in CHANGELOG.
Tests
tests/test_family_parity.py: 10 new offline tests coveringpreview_csv(gzip, short files, empty data sections),_check_for_updatecache hits + network-error path,--api-urlruntime ENDPOINT override, and--update/--previewdispatch inmain(). Network mocked throughout.- Existing 46 tests still pass. Total: 56 tests, all green.
Test plan
- AST parse + module import
-
--versionreturns 1.3.0 -
--helpshows all 5 new flags -
--preview /tmp/sample.csvprints JSON rows, no upload -
--api-url http://localhost:9999/fake --dry-runswaps ENDPOINT, doesn't POST -
python -m unittest discover tests/— 56 pass -
bash scripts/smoke.sh— all green, including the dry-run systemd unit render with no key leak - Live
--scheduleinstall against a real systemd user manager (deferred to pre-release manual checklist) - Live
--updateagainst this branch once merged (verifies the raw-GitHub path)
v1.2.1 — family-alignment housekeeping
What's Changed
- v1.2.1: family-alignment housekeeping (gungnir pin + safety nets) by @HiroAlleyCat in #5
Full Changelog: v1.2.0...v1.2.1
v1.2.0 — guided setup + auto-installed daily timer
From a fresh clone to a working dry-run daily timer in one command:
git clone https://github.com/HiroAlleyCat/wigle-to-wdgwars.git
cd wigle-to-wdgwars
./setup.sh # Linux / Mac / Pi — saves both keys, installs daily timerREM Windows
setup.batWhat's new
--setupinteractive wizard. Prompts for the WDGoWars API key, validates against/api/me, saves to~/.config/wigle-to-wdgwars/wdgwars.key(mode 600). Then the WiGLE "Encoded for use" token, validated against/api/v2/file/transactions. Then offers a daily timer.--save-key/--save-wigle-keyfor non-interactive provisioning.--schedule/--unscheduleauto-installer. Writes the right artifact for your host: systemd user.timer+.service(Linux+systemd), cron line (Mac / Linux-without-systemd), schtasks entry (Windows). Defaults daily at 03:00 with--from-wigle --chunk-size 10000, in dry-run on first install. Re-run--schedulewithout--schedule-dry-runto flip to live uploads. Marker comment in every artifact lets--unscheduleclean up without touching anything else in your crontab / unit dir.- Bootstrap shim scripts:
setup.sh/setup.bat/run.sh/run.bat/update.sh/update.bat. Double-click on Windows or run from a terminal — they handle the venv + deps + key save + timer install end-to-end. - 46 offline unit tests +
scripts/smoke.sh. Tests blockurllib.request.urlopenat the class level so any un-mocked live call fails loud. Smoke does a throwaway venv + deps + tests + a real systemd headless--schedulethat asserts no--key/--wigle-keyever leaks into the rendered unit file. - CI matrix: Python 3.10 / 3.11 / 3.12.
Security
Scheduled commands never include --key / --wigle-key on the CLI — they read keys from the saved key files at run time. Unit files are world-readable via systemctl cat; baking a secret in there would leak it. Both the unit tests and the smoke script regression-net this.
Backward compatibility
All v1.1.x CLI flags, env vars, and on-disk paths are unchanged. Existing recipes from the README still run as written. The new flags are additive.
Live-tested
| Platform | Install | Trigger | Uninstall |
|---|---|---|---|
| Ubuntu 24.04 + systemd 255 | ✓ | ✓ | ✓ |
| Windows 11 Pro 24H2 + schtasks | ✓ (239/261 chars under /TR limit) |
✓ | ✓ |
Real WDGoWars key + synthetic CSV + --dry-run |
✓ (multipart built, dry-run safety stop fired, hwm.json untouched) |
n/a | n/a |
| CI matrix 3.10 / 3.11 / 3.12 | ✓ | n/a | n/a |
Upgrade
./update.sh # bootstrap script — git pull + pip upgrade + version check
# or manually:
git pull && .venv/bin/pip install --upgrade -r requirements.txtSee CHANGELOG.md for the full release notes.
v1.1.2 — README install: PEP 668 / Bookworm fix
README install snippets now create a project-local .venv/ instead of python3 -m pip install against the system Python.
Found by sweeping the feeder family after a Pi24 user reported the same crash in Muninn's setup.sh (Muninn v2.0.8, Heimdall v0.2.2). Wigle has no wrapper scripts to fix, so this is a docs-only release.
Fixed
- README "Option A — ZIP download" and "Option B — clone with git" now use
python3 -m venv .venvfollowed by.venv/bin/pip installand.venv/bin/python wigle_to_wdgwars.py. No moreerror: externally-managed-environmenton Bookworm / Debian 12+ / Ubuntu 23.04+ / Homebrew Python. - "Updating" section matches.
- Inline note explains what to do if
python3 -m venvitself errors out (missingpython3-venvapt package).
Unchanged
- Windows install block — Windows Python has no PEP 668 enforcement.
wigle_to_wdgwars.pyitself — no code change, only the version string.
Verified
Ran the new Option B block verbatim on Ubuntu 24.04 (same PEP 668 wall as Bookworm Pi24): venv create + pip install + --help with zero externally-managed errors.
Full changelog: https://github.com/HiroAlleyCat/wigle-to-wdgwars/blob/main/CHANGELOG.md
v1.1.0 — signed-JSON transport via gungnir
Structural refactor of the signed-JSON path onto gungnir 0.1.1 — the same library Muninn v2.0 sits on. Multipart CSV and WiGLE-side fetch stay local. Wire-protocol unchanged.
Free behavior wins on the signed-JSON path
- Retry 5xx + network errors with exponential backoff (3 attempts).
- 429 stops the batch and persists a cooldown the next cron tick respects.
- Silent-drop detection (HTTP 200 ok:true with every counter zero now exits non-zero). v1.0 had no detection.
- Inter-chunk cooldown of 1s (configurable via gungnir Client).
- User-Agent now includes the repo URL per RFC bot-UA convention.
- API-key redaction in log lines.
- `--aircraft-batch` default dropped 1000 → 500 per locosp's 100-500 range.
Breaking changes
- `whoami()` emits structured stderr logs (`key OK — user=X` + summary) instead of dumping raw JSON to stdout. Cron parsers that consumed the JSON must update.
- `_post_signed()` return value: 200 on success, 1 on failure. The exact HTTP code for non-2xx is no longer surfaced (gungnir handles 5xx with retry internally).
Migration
```
pip install -r requirements.txt # new file, pulls gungnir from the pinned tag
```
API key at `~/.config/wigle-to-wdgwars/wdgwars.key` keeps working on every OS — local `load_key()` is preserved deliberately.
Verification
Live `--whoami` against wdgwars.pl: `key OK — user=AlleyCat` with correct account stats.