Skip to content

tools: read UTF-16 package files in the offline matcher - #3629

Open
cixiangtao wants to merge 1 commit into
Gathin23:mainfrom
cixiangtao:codex/match-utf16-packages
Open

cixiangtao wants to merge 1 commit into
Gathin23:mainfrom
cixiangtao:codex/match-utf16-packages

Conversation

@cixiangtao

Copy link
Copy Markdown
Contributor

match.py read saved package files only as UTF-8, so the documented pm list packages > packages.txt workflow in Windows PowerShell 5.1 produced exit 2 with "no package names". Detect a UTF-16 BOM before decoding, retaining the existing UTF-8/BOM behavior for other files, and document the accepted package-file formats.

Tested on: macOS, Python 3.13.9 and 3.14.6, with generated encoding fixtures and isolated CLI subprocesses. No native Windows PowerShell 5.1 or TV run is claimed.

Microsoft's encoding documentation confirms that Windows PowerShell redirection defaults to UTF-16LE and writes a BOM. The regression covers UTF-16LE/BE, UTF-8 with/without BOM, CRLF, bare names, pm -f lines and duplicate names. CLI output has identical counts and never markers across the formats, and input files remain byte-for-byte intact.

Validation:

  • The two new regression methods produced two failures and two errors on unmodified base 294d3d59; the UTF-8 controls passed.
  • tools/venv/bin/python -m unittest tests.test_match -v: 37 tests passed on Python 3.13.9.
  • python3 -m unittest tests.test_match -v: 37 tests passed on Python 3.14.6.
  • CLEANYOURTV_STRICT=1 tools/venv/bin/python -m unittest discover -s tests -v: 3211 tests, 30 skipped, one failure in test_relay.RelayTest.test_listens_on_loopback_only (expected the LAN connection to raise OSError, but it connected). The exact method was rerun this round on a clean worktree of unmodified base 294d3d59, and failed with the same assertion. No relay code or checks were changed.
  • git diff --check passed.

TV-list, launcher, theme and add-on checklist sections are not applicable to this offline host-tool fix. Prepared with OpenAI Codex; the checks above were run locally.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

maintainer-bot review · this one needs a maintainer decision

Thanks for the PR! Couldn't review this one automatically; it goes to the maintainer.

Checks
  • Size: size/S (+42 -1, 2 files)

Reviewed by maintainer-bot · runs on 0G Private Computer

@github-actions github-actions Bot added needs-maintainer scripts build-and-install, disable, restore, tv.py and the other tools labels Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-maintainer scripts build-and-install, disable, restore, tv.py and the other tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant