fix: pass encoding='utf-8' to Path.write_text on all text outputs#34
Open
Ori-Levi1 wants to merge 1 commit into
Open
fix: pass encoding='utf-8' to Path.write_text on all text outputs#34Ori-Levi1 wants to merge 1 commit into
Ori-Levi1 wants to merge 1 commit into
Conversation
Path.write_text() falls back to locale.getpreferredencoding(False) when no
encoding is given. On a Windows machine running a non-UTF-8 system locale
(e.g. Hebrew/cp1255, Cyrillic/cp1251, Japanese/cp932), writing strings that
contain characters outside the ANSI codepage raises UnicodeEncodeError.
Repro on Windows with Hebrew locale (cp1255):
pipx install evoskill
cd <empty-dir>
evoskill init
# Crash at init.py:355 — TASK_MD_TEMPLATE contains '->' arrow (U+2192)
# which cannot be encoded to cp1255.
Six write_text() call sites are affected. Every one writes content that may
contain non-ASCII characters (UTF-8 templates, evolved skill bodies,
proposer prompts, JSON checkpoints, markdown reports, OpenCode JSON
config). Pinning encoding='utf-8' fixes the immediate crash and makes the
output deterministic regardless of the operating system locale.
Files:
src/cli/commands/init.py # task.md + state.json
src/cli/report.py # run-<timestamp>.md
src/harness/opencode/options.py # opencode JSON config
src/harness/opencode/skill_utils.py # SKILL.md frontmatter
src/loop/helpers.py # evolved prompt files
src/loop/runner.py # loop_checkpoint.json
The two write_text() sites in src/docker/launcher.py (compose file) and
src/remote/base.py (remote run record) are out of scope for this fix
since the user paths that hit them haven't been verified on a Hebrew
locale, but the same change is recommended in a follow-up PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Path.write_text()falls back tolocale.getpreferredencoding(False)when theencodingargument is omitted. On Windows machines running a non-UTF-8 system locale (Hebrew/cp1255, Cyrillic/cp1251, Japanese/cp932, etc.), writing strings that contain characters outside the ANSI codepage raisesUnicodeEncodeError.This PR pins
encoding='utf-8'on everywrite_text()call site that emits text content.Reproduction (Windows + Hebrew locale)
Crash:
TASK_MD_TEMPLATEcontains→(U+2192), which is not representable in cp1255.Files changed
src/cli/commands/init.py:355task.md(TASK_MD_TEMPLATE — contains→)evoskill initimmediatelysrc/cli/commands/init.py:368state.jsonsrc/cli/report.py:62run-<timestamp>.md(run report)src/harness/opencode/options.py:107src/harness/opencode/skill_utils.py:76SKILL.md(frontmatter + body)src/loop/helpers.py:227src/loop/runner.py:196loop_checkpoint.jsonEach change is
write_text(x)→write_text(x, encoding='utf-8'). No behavior change on UTF-8-default systems (most Linux/macOS); fixes the crash on Windows non-UTF-8 locales.Why this matters beyond the immediate crash
Even if a user gets past
evoskill init(e.g., on a US-English Windows install), the SKILL.md and prompt files are written by the LLM-driven proposer. The proposer routinely emits Unicode (em-dashes, arrows, curly quotes) when it rewrites a skill. Soevoskill runis also affected on any non-UTF-8 locale, just on a less predictable iteration. Pinning UTF-8 once removes the entire class of bugs.Out of scope
Two more
write_text()call sites exist that were not patched here because I didn't exercise their code paths during reproduction:src/docker/launcher.py:111(compose file)src/remote/base.py:28(remote run record)The same change is recommended in a follow-up.
Test plan
evoskill initsucceeds on Windows 11 + Hebrew locale (cp1255) + Python 3.14evoskill runwritesSKILL.mdwith em-dashes and arrows without error