fix: UnicodeDecodeError when reading UTF-8 config files on Windows CJK locales(non-UTF-8; e.g. Japanese cp932) #3852
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On Japanese Windows, Python defaults to code page 932 (Shift_JIS), which causes UnicodeDecodeError when reading UTF-8 config files. Force UTF-8 via configargparse._config_file_open_func to ensure locale-independent config loading.
|
I’ve created a new issue for this fix: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi,
This patch fixes a config file decoding issue that occurs on Windows systems with non-UTF-8 locales (e.g., Japanese cp932).
On Windows systems with non-UTF-8 locales (especially CJK: Chinese, Japanese, Korean),
aiderfails to read UTF-8-encoded config files (e.g.,.aider.conf) if they contain non-ASCII characters.For example, Japanese Windows uses code page 932 (Shift_JIS),
so Python's open() fails to decode UTF-8 content by default.
This results in a
UnicodeDecodeErrorduring startup when loading config files.This patch explicitly sets
encoding="utf-8"viaconfigargparse's internal_config_file_open_func, ensuring consistent and locale-independent config parsing.The change is safe for UTF-8 environments and prevents locale-dependent errors
on non-UTF-8 systems (e.g., Japanese Windows).
Thanks!