iCalendar files are supposed to be UTF-8, but files from older clients (Lotus Notes, old Outlook) show up in Latin-1, Windows-1252, or with no declared encoding. Right now these fail with a UnicodeDecodeError before anonymization even starts.
Implementation
src/icalendar_anonymizer/io.py (or wherever file reading currently lives): try UTF-8 first, then fall back to chardet detection, then to Latin-1 as a last resort. Log what encoding was detected.
def read_calendar_file(path) -> str:
raw = Path(path).read_bytes()
try:
return raw.decode("utf-8")
except UnicodeDecodeError:
detected = chardet.detect(raw)
encoding = detected.get("encoding") or "latin-1"
return raw.decode(encoding, errors="replace")
Add chardet as an optional dep. If not installed, fall back to Latin-1 directly with a warning.
Output is always UTF-8.
iCalendar files are supposed to be UTF-8, but files from older clients (Lotus Notes, old Outlook) show up in Latin-1, Windows-1252, or with no declared encoding. Right now these fail with a UnicodeDecodeError before anonymization even starts.
Implementation
src/icalendar_anonymizer/io.py(or wherever file reading currently lives): try UTF-8 first, then fall back to chardet detection, then to Latin-1 as a last resort. Log what encoding was detected.Add
chardetas an optional dep. If not installed, fall back to Latin-1 directly with a warning.Output is always UTF-8.
--encodingflag to force a specific encoding