Skip to content

Support non-UTF-8 encoded iCalendar files #160

Description

@SashankBhamidi

iCalendar files are supposed to be UTF-8, but files from older clients (Lotus Notes, old Outlook) show up in Latin-1, Windows-1252, or with no declared encoding. Right now these fail with a UnicodeDecodeError before anonymization even starts.

Implementation

src/icalendar_anonymizer/io.py (or wherever file reading currently lives): try UTF-8 first, then fall back to chardet detection, then to Latin-1 as a last resort. Log what encoding was detected.

def read_calendar_file(path) -> str:
    raw = Path(path).read_bytes()
    try:
        return raw.decode("utf-8")
    except UnicodeDecodeError:
        detected = chardet.detect(raw)
        encoding = detected.get("encoding") or "latin-1"
        return raw.decode(encoding, errors="replace")

Add chardet as an optional dep. If not installed, fall back to Latin-1 directly with a warning.

Output is always UTF-8.

  • Test with a Latin-1 encoded file from a real client
  • Test with Windows-1252
  • Test that output is always valid UTF-8
  • CLI: add --encoding flag to force a specific encoding

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Needs triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions