Skip to content

Preserve RFC 7265 unknown property values verbatim#1450

Open
lcampanella98 wants to merge 5 commits into
collective:mainfrom
lcampanella98:fix-1445-unknown-verbatim
Open

Preserve RFC 7265 unknown property values verbatim#1450
lcampanella98 wants to merge 5 commits into
collective:mainfrom
lcampanella98:fix-1445-unknown-verbatim

Conversation

@lcampanella98

@lcampanella98 lcampanella98 commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Linked issue

Description

Values of unrecognized properties — and X- properties without a VALUE
parameter — were run through RFC 5545 TEXT escaping, so they did not round-trip
unchanged. RFC 7265 reserves the unknown value type precisely for these cases
and requires the value to be carried "without processing" (§5.1/§5.2). Treating
it as TEXT re-escapes it and, as the RFC warns, "breaks round-tripping values".

Example (before this change) — in iCal the ; gains a backslash that wasn't there; in jCal the \, loses its backslash, where RFC 7265 §5.3 gives the value as "Stenophylla;Guinea\\,Africa", preserved exactly:

>>> from icalendar.cal import Component
>>> orig = r"""BEGIN:VEVENT
UID:1
X-COFFEE-DATA:Stenophylla;Guinea\,Africa
END:VEVENT
"""
>>> ev = Component.from_ical(orig)
>>> print(ev.to_ical().decode())
BEGIN:VEVENT
UID:1
X-COFFEE-DATA:Stenophylla\;Guinea\,Africa
END:VEVENT
>>> ev.to_jcal()
['vevent', [['uid', {}, 'text', '1'], ['x-coffee-data', {}, 'unknown', 'Stenophylla;Guinea,Africa']], []]

After this change the value is preserved exactly, matching the RFC 7265 §5.3 example:

>>> print(ev.to_ical().decode())
BEGIN:VEVENT
UID:1
X-COFFEE-DATA:Stenophylla;Guinea\,Africa
END:VEVENT
>>> ev.to_jcal()
['vevent', [['uid', {}, 'text', '1'], ['x-coffee-data', {}, 'unknown', 'Stenophylla;Guinea\\,Africa']], []]

What this changes

  • unknown values are now preserved verbatim when parsing, serializing, and
    converting to/from jCal. The fix skips backslash unescaping on parse and
    emits the stored value unchanged on output.
  • Known types are untouched: an explicit VALUE (e.g. VALUE=TEXT) still gets
    normal TEXT escaping, and property parameters are still decoded as before.
    An unrecognized VALUE type keeps its type name so it round-trips losslessly.
  • vUnknown no longer inherits from vText, so the two cannot silently share
    escaping behaviour again (per maintainer review).
  • PROXIMITY (RFC 9074) is now recognized as TEXT rather than falling back to
    the unknown type — a latent gap this change surfaced. It had passed
    test_calendar_types
    only because vUnknown used to subclass vText.

Known limitation

A jCal unknown value containing a raw line break cannot be represented as a
bare iCal content line (RFC 7265 is silent here, and RFC 5545 makes it
impossible). This is only reachable from jCal input and fails loudly rather than
emitting invalid iCalendar. If a value genuinely needs a newline, it should be
declared VALUE=TEXT.

Checklist

  • I've added a change log entry to /news, following the instructions in Change log entry format.
  • I've added or updated tests if applicable.
  • I've run and ensured all tests pass locally by following Run tests.
  • I've added or edited documentation, both as docstrings to be rendered in the API documentation and narrative documentation, as necessary.

Additional information

Tests are parametrized over the affected characters and conversion directions,
and cover the VALUE boundary, unrecognized parameters, IMAGE, and the
vText/vUnknown overlap. tox -e nopytz passes.

AI disclosure: I used Claude Code (Anthropic's Claude Opus) to investigate the
bug, draft this change and its tests, and write this description. I reviewed the
output, made changes as necessary, and validated everything locally.


📚 Documentation preview 📚: https://icalendar--1450.org.readthedocs.build/en/1450/

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@read-the-docs-community

read-the-docs-community Bot commented Jun 10, 2026

Copy link
Copy Markdown

@stevepiercy stevepiercy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exceptional work. Thank you for your effort!

Most of my suggestions are minor tweaks and clarifications, with one concern about creating a new private method instead of changing the existing one. Would you please take a look? Thank you!

Comment thread news/1445.bugfix.rst Outdated
Comment thread src/icalendar/parser/content_line.py Outdated
Comment thread src/icalendar/parser/content_line.py Outdated
Comment thread src/icalendar/parser/content_line.py Outdated
Comment thread src/icalendar/parser/content_line.py Outdated
Comment thread src/icalendar/prop/unknown.py Outdated
Comment thread src/icalendar/prop/unknown.py Outdated
Comment thread src/icalendar/prop/unknown.py
Comment thread src/icalendar/prop/unknown.py Outdated
Comment thread src/icalendar/cal/component.py Outdated
@lcampanella98

Copy link
Copy Markdown
Collaborator Author

Thanks for the thorough review! Addressed your comments. There are 2 remaining open threads on the versionchanged and the class-level imports. Details in the threads.

@stevepiercy stevepiercy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes, explaining choices made, and adding an example.

I'd like to add to the example, as shown, and it surfaced an issue. Would you please take a look? Thank you!

Comment thread src/icalendar/prop/unknown.py Outdated
Comment thread src/icalendar/prop/unknown.py
Comment thread src/icalendar/prop/unknown.py Outdated
Co-authored-by: Steve Piercy <web@stevepiercy.com>
@lcampanella98

Copy link
Copy Markdown
Collaborator Author

Applied the three Example suggestions (the vText/vUnknown comparison) in one commit. I checked the combined doctest locally and it passes. Verified docs render as well.

@stevepiercy

Copy link
Copy Markdown
Member

@lcampanella98 thank you for your contribution to improve icalendar. It's a pleasure to collaborate with you.

Important

For maintainers, please note that the version number in the docstring in the hidden and unresolved conversation is currently 7.2.0. Please leave this PR open until release, and update if necessary. Thank you!

@lcampanella98

Copy link
Copy Markdown
Collaborator Author

Happy to contribute and collaborate. Thanks for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] unknown value type is escaped/unescaped instead of kept verbatim, breaking round-trips

2 participants