Skip to content

Commit b042996

Browse files
committed
fix: escape literal emphasis markers in MarkdownRenderer
MarkdownRenderer.text() only escaped backticks, so a text token that is a literal "*"/"_" emphasis delimiter was emitted unescaped and re-parsed as emphasis on the round-trip. This silently changed the document's meaning, violating the invariant asserted by TestMarkdownRendererRoundTrip ("reformatting valid Markdown must not change its meaning"). An escaped marker in the source (``\*not emphasis\*``) is stored by the inline parser as bare "*" text tokens (the backslash is consumed and the parser's internal no-emphasis flag is dropped before the AST). Re-emitting those tokens without a backslash let them pair up again: reformat(r"\*not emphasis\*") -> "*not emphasis*" html(reformat(...)) -> "<p><em>not emphasis</em></p>" # was <p>*not emphasis*</p> The same happened for "_", "**"/"__", and unmatched leftover markers. Fix: in text(), if a text token consists entirely of "*"/"_" it is a literal delimiter (an escaped marker or an unmatched leftover), so backslash-escape each character. Prose punctuation such as "2 * 3" or "snake_case" always arrives mixed with other characters, so it is left untouched and is not over-escaped (guarded by the existing test_prose_not_over_escaped and a new test_real_emphasis_not_over_escaped). Added test_escaped_emphasis_markers and test_real_emphasis_not_over_escaped to TestMarkdownRendererRoundTrip. The regression test fails without the source change (asserts <em>...</em> reappears) and passes with it; full suite 1122 passed, ruff check/format and mypy clean.
1 parent fe02f40 commit b042996

3 files changed

Lines changed: 45 additions & 1 deletion

File tree

docs/changes.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,12 @@ Changelog
33

44
Here is the full history of mistune v3.
55

6+
Unreleased
7+
----------
8+
9+
* Escape literal ``*``/``_`` emphasis markers in the Markdown renderer so
10+
round-tripping escaped text does not re-introduce emphasis.
11+
612
Version 3.3.2
713
-------------
814

src/mistune/renderers/markdown.py

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,18 @@ def render_children(self, token: Dict[str, Any], state: BlockState) -> str:
4040
return self.render_tokens(children, state)
4141

4242
def text(self, token: Dict[str, Any], state: BlockState) -> str:
43+
raw = cast(str, token["raw"])
44+
# a text token that is made up entirely of "*"/"_" is a literal
45+
# emphasis delimiter -- either an escaped marker from the source
46+
# (``\*``) or an unmatched leftover -- so every character must stay
47+
# escaped, or it would re-parse as emphasis on the round-trip. Prose
48+
# punctuation such as ``2 * 3`` or ``snake_case`` arrives mixed with
49+
# other characters and is left untouched.
50+
if raw and all(c in "*_" for c in raw):
51+
return "".join("\\" + c for c in raw)
4352
# a backtick always opens a code span, so it must stay escaped to
4453
# survive a re-parse as literal text.
45-
return cast(str, token["raw"]).replace("`", "\\`")
54+
return raw.replace("`", "\\`")
4655

4756
def emphasis(self, token: Dict[str, Any], state: BlockState) -> str:
4857
return "*" + self.render_children(token, state) + "*"

tests/test_renderers.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,35 @@ def test_escaped_marker_inside_list_item(self):
4646
def test_escaped_backtick(self):
4747
self.assert_round_trip(r"\`not code\`" + "\n")
4848

49+
def test_escaped_emphasis_markers(self):
50+
# an escaped "*"/"_" is a literal delimiter, not emphasis; re-emitting
51+
# it unescaped would turn plain text back into <em>/<strong>
52+
for text in (
53+
r"\*not emphasis\*" + "\n",
54+
r"\_not emphasis\_" + "\n",
55+
r"\*\*not strong\*\*" + "\n",
56+
r"\_\_not strong\_\_" + "\n",
57+
r"a word\_with\_underscores" + "\n",
58+
r"2 \* 3 \* 4" + "\n",
59+
r"trailing star\*" + "\n",
60+
r"\*leading star" + "\n",
61+
):
62+
self.assert_round_trip(text)
63+
64+
def test_real_emphasis_not_over_escaped(self):
65+
# genuine emphasis must survive untouched, and prose punctuation with
66+
# spaces around a "*"/"_" must not gain stray backslashes
67+
for text in (
68+
"*emphasis*\n",
69+
"**strong**\n",
70+
"***both***\n",
71+
"_under_ and __strong__\n",
72+
"a *b* and _c_ mixed\n",
73+
"2 * 3 = 6 and 4 * 5\n",
74+
"snake_case_variable\n",
75+
):
76+
self.assert_round_trip(text)
77+
4978
def test_codespan_containing_backticks(self):
5079
# the delimiter must grow past any backtick run inside the code span,
5180
# and pad away a leading/trailing backtick, or the re-parse breaks

0 commit comments

Comments
 (0)