PRO-824: Use Unicode-aware regex in ambiguous text check by SteveJonesDev · Pull Request #1730 · equalizedigital/accessibility-checker

SteveJonesDev · 2026-05-29T04:26:58Z

Summary

Replaces the ASCII-only /[^a-z]+/g regex with the Unicode-aware /[^\p{L}]+/gu variant so that non-Latin characters (e.g. æ, ø, å) are preserved during normalization rather than stripped
Normalizes the translated phrases array with the same regex so both sides of the comparison are treated consistently — prevents translator punctuation from causing false negatives
Fixes detection of ambiguous link text in non-English sites (e.g. Danish "Læs mere", "Klik her")

Closes #84

Test plan

Verify English ambiguous phrases still trigger (e.g. "Read more", "Click here", "Learn more")
Verify a non-Latin ambiguous phrase triggers when the matching translation exists in the bundle (e.g. Danish "Læs mere")
Verify phrases with punctuation variants (e.g. "more...") still match correctly
Verify aria-label and aria-labelledby paths work the same way

🤖 Generated with Claude Code

Summary by CodeRabbit

Bug Fixes
- Enhanced ambiguous text detection to provide better accuracy with international text and non-ASCII character handling.

…languages Fixes detection of translated ambiguous phrases (e.g. Danish "Læs mere") by replacing the ASCII-only /[^a-z]+/g regex with the Unicode-aware /[^\p{L}]+/gu variant. Also normalizes the phrases array on both sides of the comparison so translator punctuation variations don't cause missed matches. Closes #84 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

coderabbitai · 2026-05-29T04:27:11Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 067ed752-72a4-4d26-8deb-7297edef82cb

📥 Commits

Reviewing files that changed from the base of the PR and between 6986dfa and 42abe66.

📒 Files selected for processing (1)

src/pageScanner/checks/has-ambiguous-text.js

📝 Walkthrough

Walkthrough

The ambiguous text detection in has-ambiguous-text.js shifts from ASCII-only character filtering to Unicode letter-based normalization. The change precomputes normalized phrase variants by lowercasing and filtering to Unicode letters, then applies matching normalization to the input text before comparison. This improves consistency and performance by normalizing phrases once during initialization rather than repeatedly during checks.

Changes

Ambiguous Text Detection Normalization

Layer / File(s)	Summary
Unicode-based phrase normalization in ambiguous text detection `src/pageScanner/checks/has-ambiguous-text.js`	Precomputed `normalizedPhrases` now normalizes each translated phrase to lowercase with Unicode letter characters only. `checkAmbiguousPhrase` applies matching normalization to input text and compares against the precomputed set, replacing the previous ASCII-only filtering approach.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested labels

codex

Suggested reviewers

pattonwebz

Poem

In phrases lost to tongues diverse,
Unicode letters now converse—
No ASCII bounds restrict the way,
Ambiguity caught throughout the day! 🐰✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: switching from ASCII-only regex to Unicode-aware regex for ambiguous text detection.
Linked Issues check	✅ Passed	The PR successfully addresses the linked issue `#84` by adding Unicode-aware normalization to support non-Latin languages in ambiguous phrase detection.
Out of Scope Changes check	✅ Passed	All changes are focused on the ambiguous text check normalization logic and directly support the Unicode language support objective from issue `#84`.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch steve/pro-824-localization-and-phrase-filters

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

gemini-code-assist

Code Review

This pull request improves the ambiguous text check by pre-normalizing phrases and utilizing a Unicode-aware regular expression to strip non-letter characters. The reviewer noted that using only \p{L} will strip combining diacritical marks (\p{M}) used in many languages, potentially breaking words and causing matching to fail. They recommended including \p{M} in the regex and applying .normalize() to both the predefined phrases and the input text to ensure consistent and robust matching.

gemini-code-assist · 2026-05-29T04:28:11Z

+const normalizedPhrases = ambiguousPhrases.map(
+	( p ) => p.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim()
+);


Using only \p{L} (Letters) in the Unicode regex will strip combining diacritical marks (\p{M}), which are used in many languages (e.g., Hindi, Arabic, Thai, or even Latin languages when strings are in NFD/decomposed form, such as text copied from macOS). Stripping these marks will break words into fragments and cause matching to fail.

To prevent this, we should:

Include \p{M} in the character class: /[^\p{L}\p{M}]+/gu.

Use .normalize() to ensure consistent Unicode normalization (NFC) before processing.

Suggested change

const normalizedPhrases = ambiguousPhrases.map(

( p ) => p.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim()

);

const normalizedPhrases = ambiguousPhrases.map(

( p ) => p.normalize().toLowerCase().replace( /[^\p{L}\p{M}]+/gu, ' ' ).trim()

);

gemini-code-assist · 2026-05-29T04:28:11Z

+	text = text.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim();
+	return normalizedPhrases.includes( text );


Apply the same Unicode-aware normalization (including \p{M} and .normalize()) to the input text to ensure consistent matching with the normalized phrases.

Suggested change

text = text.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim();

return normalizedPhrases.includes( text );

text = text.normalize().toLowerCase().replace( /[^\p{L}\p{M}]+/gu, ' ' ).trim();

return normalizedPhrases.includes( text );

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42abe667f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-29T04:28:43Z

 	}
-	text = text.toLowerCase().replace( /[^a-z]+/g, ' ' ).trim();
-	return ambiguousPhrases.includes( text );
+	text = text.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim();


Preserve combining marks when normalizing Unicode text

For locales with accented translations, this drops combining marks instead of treating canonically equivalent text the same. For example, the Vietnamese translation for “continue” is tiếp tục; if the DOM contains the same visible text in decomposed form from copy/paste, textContent normalizes to something like tie p tu c while the precomposed translated phrase normalizes to tiếp tục, so the ambiguous link is missed. Include marks in the character class or normalize both strings to the same Unicode form before stripping punctuation.

Useful? React with 👍 / 👎.

gemini-code-assist Bot reviewed May 29, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed May 29, 2026

View reviewed changes

Navigation Menu

Uh oh!

PRO-824: Use Unicode-aware regex in ambiguous text check - #1730

PRO-824: Use Unicode-aware regex in ambiguous text check#1730
SteveJonesDev wants to merge 1 commit into
developfrom
steve/pro-824-localization-and-phrase-filters

SteveJonesDev commented May 29, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 29, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Suggested labels

Suggested reviewers

Poem

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		text = text.toLowerCase().replace( /[^\p{L}]+/gu, ' ' ).trim();
		return normalizedPhrases.includes( text );

Uh oh!

Conversation

SteveJonesDev commented May 29, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested labels

Suggested reviewers

Poem

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

SteveJonesDev commented May 29, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 29, 2026 •

edited

Loading