Skip to content

fix: treat semicolons as chunk separators#3426

Open
azmifarih wants to merge 1 commit into
Automattic:masterfrom
azmifarih:fix/semicolon-chunk-separator-3405
Open

fix: treat semicolons as chunk separators#3426
azmifarih wants to merge 1 commit into
Automattic:masterfrom
azmifarih:fix/semicolon-chunk-separator-3405

Conversation

@azmifarih
Copy link
Copy Markdown

@azmifarih azmifarih commented May 19, 2026

Issues

Fixes #3405

Description

Treat semicolons as chunk terminators so end-of-chunk expression behavior is consistent with commas and similar punctuation.

This patch:

  • adds Punctuation::Semicolon to TokenKind::is_chunk_terminator()
  • adds chunk-splitting tests for comma and semicolon in token_string_ext.rs
  • switches WrongApostrophe from Chunk to Sentence scope to preserve context after chunk boundary changes

Demo

N/A

How Has This Been Tested?

  • just test
    • All relevant Rust/core tests passed.
    • One local Chrome extension options UI test failed:
      [chromium] tests/options_structured_rules.spec.ts › structured rule settings › rule dropdown updates only the targeted flat rule

AI Disclosure

  • I am a human and didn't use any AI.
  • I used LLM features of my editor, but not an agent.
  • I used an AI agent interactively.
  • I am an agent or I got an agent to do the work autonomously.

If Your PR Implements or Enhances a Linter

  • I made up the sentences in the unit tests.
  • The sentences in the unit tests were generated by an AI.
  • I'm using examples from the bug report / feature request.
  • I collected real-world sentences for the unit tests.

Checklist

  • I have performed a self-review of my own code
  • I have added tests to cover my changes
  • I have considered splitting this into smaller pull requests.

@azmifarih azmifarih force-pushed the fix/semicolon-chunk-separator-3405 branch from eb43703 to e4490a1 Compare May 19, 2026 08:10
@hippietrail hippietrail added rust Pull requests that update Rust code harper-core punctuation An issue related to punctuation checking labels May 19, 2026
@hippietrail
Copy link
Copy Markdown
Collaborator

Thanks! Did you try testing everything? just test will test all the integrations as well as just the Rust stuff. I'm a bit worried it could break something unexpectedly. Watch out though as it can bog down your system if you're low on resources. It can open ten browser windows at a time over and over, etc.

@azmifarih
Copy link
Copy Markdown
Author

For just test I only have one error in test-chrome-plugin:

[chromium] tests/options_structured_rules.spec.ts › structured rule settings › rule dropdown updates only the targeted flat rule

It seems outside of the PR.

@hippietrail hippietrail requested a review from elijah-potter May 20, 2026 04:30
Copy link
Copy Markdown
Collaborator

@hippietrail hippietrail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything I see here makes sense. Since you say all tests pass even with just test that it's surely ready. Thanks!

Oh add that you did that to the testing section of your PR body above too so everyone knows without reading through the comments.

@azmifarih
Copy link
Copy Markdown
Author

Everything I see here makes sense. Since you say all tests pass even with just test that it's surely ready. Thanks!

Oh add that you did that to the testing section of your PR body above too so everyone knows without reading through the comments.

Done for this. Thank you for your advice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

harper-core punctuation An issue related to punctuation checking rust Pull requests that update Rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Semicolons are not handled as chunk separators like commas are.

2 participants