Add character count `Intl.Segmenter` support by colinrotherham · Pull Request #6995 · alphagov/govuk-frontend

colinrotherham · 2026-04-28T17:28:45Z

This PR updates the character count component to (optionally) use Intl.Segmenter

It closes #1104, ~~#1364~~ and partly #2888

Improved character count counting

I've added an optional countType option to the character count component to enable improved counting with Intl.Segmenter whilst maintaining backwards compatibility.

This feature was introduced because JavaScript counts String: length in code units not characters:

String	Length	Remarks
cafȩ́	5	The character `ȩ́` counted as 2 code units
cafȩ́	5	The character `ȩ` with combining mark `́` counted as 2 code units
cafȩ́	6	The character `e` with combining marks `́` and `̧` counted as 3 code units
😹	2	The cat emoji counted as 2 code units
👩🏻‍🚀	7	The astronaut emoji with gender and skin modifiers counted as 7 code units

Similarly when counting words, "my mother-in-law" is now counted as 4 (not 2) words to correctly follow the Unicode Default Word Boundary Specification.

To enable improved counting in supported browsers users can:

add countType: "characters" to count user-perceived characters (or graphemes)
add countType: "words" to count words between word boundaries

Where the default is countType: "length" to continue counting code points.

Unsupported browsers will default to the textareaDescriptionText message shown when JS is unavailable:

You can enter up to 350 characters

Using Nunjucks to count characters

Adding countType: "characters" to count using Intl.Segmenter

  {{ govukCharacterCount({
    label: {
      text: "Can you provide more detail?",
      size: "l",
      isPageHeading: true
    },
    name: "more-detail",
-   maxlength: 350
+   maxlength: 350,
+   countType: "characters"
  }) }}

Using Nunjucks to count words

Adding countType: "words" to count using Intl.Segmenter

  {{ govukCharacterCount({
    label: {
      text: "Can you provide more detail?",
      size: "l",
      isPageHeading: true
    },
    name: "more-detail",
-   maxwords: 150
+   maxlength: 150,
+   countType: "words"
  }) }}

Note: The character count maxwords option and word counting behaviour are deprecated and can be removed in a future release. Users should replace maxwords with maxlength when using countType: "words".

Test coverage

I've skipped on tests until you're happy with the proposal (and comments) in:

Character count's character/word count functions should be customisable #1364 (comment)

This feature was lifted from NHS.UK frontend so we can bring test coverage over from:

With some related configuration changes split out into:

[v10] Add support for functions in component configs nhsuk/nhsuk-frontend#1896

A previous version of this PR supported the countFunction option with my notes preserved below

It's now been removed whilst the team discuss potentially overloading countType instead

Customising the count function

I've also added an extra countFunction option so service teams can cater for server-side differences in:

New lines that vary due to \n versus \r\n
Word counts that vary based on empty space and punctuation
How empty space is trimmed before counting
Support for multi-byte strings

For example, services might already count multi-byte strings server-side (e.g. len() in Python) resulting in client-side count mismatches, yet support for improved character count counting may be blocked by a 3rd party library integration.

Custom count functions are called with:

text (string) - Textarea value
context (object) - Character count context

new CharacterCount($root, {
  maxlength: 350,
  countType: 'characters',
  countFunction(text, context) {
    return text.length
  }
})

Character count context objects contain the following properties:

config - Character count config
segmenter - Character count Intl.Segmenter (optional)

With the built in count functions available to call or extend via:

CharacterCount.countFunctions.length
CharacterCount.countFunctions.characters
CharacterCount.countFunctions.words

Determining the count type

I investigated a hybrid countFunction: "characters" option with (optional) count function overload.

But the character count relies on maxlength or maxwords to determine countType internally:

Overloading countFunction: "characters" with a function prevents this.

i.e. Given countFunction: (text, context) => {} how do we determine the count type?

With backwards compatibilty in mind, knowledge of the count type is necessary to pick between:

i18n.charactersUnderLimit
i18n.charactersAtLimit
i18n.charactersOverLimit
i18n.wordsUnderLimit
i18n.wordsAtLimit
i18n.wordsOverLimit

With the current component throwing a ConfigError when the count type cannot be determined:

'govuk-character-count: Either "maxlength" or "maxwords" must be provided'

Relatedly, neither GOV.UK Function schema nor Design System JSON parsing support multiple types.

At the moment, config schema types support single types (e.g. 'string') only:

static schema = Object.freeze({
  properties: {
    // … 
    countType: { type: 'string' },
    countFunction: { type: 'function' }

Although work to add a new type: 'function' property was straightforward, the investigation above shows why it's not possible to combine countType and countFunction into one.

Similarly for security reasons, it's very important to drop unsupported types, e.g. functions in data attributes

colinrotherham · 2026-04-29T07:16:37Z

+  /**
+   * @private
+   * @type {Intl.Segmenter | null}
+   */
+  segmenter = null


Not sure whether this should be @private?

For example this.segmenter can be accessed via the custom count function:

createAll(CharacterCount, { countFunction(text) { // this.segmenter } })

Saw that lingering comment that we didn't address. Given we're not having countFunction (and even in that situation, the segmenter being in the context) this should remain @private for now :)

36degrees

I haven't had a chance to do a full review, but my gut feeling is that we should avoid scenarios where the character count can give a different result depending on what browser you're using.

That being the case, in browsers that do not support Intl.Segementer I think we should fall back to the no-JS behaviour, rather than using a regex.

colinrotherham · 2026-05-06T10:36:30Z

Thanks @36degrees

Did you have any thoughts on the new countType Nunjucks option?

I haven't had a chance to do a full review, but my gut feeling is that we should avoid scenarios where the character count can give a different result depending on what browser you're using.

That makes sense, and means we can drop the fallback regexes too

For balance, there are some examples where browser differences are expected:

Internet Explorer 8 counted new lines twice (not supported in v5.x)
Plural rules are not supported in Safari 10.3–12.5 etc

But for the latter issue polyfill weight was involved

That being the case, in browsers that do not support Intl.Segementer I think we should fall back to the no-JS behaviour, rather than using a regex.

We're happy with this though and I can update the PR

colinrotherham · 2026-05-11T14:19:53Z

Pushed an update to do this:

That being the case, in browsers that do not support Intl.Segementer I think we should fall back to the no-JS behaviour, rather than using a regex.

Current options maxlength and maxwords work as usual for backwards compatibility
New options countType: "characters" or countType: "words" use Intl.Segmenter

romaricpascal

Cheers for proposing this @colinrotherham and the care around avoiding a breaking change 🙌🏻

Besides the comments about technical implementation, I'm concerned about a two things:

keeping maxlength as the source for the maximum for both characters may be a bit confusing, both to users used to it being used only for characters and when switching codebases using different versions of GOV.UK Frontend. I'd be keen to use a completely new option (say maximum) that'll be associated to the new countType to avoid confusion
only offering to count words the way Intl.Segmenter does with the countType option that the component is moving towards. I think we should check which way backends count words to make sure our default matches. It might be that we need two ways of counting: one counting like Intl.Segmenter and another only considering the whitespace as we do now, even if it does not match Unicode definition of a word.

Let me know what you think 😊

romaricpascal · 2026-05-12T17:43:09Z

-        this.count = text.match(/\S+/g)?.length ?? 0
-        break
-    }
+    this.count = this.countFunctions[countType].call(this, text)


suggestion Rather than executing the function like if it was a method of the CharacterCount, passing the segmenter as a second argument, inside an options object makes the boundary between the component and the count function clearer, allowing us to control what's exposed to the count function.

Suggested change

this.count = this.countFunctions[countType].call(this, text)

this.count = this.countFunctions[countType](text, {segmenter: this.segmenter})

Maybe we could use a getter to only instantiate the segmenter if the function accesses it, but that's more an optimisation than anything.

Sadly that means the custom countFunction would lose access to:

this.separator to split words yourself

~~this.segmenter to filter the segments yourself~~

this.$textarea to get the value yourself (e.g. trim, normalised line endings, row count etc)

Appreciate that all of these things are accessible anyway as @private isn't really private

Let me know if you'd like me to do anything

I think I'm fine witht the countFunction not having access to much at the start. We can always expand what we provide to the function in minor releases, but we can only restrict what the function receives in breaking releases if we went too far at the start.

Using an object as a second parameter would also clarify what this represents in the component's count functions (where you may think it's the countFunctions object where the functions are defined if you miss the typings).

Overall, if that's OK, I'd prefer we:

pass a second argument to the count function rather than use this (should have flagged that as an 'issue' rather than a 'suggestion')

restrict what the function receives to only the segmenter for now and expand as demand grows, keeping the separator in the function for counting words (thinking that long term, if we want people to manipulate texts before counting 'like the component does', we'd be better off exposing the countFunctions themselves rather than granular details of their implementation).

Hope that makes sense 😊

Instead of a second parameter I've set a custom (restricted) this and updated the types:

- this.count = countFunction.call(this, text) + this.count = countFunction.call({ segmenter: this.segmenter }, text)

Have a look at the diff for my last push to see this.separator has been removed too

To avoid recreating the count function context object every time, it could be persisted?

Either using .call() as this

// Limit access via `this` when calling the count function to prevent // unintended access to internal properties and methods - this.count = countFunction.call( - { - config: this.config, - segmenter: this.segmenter - }, - text - ) + this.count = countFunction.call(this.countFunctionContext, text)

Or as a 2nd param as you prefer:

// Limit access via `this` when calling the count function to prevent // unintended access to internal properties and methods - this.count = countFunction.call( - { - config: this.config, - segmenter: this.segmenter - }, - text - ) + this.count = countFunction(text, this.countFunctionContext)

Would definitely prefer a context as a parameter so that things are explicit rather than accessed through this, thanks. Feels more natural as second parameter. Not against caching it on the instance if you think that's an issue for performance to re-create it at each call.

Another thought that occured to me is that if this.config.maxwords is set, there is no this.segmenter, right? This means that the words function could branch on whether this.segmenter is definer rather than the value in the config. That would allow the public API for the context to be narrower.

Potentially, the characters function could work the same way for consistency. That would also clearly split which part of the component are responsible for what:

constructor decides whether to create a segmenter or not based on the config

count functions decide how to count based on whether they have a segmenter or not

How does that sound?

Sounds good on the 2nd parameter

I'm a little bit lost on the rest 😆

Another thought that occured to me is that if this.config.maxwords is set, there is no this.segmenter, right? This means that the words function could branch on whether this.segmenter is definer rather than the value in the config. That would allow the public API for the context to be narrower.

So this is what I did originally—I think?

Where branching on this.segmenter for countType: "words" gave different results by browser

But it differs to the feelings Ollie set in an earlier comment where he said:

I haven't had a chance to do a full review, but my gut feeling is that we should avoid scenarios where the character count can give a different result depending on what browser you're using.

That being the case, in browsers that do not support Intl.Segementer I think we should fall back to the no-JS behaviour, rather than using a regex.

So from this we've determined:

Users that set maxwords (deprecated) should the existing regex word count

Users that set countType: "words" should get segmenter word counting where supported

Users that set countType: "words" should get the no-JS behaviour where NOT supported

i.e. If you opt-in to use Intl.Segmenter then that's what you get (or the no-JS behaviour)

Hope that's still alright?

Regarding browser support

Knowing that Intl.Segmenter is in Baseline 2024 compare the following queries:

Audience coverage: 94.9 % for supports es6-module, versus

Audience coverage: 9.1 % for supports es6-module and not baseline 2024

Note: There sadly isn't a feature query intl-segmenter like there is for intl-pluralrules

Would definitely prefer a context as a parameter so that things are explicit rather than accessed through this, thanks. Feels more natural as second parameter. Not against caching it on the instance if you think that's an issue for performance to re-create it at each call.

✅ Done (pushed)

So this is what I did originally—I think?

Where branching on this.segmenter for countType: "words" gave different results by browser

But it differs to the feelings Ollie set in an earlier comment where he said:

I think I didn't explain well. Found it easier to attach a comment to the countFunctionContext to explain 😊

colinrotherham · 2026-05-12T19:12:37Z

Thanks @romaricpascal

You might have missed that word counting retains the current approach if maxwords is set 🙌

Regarding maximum versus using maxlength, wouldn't the latter mean zero changes are necessary should segmenter become the default in a future major release?

✅ GOV.UK Frontend v2.2.0+

The maxlength option always works

{{ govukCharacterCount({
  label: {
    text: "Always works"
  },
  name: "example",
  maxlength: 200
}) }}

colinrotherham · 2026-05-12T19:17:52Z

Do think about a future opt-out though, like segmenter: false?

Or having the word separator regex as an option? If set, bypassing the segmenter

Keen to lock in the API so we can release this on NHS.UK frontend

romaricpascal · 2026-05-14T12:07:33Z

Regarding maximum versus using maxlength, wouldn't the latter mean zero changes are necessary should segmenter become the default in a future major release?

That's a great point, hadn't thought of that. 🙌🏻

You might have missed that word counting retains the current approach if maxwords is set 🙌

My worry was for after we remove maxwords in the next major release (as it's being rightly deprecated in this PR).
Both your propositions of a separator and a segmenter: false opt-out would be a way to work around that, so I think that decision can be delayed until v7.0.0. Both may be useful as well:

separator to offer arbitrary splitting
segmenter: false to avoid creating segmenters unnecessarily if your countFunction does not need one

Keen to lock in the API so we can release this on NHS.UK frontend

Appreciate that'd reduce the divergence between both our Design Systems. However, we can't guarantee our responsiveness when looking at a topic we're not currently focusing on (like what happened for this PR), so please don't stay stuck because of us.

romaricpascal · 2026-05-21T16:44:43Z

+    this.countFunctionContext = {
+      config: this.config,
+      segmenter: this.segmenter
+    }


Having that context in place makes it easier to explain what I was on about with using the segmenter when counting words.

The idea is still to throw on line 148 if the Segmenter API is not available, not to fallback on the other way of counting when the API is not there.

Because we know the component will only keep initialising when a segmenter is needed if the Segmenter API is available, we can reduce the context to the following, keeping the initial public API narrower (less risk of breaking change in the future) and keeping all config related computations internal to the constructor.

Suggested change

this.countFunctionContext = {

config: this.config,

segmenter: this.segmenter

}

this.countFunctionContext = {

segmenter: this.segmenter

}

Then words can check if (this.segmenter) instead of if (this.config.maxwords).

Hope that makes more sense, aim is to control how much of a public API we offer at the start to avoid having to roll back on it down the line 😊

Thanks, glad that made sense

Hmm you might want to hold off hiding the config for a major breaking release though?

Users that provide countFunction will use the config to:

Determine whether the (deprecated) maxwords option is used

Determine whether they're counting "length", "characters" or "words"

Provide their own non-segmenter fallback based on config.countType

Especially when passing JavaScript configuration via initAll() or createAll() because a single application-wide countFunction will at least need to know the config.countType?

Without a config they can't provide their own fallbacks should the non-JS fallback be unsuitable

Here's an example of what a global count function might look like:

import { importAll, CharacterCount } from 'govuk-frontend' initAll({ characterCount: { /** * Custom global count function */ countFunction(text, context) { const { config, segmenter } = context // Defer to default word count if (config.countType === 'words') { return CharacterCount.countFunctions.words(text, context) } // Trim text before counting const trimmed = text.trim() // Restore support for older browsers if (!segmenter) { return trimmed.length } // Return custom count (using trimmed text) return Array.from(segmenter.segment(trimmed)).length } } })

…only

We must only use `characters` or `words` as the translation key prefix, regardless of the `countType` value

…Segmenter

colinrotherham · 2026-06-23T08:48:13Z

@romaricpascal I've updated this PR to remove the countFunction option

I've left in support for functions as component options in these commits:

With the changes to support a custom count function in a separate branch (see diff)

romaricpascal

Cheers for updating the PR removing the countFunction option and documenting what you did with it.

Code looks almost good to go, @NickColley and I just noticed that the component uses granularity: 'grapheme' when countType is words. It is a bit odd as we'd have expected word.

Is that how it's meant to be or something that slipped through the cracks when moving code around? 😊

romaricpascal · 2026-06-25T10:47:08Z

issue Should the ganularity be word when countType is words?

Sorry, I've had to change this so much it's in the removed commits. I'll bring it back:

this.segmenter = new Intl.Segmenter(this.i18n.locale, { granularity: this.config.countType === 'words' ? 'word' : 'grapheme' })

Don't forget the bit about Test coverage in the PR description 👀

I've held off porting them from NHS.UK frontend until you're happy with the implementation

It's fixed and pushed now

Cheers, figured it might have slipped through all the code move. Thanks for adding them back 😊

Re. test coverage, if you have extra tests around the countType option and deprecation of maxwords, we'd definitely be interested, as well as in a base CHANGELOG entry to kickstart the content process (we'll also review the docs of the new options).

Heya! Sorry, hadn't realised I could let you know which tests from the list in the description we could use 😓

I think if we grab the tests from these 3 PRs, we'll cover everything that's added here:

[v10] Deprecate character count maxwords and add countType option nhsuk/nhsuk-frontend#1893

[v10] Add character count countType: "words" option using Intl.Segmenter nhsuk/nhsuk-frontend#1899

[v10] Add character count countType: "characters" option using Intl.Segmenter nhsuk/nhsuk-frontend#1895

CHANGELOG wise, it would be this entry from NHS Frontend covering the use of the segmenter, wouldn't it?

Let me know if that's something you have space to add, otherwise I'm happy to add them to this branch if you're OK with me pushing on your branch 😊

I'd forgotten your character count tests still use page.$eval() instead of the helpers

Would you be happy if I applied changes from nhsuk/nhsuk-frontend#1888?

This diff in particular: nhsuk/nhsuk-frontend@ab87dcc#diff-b47de3c71ca81cc72b390d2216518ba095b58342579d6bd0b3490bbd637c0701

Our tests visit the review app but I'm happy to adapt that bit

Unless the (deprecated) `maxwords` option is used

…nter Unless the (deprecated) `maxwords` option is used

colinrotherham commented Apr 29, 2026

View reviewed changes

36degrees reviewed May 6, 2026

View reviewed changes

colinrotherham force-pushed the character-count-segmenter branch from 66b3afb to 405410f Compare May 11, 2026 14:11

This was referenced May 11, 2026

[v10] Add character count countFunction option nhsuk/nhsuk-frontend#1897

Merged

[v10] Add character count countType: "words" option using Intl.Segmenter nhsuk/nhsuk-frontend#1899

Merged

romaricpascal reviewed May 12, 2026

View reviewed changes

colinrotherham force-pushed the character-count-segmenter branch 2 times, most recently from 579f202 to 29c44bf Compare May 14, 2026 11:42

colinrotherham force-pushed the character-count-segmenter branch from 29c44bf to b18ddd1 Compare May 14, 2026 13:31

romaricpascal reviewed May 21, 2026

View reviewed changes

colinrotherham force-pushed the character-count-segmenter branch from c24ac3d to b454c9c Compare June 1, 2026 06:58

romaricpascal linked an issue Jun 2, 2026 that may be closed by this pull request

Character count's character/word count functions should be customisable #1364

Open

colinrotherham changed the title ~~Add character count Intl.Segmenter support with customisable count function~~ Add character count Intl.Segmenter support Jun 23, 2026

colinrotherham added 8 commits June 23, 2026 09:37

Make sure unsupported config property types are dropped

d9079c5

Add support for functions in component configs

4e03cad

Update initAll() config types to use ComponentConfig generic

5162759

Update character count config override types to show they’re partial …

ecb6c70

…only

Fix missing this.config in config override function

9fc2caa

Fix typo

a445179

Default to characters translation key prefix

d551b45

We must only use `characters` or `words` as the translation key prefix, regardless of the `countType` value

Deprecate character count maxwords and add countType option

873970f

colinrotherham added 2 commits June 23, 2026 09:37

Update GOV.UK Frontend support error to accept custom message

60534e4

Add character count support for countType: "characters" using Intl.…

e36b9ac

…Segmenter

colinrotherham force-pushed the character-count-segmenter branch from b454c9c to 16c93b4 Compare June 23, 2026 08:40

romaricpascal reviewed Jun 25, 2026

View reviewed changes

colinrotherham added 3 commits June 25, 2026 11:58

Expose segmenter for countType: "words"

9ff8429

Unless the (deprecated) `maxwords` option is used

Add character count support for countType: "words" using Intl.Segme…

437b337

…nter Unless the (deprecated) `maxwords` option is used

Update comments

6161fa1

colinrotherham force-pushed the character-count-segmenter branch from 16c93b4 to 6161fa1 Compare June 25, 2026 11:01

	this.count = this.countFunctions[countType].call(this, text)
	this.count = this.countFunctions[countType](text, {segmenter: this.segmenter})

Uh oh!

Conversation

colinrotherham commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Improved character count counting

Using Nunjucks to count characters

Using Nunjucks to count words

Test coverage

Customising the count function

Determining the count type

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

36degrees left a comment

Choose a reason for hiding this comment

Uh oh!

colinrotherham commented May 6, 2026

Uh oh!

colinrotherham commented May 11, 2026

Uh oh!

romaricpascal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

colinrotherham May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

colinrotherham May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Regarding browser support

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

colinrotherham commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ GOV.UK Frontend v2.2.0+

Uh oh!

colinrotherham commented May 12, 2026

Uh oh!

romaricpascal commented May 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

colinrotherham Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

colinrotherham commented Jun 23, 2026

Uh oh!

romaricpascal left a comment

Choose a reason for hiding this comment

Uh oh!

colinrotherham commented Apr 28, 2026 •

edited

Loading

colinrotherham May 13, 2026 •

edited

Loading

colinrotherham May 21, 2026 •

edited

Loading

colinrotherham commented May 12, 2026 •

edited

Loading

colinrotherham Jun 4, 2026 •

edited

Loading