`localizedStandardContains` should match more liberally

[dabrahams]

Description

The general problem is described in this forum post.

Reproduction

let APOSTROPHE: Character = "'"
let RIGHT_SINGLE_QUOTATION_MARK: Character = "’"
let MODIFIER_LETTER_TURNED_COMMA: Character = "ʻ"
let ʻokina = MODIFIER_LETTER_TURNED_COMMA // Official glyph for Hawaiian ʻokina.

let MODIFIER_LETTER_SMALL_P: Character = "ᵖ"

assert("Joe\(RIGHT_SINGLE_QUOTATION_MARK)s".localizedStandardContains("Joe\(APOSTROPHE)s"))
assert("Joe\(MODIFIER_LETTER_TURNED_COMMA)s".localizedStandardContains("Joe\(APOSTROPHE)s"))
assert("\(ʻokina)uli\(ʻokina)uli".localizedStandardContains("\(APOSTROPHE)uli\(APOSTROPHE)uli"))
assert("\(MODIFIER_LETTER_SMALL_P)recise".localizedStandardContains("precise"))

Expected behavior

All assertions pass

Environment

swift-driver version: 1.127.15 Apple Swift version 6.2.4 (swiftlang-6.2.4.1.4 clang-1700.6.4.2)
Target: arm64-apple-macosx26.0

Additional information

No response

[xedin]

cc @jmschonfeld, should this be transferred to Foundation project?

[jmschonfeld]

Yeah the Foundation repos are likely the right place for this. Transferring to swift-corelibs-foundation since localizedStandardContains is not (yet) in swift-foundation

[parkera]

localizedStandardContains is a wrapper for using the string comparison options [.caseInsensitive, .diacriticInsensitive]. I don't believe an option exists to ignore things like curly vs straight quotes. I'm not sure any standard option should compare the example of MODIFIER_LETTER_SMALL_P with p, though.

[itingliu]

The .widthInsensitive option might help a bit

We should check the behavior of ICU's implementation though to see if it's intended or a bug in our own implementation

[dabrahams]

If you are committed to the idea that localizedStandardContains means these three options specifically, Foundation should provide some other check that is suitable for matching what users will actually type to similar looking text.