2.11.16

🏆 Main features

💎 Language support: Introducing dictionary based stemming

For handling in search the singular and plural form of nouns or conjugated versus infinitive form of verbs, Elasticsuite relies on the Elasticsearch/OpenSearch algorithmic stemming feature that support a wide variety of languages, but not all of them.

For languages not supported, we've introducing in this release in Elasticsuite Premium the ability to use Elasticsearch's / OpenSearch's dictionary based stemming feature that relies on Hunspell dictionaries used by a wide variety of spellcheckers.

This can be configured at the Store scope in Stores > Configuration > Elasticsuite > Analyzers Settings > Dictionary based stemming.

💎 Search: Detect and transform unit symbols in quantity/size expressions to preserve the meaning

By default, when indexing content and searching, most of special (non alphanumerical) characters are removed at the beginning of the text analysis and those remaining are then used to delimit/break words and finally removed.
This can be problematic if your catalog data relies heavily on units of measure and/or if your users use their symbol when searching (like " for inches or ' for feet): out of the box, there is no difference in search between 3" screws and 3' screws.

This release introduces in Elasticsuite Premium a mechanism that allows you to define "unit detection rules" rules that will replace a unit of measure symbol or shorthand notation by its full name both at indexing and search time, preserving the sense of the original expression.
For instance, if you associate the character/sequences of character ", in and in. to inches, the expressions 3" and 3in and 3in. will all be transformed to 3 inches at indexing and search time.

The new settings located in Stores > Configuration > Elasticsuite > Analyzers Settings > Unit detection configuration offer a wide coverage of use cases (location of the symbol before or after the quantity, whether the unit name replacing the symbol should be put before or after the quantity, etc).

Autocomplete & 💎 InstantSearch : Ability to force taking categories/products manual/edited URL rewrites into account

By default, for the categories autocomplete, Elasticsuite will rely on the fact that redirect (301) URLs will always be generated when the url_key of a category changes.
Unfortunately, if you manually edit the URL rewrite of a category in admin (Marketing > SEO & Search > URL Rewrites), or if you use a third party module that generates them according to specific rules without putting in place redirect (301) URLs, you could end up with a category being shown in the autocomplete results with a link leading to a 404 page.
The same applies on Elasticsuite Premium for products in addition to categories.

We've decided to introduce settings in Stores > Configuration > Elasticsuite > Catalog Search > Explicit URL rewrites support that can force Elasticsuite to look into the URL rewrites table at indexing :

• Enabled for categories : which is set to Yes by default, since the overhead of looking for categories URL rewrites should be negligible to minimal
• 💎 Enabled for products : (available on Elasticsuite Premium) which is set to No by default since the overhead can be noticeable depending on your catalog size and the number of stores you have.

Please be aware that at this time, there is no automated mechanism that flags a category or a product for being re-indexed in the search engine when an URL rewrite that targets it is changed.

Thesaurus: ability to apply the synonyms rules without regard to singular/plural forms of words

A known limitation about the thesaurus feature is that synonyms and expansions rules are triggered only when the word searched by a user is present exactly as is in the rules.
For instance

• searching for "stories" would not trigger a "story,tale" synonym rule
• searching for "sports" would not trigger a "sport => football,tennis,soccer,running" expansion rule

The common way to deal with that is to include the extra singular/plural form of words in the rule, for instance

• changing the synonym rule from "story,tale" to "story,stories,tale,tales"
• changing the expansion rule from "sport => football,tennis,soccer,running" to "sport,sports => football,tennis,soccer,running"

This can be avoided starting from this release by switching to Yes the new setting in Stores > Configuration > Elasticsuite > Thesaurus (global settings) > Enable stemming on synonym and expansion rules.

It is not enabled by default because we already know of some limitations

• ignoring the singular/plural aspect of words will not work for multi-word synonym rules (e.g. "red dragon,blue vampire")
• in some languages, it can happen that "double stemming" occurs, losing some precision in alternative queries
  • in french "chaises" ("chair") will be stemmed into "chais" at the thesaurus time and possibly "chai" (stem for "cellar") at search time

But do not hesitate to try the feature and give us some much welcomed feedback.

📦 Features

• 💎 [Analysis] Detect quantity/size with units by @rbayet
• 💎 [Analysis] Preserve some special chars by @PierreGauthier
• 💎 [AdvancedAnalysis] Introduce Dictionary based stemming by @rbayet
• 💎 [AB Campaign] Add the number of uses and number of results of search term in the campaign optimizer form by @rbayet
• [Catalog] Accessibility adjustments by @saltomorales in #3688
• [Catalog] [Autocomplete] Take categories manual/edited URL rewrites into account by @rbayet in #3731
• [Core] Support custom scoring model/similarity at the attribute/field level by @rbayet in #3712
• 💎 [Explain] Document thesaurus stemming use by @rbayet
• 💎 [Explain] Add clarification about the weight in matches. Mentions the scoring model/algorithm. by @rbayet
• [Indices] Analysis: adding token type info by @rbayet in #3735
• 💎 [InstantSearch] Take categories manual/edited URL rewrites into account by @rbayet
• 💎 [InstantSearch] Take products manual/edited URL rewrites into account by @rbayet
• 💎 [Recommender][Healthcheck] Critical healthcheck for the similarity attribute(s) by @rbayet
• [Thesaurus] Allow using stemmed words to match rules by @rbayet in #3708

🧰 Technical

• [Analysis] Handle uppercase elision by @PierreGauthier in #3729
• 💎 [CascadeSearch] Analysis: add implicit word delimiter parameter by @rbayet
• [Catalog][Autocomplete] Pull category URL suffix from store scope by @rbayet in #3728
• 💎 [Core] Metapackages: clean alphabetical list of modules by @rbayet
• [Core] Prevent firebear/importexport messing up analysis config by @rbayet in #3696
• [Core] Analysis: add implicit word delimiter parameter by @rbayet in #3737
• [Core] Analysis: introducing constant and token filters for special characters handling by @rbayet in #3738
• [Core] Stemmer selection in admin, minor CSS tweak by @rbayet in #3702
• [Core] Fix for upcoming PHP 8.5 deprecation, casting to double is no longer allowed by @hostep in #3732
• 💎 [Hyva][Recommender] Recommender blocks (loader) to the correct location by @rbayet
• 💎 [HyvaCompatibility] CSP: register inline script tags by @rbayet
• [Indices] Escape analysis tokens by @rbayet in #3734
• 💎 [InstantSearch] Pull Category URL suffix from store scope config by @rbayet
• 💎 [InstantSearch] Pull Product URL suffix from store scope config by @rbayet
• [Optimizers/Catalog] UI component: reposition search terms' Uses/Popularity' by @rbayet in #3691
• [Optimizers] Conditions engine useable when Virtual Categories module disabled by @rbayet in #3721
• [Thesaurus] Max number of rewritten queries to 50 by @rbayet in #3711
• [Thesaurus] #3713 Force leniency when creating index by @rbayet in #3716
• [Thesaurus] Disabling by default stemming in thesaurus rules. by @rbayet in #3733

🐛 Fixes

• [CatalogGraphQl] Fixes #3643 functionally (ES searchContext update) properly by @rbayet in #3686
• [Catalog][GraphQL] Fix #3574, add customer group condition for price filtering by @vahonc in #3714
• [GraphQl] Fix Numeric Attributes filter type by @PierreGauthier in #3720
• [Core] Fix #3623, add supporting getMinPrice() and getMaxPrice() as Magento Core functionality by @PierreGauthier in #3692
• [Layer] Fix category in search context by @PierreGauthier in #3693
• [Optimizers] Fixes search terms num. results not initially shown after selection by @rbayet in #3687
• 💎 [Recommender] Fix invalid form key in add to cart form by @PierreGauthier
• 💎 [HyvaCompatibility][Recommender] Fixes 500 errors in recommender ajax endpoints by @rbayet

New Contributors

• @saltomorales made their first contribution in #3688

Full Changelog: 2.11.15.1...2.11.16