Add Migration.collated/2 for building collated indexes with index/3

Author: kipcole9

CHANGELOG.md

@@ -14,6 +14,6 @@ Initial release.
 
 * Locale-to-collation resolution via CLDR language matching, including BCP 47 `-u-co-` collation types and direct collation names.
 
-* `Localize.Ecto.Migration.create_collation/2` and `drop_collation/2` for creating ICU collations reversibly in migrations.
+* `Localize.Ecto.Migration.create_collation/2` and `drop_collation/2` for creating ICU collations reversibly in migrations, and `collated/2` for building collated indexes with `Ecto.Migration.index/3`.
 
 See the [README](https://hexdocs.pm/localize_ecto/readme.html) for full documentation.

README.md

@@ -60,13 +60,13 @@ If your server's set differs materially from the snapshot, pass your own list wi
 
 ## Performance considerations
 
-Linguistic comparison is more expensive than PostgreSQL's default byte-order comparison, and an `ORDER BY ... COLLATE` clause can only use an index that was created with the same collation. For hot queries, create an expression index with the collation you sort by:
+Linguistic comparison is more expensive than PostgreSQL's default byte-order comparison, and an `ORDER BY ... COLLATE` clause can only use an index that was created with the same collation. For hot queries, create an index with the collation you sort by using [Localize.Ecto.Migration.collated/2](https://hexdocs.pm/localize_ecto/Localize.Ecto.Migration.html#collated/2):
 
-```sql
-CREATE INDEX products_name_de ON products (name COLLATE "de-x-icu");
+```elixir
+create index("products", [collated(:name, "de")])
 ```
 
-Nondeterministic collations carry an additional performance penalty. See the [PostgreSQL collation documentation](https://www.postgresql.org/docs/current/collation.html) for the details of collation selection, index compatibility, and the trade-offs between providers.
+If a PostgreSQL upgrade links a newer ICU library whose collation data changed — uncommon, but it happens — PostgreSQL warns of a collation version mismatch and indexes built with that collation must be reindexed. Nondeterministic collations carry an additional performance penalty. See the [PostgreSQL collation documentation](https://www.postgresql.org/docs/current/collation.html) for the details of collation selection, index compatibility, and the trade-offs between providers.
 
 ## Guides
 

guides/collations_in_postgres.md

@@ -6,13 +6,21 @@ Collation is the set of rules that determines how text sorts and compares. This
 
 Every PostgreSQL database has a default collation, fixed at `CREATE DATABASE` time, that applies to every text comparison and sort that does not name a collation explicitly. It also determines the behavior of case conversion (`upper`, `lower`, `ILIKE`) through its character-classification (ctype) side.
 
-Our recommendation is to keep the database default simple and stable, and to apply linguistic collation explicitly in queries with `COLLATE` — which is exactly what this library does. Two defaults fit that recommendation:
+Our recommendation: make the database default the builtin `C.UTF-8` locale (PostgreSQL 17 and later), falling back to the `C` locale on older PostgreSQL releases, and apply linguistic collation explicitly in queries with `COLLATE` — which is exactly what this library does.
 
-* The `C` locale. Sorting is plain byte order: fast, immutable across operating system and library upgrades, and free of surprises in indexes. The trade-off is that its ctype is ASCII-only, so `upper('öl')` returns `öL` — the `ö` is untouched. If you use case conversion on non-ASCII text, apply a collation to the expression (`upper('öl' COLLATE "de-x-icu")` returns `ÖL`) or prefer the next option.
+* `C.UTF-8` (provider `builtin`, PostgreSQL 17+) sorts in Unicode code-point order — fast and permanently stable — while its ctype is full Unicode, so case conversion (`upper`, `lower`, `ILIKE`) works for all scripts without naming a collation.
 
-* The builtin `C.UTF-8` locale (PostgreSQL 17 and later, provider `builtin`). Sorting is Unicode code-point order — equally fast and stable — while ctype is full Unicode, so case conversion works for all scripts without an explicit collation.
+* `C` is the fallback for PostgreSQL 16 and earlier. Sorting is plain byte order, equally fast and stable, but its ctype is ASCII-only: `upper('öl')` returns `öL` with the `ö` untouched. Where you need case conversion on non-ASCII text, apply a collation to the expression — `upper('öl' COLLATE "de-x-icu")` returns `ÖL`.
 
-A word on why stability matters: a database default drawn from an operating system locale (the libc provider, for example `en_US.UTF-8`) changes behavior when the operating system's locale data changes. Because indexes are built in collation order, a changed sort order can silently corrupt index correctness after an OS upgrade. Byte order and code-point order never change.
+The reason for this recommendation is stability. A database default is fixed at `CREATE DATABASE` time, every index on text is built in its sort order, and a sort order that changes underneath an existing index silently corrupts the index's correctness. Each external collation provider carries exactly that risk:
+
+* Operating system releases. A libc default such as `en_US.UTF-8` sorts according to the OS locale data, which changes with OS upgrades — the classic cause of index corruption after a glibc update.
+
+* ICU releases. An ICU default would tie the database's sort order to the ICU library version, which changes as CLDR data evolves; PostgreSQL records collation versions and warns of mismatches, but the remedy is still reindexing the affected database.
+
+* PostgreSQL releases. Byte order and code-point order are defined by Unicode itself, not by any library's tailoring data, so a `C` or `C.UTF-8` default behaves identically across PostgreSQL upgrades and never demands a reindex.
+
+Scoping linguistic collation to query expressions confines the versioned, changeable part of collation to the places that opt into it — and any index created with an explicit ICU collation (see `Localize.Ecto.Migration.collated/2`) is a known, listed object that can be reindexed deliberately when the ICU version moves.
 
 Neither `C` nor `C.UTF-8` sorts linguistically — `Zebra` sorts before `apple` because `Z` has a smaller code point than `a`. That is the point of the division of labor: the default collation keeps storage and indexes fast and stable, and queries opt into linguistic ordering per expression:
 

guides/using_localize_ecto.md

@@ -133,11 +133,16 @@ Localize.Ecto.Collation.collation_for!("de-AT", available: locales)
 
 ## Indexes
 
-An `ORDER BY ... COLLATE` clause only uses an index created with the same collation. If a collated sort is on a hot path, add a matching expression index in a migration:
+An `ORDER BY ... COLLATE` clause only uses an index created with the same collation. If a collated sort is on a hot path, add a matching index in a migration with `Localize.Ecto.Migration.collated/2`, which builds a collated column expression for `Ecto.Migration.index/3`:
 
 ```elixir
-execute(
-  ~s[CREATE INDEX products_name_de ON products (name COLLATE "de-x-icu")],
-  ~s[DROP INDEX products_name_de]
-)
+import Localize.Ecto.Migration
+
+create index("products", [collated(:name, "de")], name: :products_name_de)
+
+create index("products", [collated(:name, collation: "german_phonebook")], name: :products_name_phonebook)
 ```
+
+With such an index in place, `order_by: collate(p.name, "de")` is satisfied directly by an index scan with no sort step. All of `index/3`'s options compose as usual — `unique: true`, `concurrently: true`, `where:` and so on. Passing an explicit `:name` is recommended, since Ecto derives index names poorly from expression columns.
+
+One maintenance note: ICU collation data occasionally changes between ICU releases. If a PostgreSQL upgrade links a newer ICU, PostgreSQL warns of a collation version mismatch, and indexes built with that collation should be rebuilt — `REINDEX INDEX products_name_de` followed by `ALTER COLLATION "de-x-icu" REFRESH VERSION` clears the warning. This is uncommon, and it is the trade-off accepted by scoping linguistic collation to explicit expressions rather than the database default.

lib/localize/ecto/migration.ex

@@ -190,6 +190,63 @@ if Code.ensure_loaded?(Ecto.Migration) do
       "DROP COLLATION #{if_exists}#{quote_name(name, options[:schema])}"
     end
 
+    @doc """
+    Returns a collated column expression for use in an index definition.
+
+    PostgreSQL only uses an index for a collated sort or comparison when
+    the index was created with the same collation. This function builds
+    the column expression for such an index, for use with
+    `Ecto.Migration.index/3`:
+
+        create index("products", [collated(:name, "de")])
+
+        create index("products", [collated(:name, collation: "german_phonebook")])
+
+    Note that if a PostgreSQL upgrade links a newer ICU library whose
+    collation data changed — uncommon, but it happens — PostgreSQL warns
+    of a collation version mismatch and indexes built with that
+    collation must be reindexed (`REINDEX INDEX index_name`, then
+    `ALTER COLLATION collation_name REFRESH VERSION`).
+
+    ### Arguments
+
+    * `column` is the column name as an atom or string.
+
+    * `locale_or_options` is a locale accepted by
+      `Localize.Ecto.Collation.collation_for!/2`, or a keyword list with
+      a `:collation` option naming a collation directly. The default is
+      the current locale from `Localize.get_locale/0`.
+
+    ### Returns
+
+    * A column expression string such as `"name" COLLATE "de-x-icu"`.
+
+    ### Examples
+
+        iex> Localize.Ecto.Migration.collated(:name, "de")
+        ~s["name" COLLATE "de-x-icu"]
+
+        iex> Localize.Ecto.Migration.collated(:name, collation: "german_phonebook")
+        ~s["name" COLLATE "german_phonebook"]
+
+    """
+    @spec collated(atom() | String.t(), Localize.locale() | String.t() | Keyword.t()) ::
+            String.t()
+    def collated(column, locale_or_options \\ Localize.get_locale()) do
+      column = to_string(column)
+      collation = Collation.resolve!(locale_or_options)
+
+      if String.contains?(column, ~s(")) do
+        raise ArgumentError, "column name #{inspect(column)} contains a double quote"
+      end
+
+      if String.contains?(collation, ~s(")) do
+        raise ArgumentError, "collation name #{inspect(collation)} contains a double quote"
+      end
+
+      ~s("#{column}" COLLATE "#{collation}")
+    end
+
     defp collation_sql_pair(locale, options) do
       {create_collation_sql(locale, options), drop_collation_sql(locale, options)}
     end

test/localize/ecto/migration_test.exs

@@ -59,6 +59,43 @@ defmodule Localize.Ecto.MigrationTest do
     end
   end
 
+  describe "collated/2" do
+    test "builds a collated column expression from a locale" do
+      assert Migration.collated(:name, "de") == ~s["name" COLLATE "de-x-icu"]
+
+      assert Migration.collated("name", "de-u-co-phonebk") ==
+               ~s["name" COLLATE "de-u-co-phonebk-x-icu"]
+    end
+
+    test "builds a collated column expression from a collation name" do
+      assert Migration.collated(:name, collation: "german_phonebook") ==
+               ~s["name" COLLATE "german_phonebook"]
+    end
+
+    test "defaults to the current locale" do
+      {:ok, _} = Localize.put_locale("sv")
+      assert Migration.collated(:name) == ~s["name" COLLATE "sv-x-icu"]
+    after
+      Localize.put_locale("en")
+    end
+
+    test "raises for an invalid locale" do
+      assert_raise Localize.InvalidLocaleError, fn ->
+        Migration.collated(:name, "zzzz")
+      end
+    end
+
+    test "rejects identifiers containing double quotes" do
+      assert_raise ArgumentError, fn ->
+        Migration.collated(~s(bad"name), "de")
+      end
+
+      assert_raise ArgumentError, fn ->
+        Migration.collated(:name, collation: ~s(bad"collation))
+      end
+    end
+  end
+
   describe "drop_collation_sql/2" do
     test "drops by the resolver-symmetric name" do
       assert Migration.drop_collation_sql("de-u-co-phonebk") ==