You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,22 @@ All notable changes to this project will be documented in this file.
4
4
5
5
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
7
+
## [Unreleased]
8
+
9
+
Renamed from `localize_ecto` to **`localize_sql`** — the package is about SQL databases (collations, serialization, migrations), not Ecto in general, which can be used without a database. The `Localize.Ecto.*` module namespace and the `mix localize.ecto.audit` task are unchanged; the app, config key (`:localize_sql`), env var (`LOCALIZE_SQL_SQLITE_ICU`) and `priv/localize_sql/` path are renamed.
10
+
11
+
`localize_sql` grows from locale-aware collation into the SQL home for every Localize type — serializing the full set of Localize and Elixir data types and generating the tagged-decimal aggregate machinery `ex_money_sql` builds on.
12
+
13
+
### Added
14
+
15
+
*`Localize.Ecto.TaggedDecimal` and its `DDL` generate a PostgreSQL composite type with tag-guarded `sum`/`avg`/`min`/`max` aggregates and `+`/`-` operators for any tag-plus-decimal value; a mismatched tag (euros plus yen) raises in the database. `ex_money_sql` now builds `money_with_currency` on this rather than its own SQL.
16
+
17
+
* Unit serialization, folded in from `localize_units_sql`: `Localize.Unit.Ecto.{Composite,Map}.Type` and the `Localize.UnitWithUsage` variants, `Localize.Unit.DDL` (now `sum`/`min`/`max`/`avg` for `cldr_unit` and `cldr_unit_with_usage`), and `mix localize.unit.gen.migration`.
18
+
19
+
* Ecto types for the remaining data types: `Localize.Currency.Ecto.Type`, `Localize.LanguageTag.Ecto.Type`, `Localize.Territory.Ecto.Type` and `Localize.Script.Ecto.Type` (text); `Localize.Ecto.Type.IntegerRange` and `.DateRange` (`int8range`/`daterange`); `Localize.Duration.Ecto.Type` and `Localize.Ecto.Type.Duration` (`interval`).
20
+
21
+
*`Localize.Ecto.Migration.Generator` renders and writes migrations for the SQL-generating mix tasks.
22
+
7
23
## [1.0.0-rc.0] - 2026-07-28
8
24
9
25
The 1.0 release candidate. `localize_ecto` now supports SQLite as well as PostgreSQL, with the same locale resolving to the same collation on both.
Copy file name to clipboardExpand all lines: README.md
+65-19Lines changed: 65 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,8 @@
1
-
# Localize Ecto
1
+
# Localize SQL
2
2
3
-
Locale-aware ICU collation for [Ecto](https://hexdocs.pm/ecto) queries on PostgreSQL and SQLite. `localize_ecto` resolves a [Localize](https://hexdocs.pm/localize) language tag to the best matching ICU collation and applies it with a `COLLATE` clause, so query results sort and compare according to the conventions of the user's locale.
3
+
SQL database support for [Localize](https://hexdocs.pm/localize) and [Ecto](https://hexdocs.pm/ecto), in two parts: **locale-aware ICU collation** for query ordering, and **Ecto types** that store every Localize and relevant Elixir data type — money and units of measure with database-side aggregates, currencies, locales, territories, ranges, durations and more.
4
+
5
+
The first part resolves a Localize language tag to the best matching ICU collation and applies it with a `COLLATE` clause, so query results sort and compare according to the conventions of the user's locale:
4
6
5
7
```elixir
6
8
importEcto.Query
@@ -19,9 +21,9 @@ from p in Product, where: collate(p.name < "münchen", "de")
19
21
from p inProduct, order_by:collate(p.name, collation:"german_phonebook")
20
22
```
21
23
22
-
The examples above are PostgreSQL. `Localize.Ecto` exposes the PostgreSQL macros directly; for SQLite, import [Localize.Ecto.SQLite3](https://hexdocs.pm/localize_ecto/Localize.Ecto.SQLite3.html) instead — see [SQLite](#sqlite) below.
24
+
The examples above are PostgreSQL. `Localize.Ecto` exposes the PostgreSQL macros directly; for SQLite, import [Localize.Ecto.SQLite3](https://hexdocs.pm/localize_sql/Localize.Ecto.SQLite3.html) instead — see [SQLite](#sqlite) below.
23
25
24
-
Locale resolution uses the [CLDR Language Matching](https://www.unicode.org/reports/tr35/tr35.html#LanguageMatching) algorithm, so any valid locale finds its best available collation — `"zh-TW"` resolves to `zh-Hant-x-icu`, `"de-DE"` to `de-x-icu`, and an unknown locale falls back gracefully. Locales that carry a BCP 47 collation type, such as `de-u-co-phonebk` (German phonebook order), resolve to collations you create once in a migration with [Localize.Ecto.Migration.create_collation/2](https://hexdocs.pm/localize_ecto/Localize.Ecto.Migration.html#create_collation/2).
26
+
Locale resolution uses the [CLDR Language Matching](https://www.unicode.org/reports/tr35/tr35.html#LanguageMatching) algorithm, so any valid locale finds its best available collation — `"zh-TW"` resolves to `zh-Hant-x-icu`, `"de-DE"` to `de-x-icu`, and an unknown locale falls back gracefully. Locales that carry a BCP 47 collation type, such as `de-u-co-phonebk` (German phonebook order), resolve to collations you create once in a migration with [Localize.Ecto.Migration.create_collation/2](https://hexdocs.pm/localize_sql/Localize.Ecto.Migration.html#create_collation/2).
25
27
26
28
## Beyond ordering
27
29
@@ -48,7 +50,7 @@ field :time_zone, Localize.Ecto.Type.TimeZone
48
50
from e inEvent, select:at_time_zone(e.starts_at, "Australia/Sydney")
49
51
```
50
52
51
-
The [audit task](https://hexdocs.pm/localize_ecto/Mix.Tasks.Localize.Ecto.Audit.html) reports collation version drift after PostgreSQL/ICU upgrades — with the `REINDEX` and `ALTER COLLATION … REFRESH VERSION` remediation for every dependent index — and compares the server's Unicode, ICU and time zone inventories against the application's:
53
+
The [audit task](https://hexdocs.pm/localize_sql/Mix.Tasks.Localize.Ecto.Audit.html) reports collation version drift after PostgreSQL/ICU upgrades — with the `REINDEX` and `ALTER COLLATION … REFRESH VERSION` remediation for every dependent index — and compares the server's Unicode, ICU and time zone inventories against the application's:
52
54
53
55
```console
54
56
$ mix localize.ecto.audit
@@ -60,34 +62,78 @@ Audit for MyApp.Repo
60
62
Time zones: all application zones known to the server (100 server-only zones)
61
63
```
62
64
65
+
## Serializing Localize types
66
+
67
+
The second part is an `Ecto.Type` for every Localize and relevant Elixir data type, so a locale, a currency, a unit of measure or a range is a first-class value in a schema. Most store as ordinary text or `jsonb` and need no migration beyond the column:
68
+
69
+
```elixir
70
+
schema "listings"do
71
+
field :locale, Localize.LanguageTag.Ecto.Type# text, e.g. "en-GB"
72
+
field :currency, Localize.Currency.Ecto.Type# text, e.g. "USD"
73
+
field :country, Localize.Territory.Ecto.Type# text, e.g. "US"
74
+
field :levels, Localize.Ecto.Type.IntegerRange# int8range, e.g. 1..10
75
+
field :period, Localize.Ecto.Type.DateRange# daterange
76
+
field :retention, Localize.Ecto.Type.Duration# interval
77
+
end
78
+
```
79
+
80
+
A value cast from user input is validated and canonicalized on the way in — an unknown currency or an invalid locale is rejected, and `"en_US"` becomes the resolved `"en-US"` language tag — and loaded back as the rich Localize value rather than the raw string.
81
+
82
+
### Money and units: database-side aggregates
83
+
84
+
A money amount or a unit of measure is a *tagged decimal* — a decimal paired with a tag (a currency code, a unit name) that is meaningless without it. `localize_sql` stores these as a PostgreSQL composite type and generates tag-guarded `sum`, `avg`, `min` and `max` aggregates and `+`/`-` operators, so the database itself refuses to add euros to yen or metres to feet:
The same machinery, [Localize.Ecto.TaggedDecimal](https://hexdocs.pm/localize_sql/Localize.Ecto.TaggedDecimal.html), backs [ex_money_sql](https://hexdocs.pm/ex_money_sql), which builds its `money_with_currency` type and aggregates on `localize_sql` rather than defining its own.
96
+
97
+
| Type | Ecto type | Column |
98
+
|---|---|---|
99
+
| Currency |`Localize.Currency.Ecto.Type`| text |
100
+
| Language tag |`Localize.LanguageTag.Ecto.Type`| text |
101
+
| Territory / Script |`Localize.Territory.Ecto.Type` / `Localize.Script.Ecto.Type`| text |
102
+
| Time zone |`Localize.Ecto.Type.TimeZone`| text |
103
+
| Unit of measure |`Localize.Unit.Ecto.Composite.Type` (also `Map`, and the `WithUsage` variants) | composite / jsonb |
104
+
| Integer / date range |`Localize.Ecto.Type.IntegerRange` / `.DateRange`| int8range / daterange |
105
+
| Duration |`Localize.Duration.Ecto.Type` and `Localize.Ecto.Type.Duration`| interval |
106
+
107
+
The composite, range and duration types map to PostgreSQL types (via `postgrex`) and light up only when it is present; the text and `jsonb` types work on any Ecto adapter.
108
+
63
109
## Installation
64
110
65
-
Add `localize_ecto` to your dependencies:
111
+
Add `localize_sql` to your dependencies:
66
112
67
113
```elixir
68
114
defdepsdo
69
115
[
70
-
{:localize_ecto, "~> 1.0-rc"}
116
+
{:localize_sql, "~> 1.0-rc"}
71
117
]
72
118
end
73
119
```
74
120
75
121
## SQLite
76
122
77
-
SQLite has no ICU collations of its own, so `localize_ecto` ships one: `localize_icu`, a SQLite loadable extension that registers ICU collations under the same names PostgreSQL uses. The same locale resolves to the same collation name and produces the same ordering on either database, so a query ported between them sorts identically — the test suite asserts this for a range of languages and tailorings.
123
+
SQLite has no ICU collations of its own, so `localize_sql` ships one: `localize_icu`, a SQLite loadable extension that registers ICU collations under the same names PostgreSQL uses. The same locale resolves to the same collation name and produces the same ordering on either database, so a query ported between them sorts identically — the test suite asserts this for a range of languages and tailorings.
78
124
79
125
Install ICU (`brew install icu4c`, or `apt-get install libicu-dev`), enable the build in `config/config.exs`, and load the extension on every connection:
80
126
81
127
```elixir
82
128
# config/config.exs — read at compile time, so not runtime.exs
Then import [Localize.Ecto.SQLite3](https://hexdocs.pm/localize_ecto/Localize.Ecto.SQLite3.html) in place of `Localize.Ecto`:
136
+
Then import [Localize.Ecto.SQLite3](https://hexdocs.pm/localize_sql/Localize.Ecto.SQLite3.html) in place of `Localize.Ecto`:
91
137
92
138
```elixir
93
139
importEcto.Query
@@ -99,15 +145,15 @@ from p in Product, order_by: collate(p.name, "sv")
99
145
from p inProduct, order_by:collate(p.name, "de-u-co-phonebk")
100
146
```
101
147
102
-
Collations are built the first time a query names one, so unlike PostgreSQL the BCP 47 tailorings — `-u-co-phonebk`, `-u-kn-true`, `-u-ks-level1` — need no `CREATE COLLATION` migration. The build is opt-in: without it `localize_ecto` stays a pure-Elixir package needing no C toolchain or ICU, and PostgreSQL users are unaffected.
148
+
Collations are built the first time a query names one, so unlike PostgreSQL the BCP 47 tailorings — `-u-co-phonebk`, `-u-kn-true`, `-u-ks-level1` — need no `CREATE COLLATION` migration. The build is opt-in: without it `localize_sql` stays a pure-Elixir package needing no C toolchain or ICU, and PostgreSQL users are unaffected.
103
149
104
-
One consequence to weigh before indexing with an ICU collation: an index that names a collation makes the database unreadable by any connection that has not loaded the extension, including the `sqlite3` CLI. The [Collations in SQLite](https://hexdocs.pm/localize_ecto/collations_in_sqlite.html) guide covers this, the case mapping differences, and what does not port from PostgreSQL.
150
+
One consequence to weigh before indexing with an ICU collation: an index that names a collation makes the database unreadable by any connection that has not loaded the extension, including the `sqlite3` CLI. The [Collations in SQLite](https://hexdocs.pm/localize_sql/collations_in_sqlite.html) guide covers this, the case mapping differences, and what does not port from PostgreSQL.
105
151
106
152
## Deterministic collations and Unicode normalization
107
153
108
154
On PostgreSQL, the collations this library resolves to are deterministic, which is PostgreSQL's default. SQLite has no such concept and behaves differently here — see the end of this section. A deterministic collation never treats two strings as equal unless they are byte-for-byte identical: comparison first uses the linguistic collation order, then breaks ties bytewise. This has a practical consequence for Unicode text that is not normalized. Canonically equivalent strings in different normalization forms — for example `é` as the single code point U+00E9 versus `e` followed by combining U+0301 — will sort adjacently but will never compare equal, so equality tests, `DISTINCT`, `GROUP BY`, joins on text keys, and unique indexes all see them as different values.
109
155
110
-
To get expected results, normalize text (NFC is the usual choice) before writing it to the database. In Elixir use [String.normalize/2](https://hexdocs.pm/elixir/String.html#normalize/2); in PostgreSQL the [normalize](https://www.postgresql.org/docs/current/functions-string.html) function and `IS NFC NORMALIZED` predicate are available for checking or repairing existing data. Alternatively, PostgreSQL supports nondeterministic collations that do compare canonically equivalent strings as equal — [Localize.Ecto.Migration.create_collation/2](https://hexdocs.pm/localize_ecto/Localize.Ecto.Migration.html#create_collation/2) can create one with `deterministic: false` — but they cannot be used with `LIKE` or pattern matching and are slower.
156
+
To get expected results, normalize text (NFC is the usual choice) before writing it to the database. In Elixir use [String.normalize/2](https://hexdocs.pm/elixir/String.html#normalize/2); in PostgreSQL the [normalize](https://www.postgresql.org/docs/current/functions-string.html) function and `IS NFC NORMALIZED` predicate are available for checking or repairing existing data. Alternatively, PostgreSQL supports nondeterministic collations that do compare canonically equivalent strings as equal — [Localize.Ecto.Migration.create_collation/2](https://hexdocs.pm/localize_sql/Localize.Ecto.Migration.html#create_collation/2) can create one with `deterministic: false` — but they cannot be used with `LIKE` or pattern matching and are slower.
111
157
112
158
SQLite behaves as PostgreSQL's nondeterministic collations do, and it is the one place the two databases disagree. SQLite has no deterministic/nondeterministic distinction, so a comparison returns exactly what ICU says — and ICU treats canonically equivalent strings as equal. The NFC and NFD forms of `café` compare **equal** on SQLite and **unequal** on PostgreSQL. Ordering is unaffected on both, which is why sorting still agrees; it is equality, `DISTINCT`, `GROUP BY` and unique indexes that differ. Normalizing on write makes the question moot on either database, and is worth doing regardless.
113
159
@@ -124,11 +170,11 @@ WHERE collprovider = 'i'
124
170
ORDER BY collname;
125
171
```
126
172
127
-
If your server's set differs materially from the snapshot, pass your own list with the `:available` option of [Localize.Ecto.Collation.collation_for/2](https://hexdocs.pm/localize_ecto/Localize.Ecto.Collation.html#collation_for/2).
173
+
If your server's set differs materially from the snapshot, pass your own list with the `:available` option of [Localize.Ecto.Collation.collation_for/2](https://hexdocs.pm/localize_sql/Localize.Ecto.Collation.html#collation_for/2).
128
174
129
175
## Performance considerations
130
176
131
-
Linguistic comparison is more expensive than PostgreSQL's default byte-order comparison, and an `ORDER BY ... COLLATE` clause can only use an index that was created with the same collation. For hot queries, create an index with the collation you sort by using [Localize.Ecto.Migration.collated/2](https://hexdocs.pm/localize_ecto/Localize.Ecto.Migration.html#collated/2):
177
+
Linguistic comparison is more expensive than PostgreSQL's default byte-order comparison, and an `ORDER BY ... COLLATE` clause can only use an index that was created with the same collation. For hot queries, create an index with the collation you sort by using [Localize.Ecto.Migration.collated/2](https://hexdocs.pm/localize_sql/Localize.Ecto.Migration.html#collated/2):
132
178
133
179
```elixir
134
180
create index("products", [collated(:name, "de")])
@@ -138,14 +184,14 @@ If a PostgreSQL upgrade links a newer ICU library whose collation data changed
138
184
139
185
## Guides
140
186
141
-
*[Using Localize Ecto](https://hexdocs.pm/localize_ecto/using_localize_ecto.html) — the `collate/1,2` macros, locale resolution, and migrations.
187
+
*[Using Localize SQL](https://hexdocs.pm/localize_sql/using_localize_sql.html) — the `collate/1,2` macros, locale resolution, and migrations.
142
188
143
-
*[Collations in PostgreSQL](https://hexdocs.pm/localize_ecto/collations_in_postgres.html) — how PostgreSQL collation works, choosing a database default, and how ICU collations relate to Localize.
189
+
*[Collations in PostgreSQL](https://hexdocs.pm/localize_sql/collations_in_postgres.html) — how PostgreSQL collation works, choosing a database default, and how ICU collations relate to Localize.
144
190
145
-
*[Collations in SQLite](https://hexdocs.pm/localize_ecto/collations_in_sqlite.html) — the `localize_icu` extension, on-demand collation registration, and the trade-offs of indexing with an ICU collation.
191
+
*[Collations in SQLite](https://hexdocs.pm/localize_sql/collations_in_sqlite.html) — the `localize_icu` extension, on-demand collation registration, and the trade-offs of indexing with an ICU collation.
146
192
147
193
## License
148
194
149
195
Copyright 2026 Kip Cole
150
196
151
-
Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/elixir-localize/localize_ecto/blob/v1.0.0-rc.0/LICENSE.md).
197
+
Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/elixir-localize/localize_sql/blob/v1.0.0-rc.0/LICENSE.md).
0 commit comments