Skip to content

Commit 319f956

Browse files
authored
Merge branch 'main' into redo-pr
2 parents 5332f59 + 73bd66d commit 319f956

File tree

57 files changed

+4795
-274
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+4795
-274
lines changed

.github/workflows/elixir.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
strategy:
1616
fail-fast: false
1717
matrix:
18-
postgres-version: ["14", "15", "16"]
18+
postgres-version: ["14", "15", "16", "17", "18"]
1919
uses: ash-project/ash/.github/workflows/ash-ci.yml@main
2020
with:
2121
postgres: true

CHANGELOG.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,46 @@ See [Conventional Commits](https://www.conventionalcommits.org) for commit guide
1111

1212
<!-- changelog -->
1313

14+
## [v2.8.0](https://github.com/ash-project/ash_postgres/compare/v2.7.0...v2.8.0) (2026-03-09)
15+
16+
17+
18+
19+
### Features:
20+
21+
* add --use_fragments option to resource generator | Closes #437 (#709) by henryzhan013
22+
23+
### Bug Fixes:
24+
25+
* test setup by Philip Capel
26+
27+
* formatting by Philip Capel
28+
29+
## [v2.7.0](https://github.com/ash-project/ash_postgres/compare/v2.6.32...v2.7.0) (2026-03-05)
30+
31+
32+
33+
34+
### Features:
35+
36+
* support offset option in lateral join queries (#700) by Jinkyou Son
37+
38+
* support touch_update_defaults? option to skip update_default fields on upsert by Michael Bärtschi
39+
40+
### Bug Fixes:
41+
42+
* bulk_create with upsert now updates update_timestamp fields on conflict by Michael Bärtschi
43+
44+
* Fix locks handling for WAIT and SKIP_LOCKED (#704) by sezaru
45+
46+
* set size when type changes in migrations (Issue #150) (#694) by Jatanasio
47+
48+
* bulk_create with upsert now updates update_timestamp fields (#697) by Michael Bärtschi
49+
50+
### Improvements:
51+
52+
* read touch_update_defaults? from options instead of changeset context (#701) by Michael Bärtschi
53+
1454
## [v2.6.32](https://github.com/ash-project/ash_postgres/compare/v2.6.31...v2.6.32) (2026-02-11)
1555

1656

config/config.exs

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,9 @@ if Mix.env() == :test do
3535

3636
config :ash_postgres, :ash_domains, [AshPostgres.Test.Domain]
3737

38-
config :ash, :custom_expressions, [AshPostgres.Expressions.TrigramWordSimilarity]
38+
config :ash, :custom_expressions, [
39+
AshPostgres.Expressions.TrigramWordSimilarity
40+
]
3941

4042
config :ash, :known_types, [AshPostgres.Timestamptz, AshPostgres.TimestamptzUsec]
4143

documentation/topics/advanced/expressions.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,3 +81,4 @@ For example:
8181
```elixir
8282
Ash.Query.filter(User, trigram_similarity(first_name, "fred") > 0.8)
8383
```
84+

documentation/topics/development/migrations-and-tasks.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ dev migrations and run them.
2525

2626
For more information on generating migrations, run `mix help ash_postgres.generate_migrations` (the underlying task that is called by `mix ash.migrate`)
2727

28+
When you remove a resource from your domain, run the migration generator (e.g. `mix ash_postgres.generate_migrations --name remove_my_resource`). It will generate a migration to drop the table and remove the snapshot for that table.
29+
30+
When you rename a resource's table (e.g. change the `table "..."` in the `postgres do` block), the generator will ask whether you are renaming the table. If you answer yes, it generates a single `rename table(...), to: table(...)` migration so the table is renamed in place and data and foreign keys are preserved.
31+
2832
> ### all_tenants/0 {: .info}
2933
>
3034
> If you are using schema-based multitenancy, you will also need to define a `all_tenants/0` function in your repo module. See `AshPostgres.Repo` for more.
Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2020 Zach Daniel
3+
4+
SPDX-License-Identifier: MIT
5+
-->
6+
7+
# Working With Existing Databases
8+
9+
When you're building an Ash application against a database you don't own or control — such as a shared company database, a legacy system, or a third-party service's database — you need a workflow that lets you iterate on your Ash resources without generating migrations. The `--fragments` and `--no-migrations` options to `mix ash_postgres.gen.resources` are designed for exactly this.
10+
11+
## The Problem
12+
13+
Normally, Ash resources are the source of truth for your database schema, and migrations are generated from them. But when the database is managed externally:
14+
15+
- You don't want Ash generating migrations for a schema you don't control
16+
- The upstream schema may change, and you need to regenerate your resources to match
17+
- You still want to customize your resources with actions, calculations, validations, and other Ash features — without losing those customizations on regeneration
18+
19+
## The Workflow
20+
21+
### 1. Generate resources with `--fragments` and `--no-migrations`
22+
23+
```bash
24+
mix ash_postgres.gen.resources MyApp.ExternalDb \
25+
--tables users,orders,products \
26+
--no-migrations \
27+
--fragments
28+
```
29+
30+
This creates two files per table:
31+
32+
- **The resource file** (e.g., `lib/my_app/external_db/user.ex`) — contains `use Ash.Resource`, the `postgres` block, and any actions. This is *your* file to customize.
33+
- **The fragment file** (e.g., `lib/my_app/external_db/user/model.ex`) — contains the attributes, relationships, and identities introspected from the database. This file is regenerated by the tool.
34+
35+
The resource file will include `migrate? false` in its `postgres` block (from `--no-migrations`), telling Ash not to generate migrations for it:
36+
37+
```elixir
38+
defmodule MyApp.ExternalDb.User do
39+
use Ash.Resource,
40+
domain: MyApp.ExternalDb,
41+
data_layer: AshPostgres.DataLayer,
42+
fragments: [MyApp.ExternalDb.User.Model]
43+
44+
postgres do
45+
table "users"
46+
repo MyApp.Repo
47+
migrate? false
48+
end
49+
end
50+
```
51+
52+
The fragment file contains the schema details:
53+
54+
```elixir
55+
defmodule MyApp.ExternalDb.User.Model do
56+
use Spark.Dsl.Fragment,
57+
of: Ash.Resource
58+
59+
attributes do
60+
uuid_primary_key :id
61+
attribute :email, :string, public?: true
62+
attribute :name, :string, public?: true
63+
# ...
64+
end
65+
66+
relationships do
67+
has_many :orders, MyApp.ExternalDb.Order
68+
# ...
69+
end
70+
71+
identities do
72+
identity :unique_email, [:email]
73+
end
74+
end
75+
```
76+
77+
### 2. Customize your resources
78+
79+
Add actions, calculations, validations, changes, and anything else to the **resource file**. This is your space:
80+
81+
```elixir
82+
defmodule MyApp.ExternalDb.User do
83+
use Ash.Resource,
84+
domain: MyApp.ExternalDb,
85+
data_layer: AshPostgres.DataLayer,
86+
fragments: [MyApp.ExternalDb.User.Model]
87+
88+
actions do
89+
defaults [:read]
90+
91+
read :by_email do
92+
argument :email, :string, allow_nil?: false
93+
filter expr(email == ^arg(:email))
94+
end
95+
end
96+
97+
calculations do
98+
calculate :display_name, :string, expr(name || email)
99+
end
100+
101+
postgres do
102+
table "users"
103+
repo MyApp.Repo
104+
migrate? false
105+
end
106+
end
107+
```
108+
109+
### 3. Regenerate fragments when the schema changes
110+
111+
When the upstream database schema changes (new columns, new tables, changed relationships), re-run the same command:
112+
113+
```bash
114+
mix ash_postgres.gen.resources MyApp.ExternalDb \
115+
--tables users,orders,products \
116+
--no-migrations \
117+
--fragments
118+
```
119+
120+
Because the resource files already exist, **only the fragment files are regenerated**. Your customizations in the resource files are untouched.
121+
122+
### 4. Review the diff
123+
124+
After regeneration, review the changes with `git diff` to see what changed in the schema. New columns will appear as new attributes, altered relationships will be updated, and so on.
125+
126+
## Key Points
127+
128+
- **`--fragments`** splits generated schema details into a separate `Model` fragment module, keeping your resource file safe from regeneration
129+
- **`--no-migrations`** prevents migration generation and adds `migrate? false` to the `postgres` block
130+
- **Fragment files are disposable** — they are regenerated from the database each time. Don't put custom code in them.
131+
- **Resource files are yours** — once created on the first run, they won't be overwritten by subsequent runs
132+
- You can also use `--skip-tables` to exclude tables, `--tables` to scope to specific schemas (e.g., `accounts.`), and `--extend` to apply extensions to generated resources

lib/data_layer.ex

Lines changed: 55 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -693,16 +693,7 @@ defmodule AshPostgres.DataLayer do
693693
def can?(_, {:lock, :for_update}), do: true
694694
def can?(_, :composite_types), do: true
695695

696-
def can?(_, {:lock, string}) do
697-
string = String.trim_trailing(string, " NOWAIT")
698-
699-
String.upcase(string) in [
700-
"FOR UPDATE",
701-
"FOR NO KEY UPDATE",
702-
"FOR SHARE",
703-
"FOR KEY SHARE"
704-
]
705-
end
696+
def can?(_, {:lock, string}), do: string |> String.upcase() |> can_lock?()
706697

707698
def can?(_, :transact), do: true
708699
def can?(_, :composite_primary_key), do: true
@@ -784,6 +775,12 @@ defmodule AshPostgres.DataLayer do
784775
def can?(resource, :expr_error),
785776
do: not AshPostgres.DataLayer.Info.repo(resource, :mutate).disable_expr_error?()
786777

778+
def can?(resource, :required_error) do
779+
not AshPostgres.DataLayer.Info.repo(resource, :mutate).disable_expr_error?() &&
780+
"ash-functions" in AshPostgres.DataLayer.Info.repo(resource, :read).installed_extensions() &&
781+
"ash-functions" in AshPostgres.DataLayer.Info.repo(resource, :mutate).installed_extensions()
782+
end
783+
787784
def can?(resource, {:filter_expr, %Ash.Query.Function.Error{}}) do
788785
not AshPostgres.DataLayer.Info.repo(resource, :mutate).disable_expr_error?() &&
789786
"ash-functions" in AshPostgres.DataLayer.Info.repo(resource, :read).installed_extensions() &&
@@ -799,6 +796,23 @@ defmodule AshPostgres.DataLayer do
799796
def can?(_, {:sort, _}), do: true
800797
def can?(_, _), do: false
801798

799+
@locks [
800+
"FOR UPDATE",
801+
"FOR NO KEY UPDATE",
802+
"FOR SHARE",
803+
"FOR KEY SHARE"
804+
]
805+
806+
for lock <- @locks do
807+
defp can_lock?(unquote(lock)), do: true
808+
809+
for suffix <- ["NOWAIT", "SKIP LOCKED"] do
810+
defp can_lock?(unquote("#{lock} #{suffix}")), do: true
811+
end
812+
end
813+
814+
defp can_lock?(_), do: false
815+
802816
@impl true
803817
def in_transaction?(resource) do
804818
AshPostgres.DataLayer.Info.repo(resource, :mutate).in_transaction?()
@@ -884,9 +898,12 @@ defmodule AshPostgres.DataLayer do
884898
functions = [
885899
AshPostgres.Functions.Like,
886900
AshPostgres.Functions.ILike,
887-
AshPostgres.Functions.Binding
901+
AshPostgres.Functions.Binding,
902+
AshPostgres.Functions.PostgresIn
888903
]
889904

905+
functions = [Ash.Query.Function.RequiredError | functions]
906+
890907
functions =
891908
if "pg_trgm" in (config[:installed_extensions] || []) do
892909
functions ++
@@ -1123,6 +1140,13 @@ defmodule AshPostgres.DataLayer do
11231140
base_query
11241141
end
11251142

1143+
base_query =
1144+
if Map.get(relationship, :offset) do
1145+
from(row in base_query, offset: ^relationship.offset)
1146+
else
1147+
base_query
1148+
end
1149+
11261150
base_query =
11271151
cond do
11281152
Map.get(relationship, :manual) ->
@@ -2389,9 +2413,12 @@ defmodule AshPostgres.DataLayer do
23892413
# Include fields with update_defaults (e.g. update_timestamp)
23902414
# even if they aren't in the changeset attributes or upsert_fields.
23912415
# These fields should always be refreshed when an upsert modifies fields.
2392-
# Can be disabled via context: %{data_layer: %{touch_update_defaults?: false}}
2416+
# Can be disabled via touch_update_defaults?: false in the changeset
2417+
# context (either in [:private] or [:data_layer]) or via options map
23932418
touch_update_defaults? =
2394-
Enum.at(changesets, 0).context[:data_layer][:touch_update_defaults?] != false
2419+
Map.get(options, :touch_update_defaults?, true) &&
2420+
Enum.at(changesets, 0).context[:private][:touch_update_defaults?] != false &&
2421+
Enum.at(changesets, 0).context[:data_layer][:touch_update_defaults?] != false
23952422

23962423
if touch_update_defaults? do
23972424
update_default_fields =
@@ -3220,12 +3247,21 @@ defmodule AshPostgres.DataLayer do
32203247
else
32213248
keys = keys || Ash.Resource.Info.primary_key(keys)
32223249

3250+
touch_update_defaults? =
3251+
changeset.context[:private][:touch_update_defaults?] != false
3252+
32233253
update_defaults = update_defaults(resource)
32243254

32253255
explicitly_changing_attributes =
32263256
changeset.attributes
32273257
|> Map.keys()
3228-
|> Enum.concat(Keyword.keys(update_defaults))
3258+
|> then(fn attrs ->
3259+
if touch_update_defaults? do
3260+
Enum.concat(attrs, Keyword.keys(update_defaults))
3261+
else
3262+
attrs
3263+
end
3264+
end)
32293265
|> Kernel.--(Map.get(changeset, :defaults, []))
32303266
|> Kernel.--(keys)
32313267

@@ -3240,6 +3276,7 @@ defmodule AshPostgres.DataLayer do
32403276
upsert_keys: keys,
32413277
action_select: changeset.action_select,
32423278
upsert_fields: upsert_fields,
3279+
touch_update_defaults?: touch_update_defaults?,
32433280
return_records?: true
32443281
}) do
32453282
{:ok, []} ->
@@ -3575,13 +3612,6 @@ defmodule AshPostgres.DataLayer do
35753612
end
35763613
end
35773614

3578-
@locks [
3579-
"FOR UPDATE",
3580-
"FOR NO KEY UPDATE",
3581-
"FOR SHARE",
3582-
"FOR KEY SHARE"
3583-
]
3584-
35853615
for lock <- @locks do
35863616
frag = "#{lock} OF ?"
35873617

@@ -3590,16 +3620,16 @@ defmodule AshPostgres.DataLayer do
35903620
end
35913621

35923622
frag = "#{lock} OF ? NOWAIT"
3593-
lock = "#{lock} NOWAIT"
3623+
new_lock = "#{lock} NOWAIT"
35943624

3595-
def lock(query, unquote(lock), _) do
3625+
def lock(query, unquote(new_lock), _) do
35963626
{:ok, Ecto.Query.lock(query, [{^0, a}], fragment(unquote(frag), a))}
35973627
end
35983628

35993629
frag = "#{lock} OF ? SKIP LOCKED"
3600-
lock = "#{lock} SKIP LOCKED"
3630+
new_lock = "#{lock} SKIP LOCKED"
36013631

3602-
def lock(query, unquote(lock), _) do
3632+
def lock(query, unquote(new_lock), _) do
36033633
{:ok, Ecto.Query.lock(query, [{^0, a}], fragment(unquote(frag), a))}
36043634
end
36053635
end

0 commit comments

Comments
 (0)