Commit 26bde72
authored
PG to PG: Add ORDER BY in MERGE to handle non-PK unique columns (#4169)
## Problem statement
Consider a Postgres table such with `id` PRIMARY KEY and `email` TEXT
with a UNIQUE constraint on it. Let's say there is a mirror of this
table to another PG database, and the following changes are made on
source:
```sql
DELETE FROM table WHERE id = 1 (where email = 'alice@example.com')
INSERT INTO table (id, email) VALUES (2, 'alice@example.com')
```
This can result in a unique constraint violation error in normalize:
```
error executing normalize statement for table public.accounts:
ERROR: duplicate key value violates unique constraint
```
This is because MERGE in Postgres processes rows in non-deterministic
fashion, therefore across PKeys we could attempt replicating the INSERT
first before DELETE (note: for rows pertaining to the same PKey this is
not possible as we rank there), resulting in us hitting a non-PK unique
constraint.
## Fix
According to [Postgres docs for
MERGE](https://www.postgresql.org/docs/current/sql-merge.html), it says:
```
The order in which rows are generated from the data source is indeterminate by default.
A source_query can be used to specify a consistent ordering, if required, which might be needed to avoid deadlocks between concurrent transactions.
```
We are already using the `source_query` syntax in our MERGE. This PR
adds an ORDER BY to our subquery which sorts by `_peerdb_timestamp`
which is representative of WAL order, thereby not running into this
issue.
- [x] Functionally tested1 parent b0735da commit 26bde72
1 file changed
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| |||
0 commit comments