Skip to content

Commit 76e97be

Browse files
hlinnakamrdrivingduck
authored andcommitted
Fix data loss when restarting the bulk_write facility
If a user started a bulk write operation on a fork with existing data to append data in bulk, the bulk_write machinery would zero out all previously written pages up to the last page written by the new bulk_write operation. This is not an issue for PostgreSQL itself, because we never use the bulk_write facility on a non-empty fork. But there are use cases where it makes sense. TimescaleDB extension is known to do that to merge partitions, for example. Backpatch to v17, where the bulk_write machinery was introduced. Author: Matthias van de Meent <[email protected]> Reported-By: Erik Nordström <[email protected]> Reviewed-by: Erik Nordström <[email protected]> Discussion: https://www.postgresql.org/message-id/CACAa4VJ%[email protected]
1 parent 239d923 commit 76e97be

File tree

1 file changed

+10
-8
lines changed

1 file changed

+10
-8
lines changed

src/backend/storage/smgr/bulk_write.c

+10-8
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@
44
* Efficiently and reliably populate a new relation
55
*
66
* The assumption is that no other backends access the relation while we are
7-
* loading it, so we can take some shortcuts. Do not mix operations through
8-
* the regular buffer manager and the bulk loading interface!
7+
* loading it, so we can take some shortcuts. Pages already present in the
8+
* indicated fork when the bulk write operation is started are not modified
9+
* unless explicitly written to. Do not mix operations through the regular
10+
* buffer manager and the bulk loading interface!
911
*
1012
* We bypass the buffer manager to avoid the locking overhead, and call
1113
* smgrextend() directly. A downside is that the pages will need to be
@@ -71,7 +73,7 @@ struct BulkWriteState
7173
PendingWrite pending_writes[MAX_PENDING_WRITES];
7274

7375
/* Current size of the relation */
74-
BlockNumber pages_written;
76+
BlockNumber relsize;
7577

7678
MemoryContext memcxt;
7779
};
@@ -110,7 +112,7 @@ smgr_bulk_start_smgr(SMgrRelation smgr, ForkNumber forknum, bool use_wal, char r
110112
state->relpersistence = relpersistence;
111113

112114
state->npending = 0;
113-
state->pages_written = 0;
115+
state->relsize = smgrnblocks(smgr, forknum);
114116

115117
/*
116118
* Remember the memory context. We will use it to allocate all the
@@ -167,11 +169,11 @@ smgr_bulk_flush(BulkWriteState *bulkstate)
167169
* Before we alloc buffers from buffer pool for those pages, extend the
168170
* underlying file first.
169171
*/
170-
if (nblocks > bulkstate->pages_written)
172+
if (nblocks > bulkstate->relsize)
171173
{
172-
smgrzeroextend(bulkstate->smgr, bulkstate->forknum, bulkstate->pages_written,
173-
nblocks - bulkstate->pages_written, true);
174-
bulkstate->pages_written = nblocks;
174+
smgrzeroextend(bulkstate->smgr, bulkstate->forknum, bulkstate->relsize,
175+
nblocks - bulkstate->relsize, true);
176+
bulkstate->relsize = nblocks;
175177
}
176178

177179
for (int i = 0; i < npending;)

0 commit comments

Comments
 (0)