Description
While pg_stat_monitor is tracking a session, every execution of a named prepared statement (Parse once / Execute repeatedly) leaks roughly 4 kB of private backend memory. The allocation is outside PostgreSQL memory contexts: pg_log_backend_memory_contexts() reports a ~10 MB grand total while the backend's private RSS is in gigabytes.
Simple and extended (unnamed portal) protocols do not leak. The same workload tracked only by pg_stat_statements does not leak.
Control matrix, same host, same statement (SELECT 1;), pgbench single connection, 24 s:
| pgbench mode |
pgsm_track=top |
pgsm_track=none |
| -M simple |
flat |
flat |
| -M extended |
flat |
flat |
| -M prepared |
243 MB -> 2168 MB |
flat |
Real-world impact: any driver that auto-prepares statements (psycopg3, Go pgx, JDBC) triggers this on long-lived connections. In our case Patroni 4.1.1 (psycopg3 3.3.4) holds two permanent connections per node; the backend serving its REST-API health checks grew ~460 MB/day, reached 4.4 GB RSS in 10 days and drove an 8 GB node into OOM livelock.
Workaround: ALTER ROLE <role> SET pg_stat_monitor.pgsm_track = 'none'; + reconnect stops the growth immediately.
Expected Results
Backend private memory stays bounded when a prepared statement is executed repeatedly with pg_stat_monitor enabled, as it does with pg_stat_statements or with pgsm_track = 'none'.
Actual Results
Backend Private_Dirty (from /proc//smaps_rollup) grows linearly at ~4.0 kB per Execute of a prepared statement: from ~1.4 MB to 2.17 GB within 24 seconds at ~20k tps. Growth rate scales with execution rate and never plateaus; the backend is eventually OOM-killed or exhausts the host. Memory context dump of the bloated backend stays ~10 MB, so the leaked memory is allocated outside memory contexts.
Version
PostgreSQL 18.4 (postgresql18-18.4-2PGDG.rhel10.2.x86_64)
pg_stat_monitor 2.3.2 (pg_stat_monitor_18-2.3.2-1PGDG.rhel10.2.x86_64)
CentOS Stream 10, x86_64
pgsm settings at defaults: pgsm_track=top, pgsm_bucket_time=60, pgsm_max=256, pgsm_query_shared_buffer=20
Originally hit via Patroni 4.1.1 + psycopg3 3.3.4 (auto-prepared statements). Not yet tested on PG 16/17 builds.
Steps to reproduce
-
PostgreSQL 18.4 with shared_preload_libraries = 'pg_stat_monitor', CREATE EXTENSION pg_stat_monitor.
-
Run a single-connection prepared-statement loop:
echo 'SELECT 1;' > s.sql
PGOPTIONS='-c pg_stat_monitor.pgsm_track=top' pgbench -n -M prepared -f s.sql -T 30 -c 1
-
Watch the backend's private memory:
grep Private_Dirty /proc/<backend_pid>/smaps_rollup
It grows by ~4 kB per executed statement (tens of MB per second at pgbench speed).
-
Repeat with -M simple or -M extended, or with PGOPTIONS='-c pg_stat_monitor.pgsm_track=none' — memory stays flat in all three cases.
Relevant logs
# pgbench -n -f s.sql -T 30 -c 1 (SELECT 1), sampling Private_Dirty of the backend at t=3s and t=27s:
proto=simple pgsm_track=top: Private_Dirty 1408 -> 1408 kB, tps=17513
proto=simple pgsm_track=none: Private_Dirty 1324 -> 1324 kB, tps=19516
proto=extended pgsm_track=top: Private_Dirty 1420 -> 1420 kB, tps=16448
proto=extended pgsm_track=none: Private_Dirty 1324 -> 1324 kB, tps=19172
proto=prepared pgsm_track=top: Private_Dirty 243244 -> 2168632 kB, tps=19978 <-- leak
proto=prepared pgsm_track=none: Private_Dirty 1328 -> 1328 kB, tps=25302
# memory context dump of a production backend bloated to 4.4 GB RSS (Patroni connection, 10 days old):
Grand total: 10448216 bytes in 273 blocks; 4463576 free (396 chunks); 5984640 used
Code of Conduct
Description
While pg_stat_monitor is tracking a session, every execution of a named prepared statement (Parse once / Execute repeatedly) leaks roughly 4 kB of private backend memory. The allocation is outside PostgreSQL memory contexts:
pg_log_backend_memory_contexts()reports a ~10 MB grand total while the backend's private RSS is in gigabytes.Simple and extended (unnamed portal) protocols do not leak. The same workload tracked only by pg_stat_statements does not leak.
Control matrix, same host, same statement (
SELECT 1;), pgbench single connection, 24 s:Real-world impact: any driver that auto-prepares statements (psycopg3, Go pgx, JDBC) triggers this on long-lived connections. In our case Patroni 4.1.1 (psycopg3 3.3.4) holds two permanent connections per node; the backend serving its REST-API health checks grew ~460 MB/day, reached 4.4 GB RSS in 10 days and drove an 8 GB node into OOM livelock.
Workaround:
ALTER ROLE <role> SET pg_stat_monitor.pgsm_track = 'none';+ reconnect stops the growth immediately.Expected Results
Backend private memory stays bounded when a prepared statement is executed repeatedly with pg_stat_monitor enabled, as it does with pg_stat_statements or with pgsm_track = 'none'.
Actual Results
Backend Private_Dirty (from /proc//smaps_rollup) grows linearly at ~4.0 kB per Execute of a prepared statement: from ~1.4 MB to 2.17 GB within 24 seconds at ~20k tps. Growth rate scales with execution rate and never plateaus; the backend is eventually OOM-killed or exhausts the host. Memory context dump of the bloated backend stays ~10 MB, so the leaked memory is allocated outside memory contexts.
Version
PostgreSQL 18.4 (postgresql18-18.4-2PGDG.rhel10.2.x86_64)
pg_stat_monitor 2.3.2 (pg_stat_monitor_18-2.3.2-1PGDG.rhel10.2.x86_64)
CentOS Stream 10, x86_64
pgsm settings at defaults: pgsm_track=top, pgsm_bucket_time=60, pgsm_max=256, pgsm_query_shared_buffer=20
Originally hit via Patroni 4.1.1 + psycopg3 3.3.4 (auto-prepared statements). Not yet tested on PG 16/17 builds.
Steps to reproduce
PostgreSQL 18.4 with shared_preload_libraries = 'pg_stat_monitor', CREATE EXTENSION pg_stat_monitor.
Run a single-connection prepared-statement loop:
echo 'SELECT 1;' > s.sql
PGOPTIONS='-c pg_stat_monitor.pgsm_track=top' pgbench -n -M prepared -f s.sql -T 30 -c 1
Watch the backend's private memory:
grep Private_Dirty /proc/<backend_pid>/smaps_rollup
It grows by ~4 kB per executed statement (tens of MB per second at pgbench speed).
Repeat with -M simple or -M extended, or with PGOPTIONS='-c pg_stat_monitor.pgsm_track=none' — memory stays flat in all three cases.
Relevant logs
Code of Conduct