Skip to content

pg_stat_monitor 2.3.2 leaks ~4 kB of backend memory per execution of a prepared statement on PostgreSQL 18.4 #792

Description

@developer-Bushido

Description

While pg_stat_monitor is tracking a session, every execution of a named prepared statement (Parse once / Execute repeatedly) leaks roughly 4 kB of private backend memory. The allocation is outside PostgreSQL memory contexts: pg_log_backend_memory_contexts() reports a ~10 MB grand total while the backend's private RSS is in gigabytes.

Simple and extended (unnamed portal) protocols do not leak. The same workload tracked only by pg_stat_statements does not leak.

Control matrix, same host, same statement (SELECT 1;), pgbench single connection, 24 s:

pgbench mode pgsm_track=top pgsm_track=none
-M simple flat flat
-M extended flat flat
-M prepared 243 MB -> 2168 MB flat

Real-world impact: any driver that auto-prepares statements (psycopg3, Go pgx, JDBC) triggers this on long-lived connections. In our case Patroni 4.1.1 (psycopg3 3.3.4) holds two permanent connections per node; the backend serving its REST-API health checks grew ~460 MB/day, reached 4.4 GB RSS in 10 days and drove an 8 GB node into OOM livelock.

Workaround: ALTER ROLE <role> SET pg_stat_monitor.pgsm_track = 'none'; + reconnect stops the growth immediately.

Expected Results

Backend private memory stays bounded when a prepared statement is executed repeatedly with pg_stat_monitor enabled, as it does with pg_stat_statements or with pgsm_track = 'none'.

Actual Results

Backend Private_Dirty (from /proc//smaps_rollup) grows linearly at ~4.0 kB per Execute of a prepared statement: from ~1.4 MB to 2.17 GB within 24 seconds at ~20k tps. Growth rate scales with execution rate and never plateaus; the backend is eventually OOM-killed or exhausts the host. Memory context dump of the bloated backend stays ~10 MB, so the leaked memory is allocated outside memory contexts.

Version

PostgreSQL 18.4 (postgresql18-18.4-2PGDG.rhel10.2.x86_64)
pg_stat_monitor 2.3.2 (pg_stat_monitor_18-2.3.2-1PGDG.rhel10.2.x86_64)
CentOS Stream 10, x86_64
pgsm settings at defaults: pgsm_track=top, pgsm_bucket_time=60, pgsm_max=256, pgsm_query_shared_buffer=20
Originally hit via Patroni 4.1.1 + psycopg3 3.3.4 (auto-prepared statements). Not yet tested on PG 16/17 builds.

Steps to reproduce

  1. PostgreSQL 18.4 with shared_preload_libraries = 'pg_stat_monitor', CREATE EXTENSION pg_stat_monitor.

  2. Run a single-connection prepared-statement loop:

    echo 'SELECT 1;' > s.sql
    PGOPTIONS='-c pg_stat_monitor.pgsm_track=top' pgbench -n -M prepared -f s.sql -T 30 -c 1

  3. Watch the backend's private memory:

    grep Private_Dirty /proc/<backend_pid>/smaps_rollup

    It grows by ~4 kB per executed statement (tens of MB per second at pgbench speed).

  4. Repeat with -M simple or -M extended, or with PGOPTIONS='-c pg_stat_monitor.pgsm_track=none' — memory stays flat in all three cases.

Relevant logs

# pgbench -n -f s.sql -T 30 -c 1 (SELECT 1), sampling Private_Dirty of the backend at t=3s and t=27s:

proto=simple   pgsm_track=top:  Private_Dirty 1408 -> 1408 kB,      tps=17513
proto=simple   pgsm_track=none: Private_Dirty 1324 -> 1324 kB,      tps=19516
proto=extended pgsm_track=top:  Private_Dirty 1420 -> 1420 kB,      tps=16448
proto=extended pgsm_track=none: Private_Dirty 1324 -> 1324 kB,      tps=19172
proto=prepared pgsm_track=top:  Private_Dirty 243244 -> 2168632 kB, tps=19978   <-- leak
proto=prepared pgsm_track=none: Private_Dirty 1328 -> 1328 kB,      tps=25302

# memory context dump of a production backend bloated to 4.4 GB RSS (Patroni connection, 10 days old):
Grand total: 10448216 bytes in 273 blocks; 4463576 free (396 chunks); 5984640 used

Code of Conduct

  • I agree to follow Percona Community Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions