[Bug]: `freq_agg` panics with "called `Option::unwrap()` on a `None` value" under parallel aggregation when a `FILTER` clause yields an empty group

[FrancescEthon]

What type of bug is this?

Crash

What subsystems and features are affected?

Query executor

What happened?

Summary

toolkit_experimental.freq_agg(...) aborts the query with

ERROR:  called `Option::unwrap()` on a `None` value

when all of the following hold:

1. the aggregate is evaluated with a parallel plan (partial aggregation + a Gather/Finalize),
2. the aggregate call carries a FILTER (WHERE ...) clause,
3. at least one group has all of its rows removed by the FILTER (i.e. that group's partial aggregate state is never initialized / is empty), and
4. the resulting sketch is actually finalized/returned (a query that discards the sketch — e.g. wraps it in an outer count(*) — does not crash).

The same query runs fine serially, or when no group is fully filtered out, or without a FILTER.

Minimal, self-contained reproduction

CREATE EXTENSION IF NOT EXISTS timescaledb;
CREATE EXTENSION IF NOT EXISTS timescaledb_toolkit;

CREATE TABLE freq_bug (g int, keep boolean, val double precision);

-- 100 groups across 1,000,000 rows:
--   groups 0..49   -> keep = true   (freq_agg receives rows)
--   groups 50..99  -> keep = false  (FILTER removes ALL their rows -> empty partial state)
INSERT INTO freq_bug
SELECT (i % 100), ((i % 100) < 50), (i % 1000)::float8
FROM generate_series(1, 1000000) i;
ANALYZE freq_bug;

-- Force a parallel aggregation plan
SET max_parallel_workers_per_gather = 4;
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET min_parallel_table_scan_size = 0;

-- >>> PANICS: ERROR: called `Option::unwrap()` on a `None` value
SELECT g, toolkit_experimental.freq_agg(0.01, val::text) FILTER (WHERE keep)
FROM freq_bug
GROUP BY g;

DROP TABLE freq_bug;

Plan (confirms it is parallel)

 Finalize GroupAggregate
   ->  Gather Merge
         ->  Partial HashAggregate
               ->  Parallel Seq Scan on freq_bug

Expected behavior

A group whose rows are all removed by the FILTER should yield NULL (or an empty sketch) for that group — exactly as it does under a serial plan — not abort the whole statement.

Actual behavior

ERROR:  called `Option::unwrap()` on a `None` value

Isolation matrix

All rows below use the table above; only the stated dimension changes.

#	aggregate	parallel	FILTER	a fully-filtered group?	result
1	freq_agg	yes	yes	yes	ERROR (unwrap on None)
2	freq_agg	no	yes	yes	OK (empty groups → NULL)
3	freq_agg	yes	yes	no	OK
4	freq_agg	yes	no	—	OK
5	hyperloglog	yes	yes	yes	OK
6	stats_agg	yes	yes	yes	OK

Reproductions for the key controls (each replaces only the relevant clause):

-- #2 serial: OK
SET max_parallel_workers_per_gather = 0;
SELECT g, toolkit_experimental.freq_agg(0.01, val::text) FILTER (WHERE keep)
FROM freq_bug GROUP BY g;

-- #3 parallel, but every group keeps some rows (val < 900) → no empty partial: OK
-- (re-apply the parallel SETs from above)
SELECT g, toolkit_experimental.freq_agg(0.01, val::text) FILTER (WHERE val < 900)
FROM freq_bug GROUP BY g;

-- #5/#6 parallel + FILTER + empty groups with other aggregates: OK
SELECT g, hyperloglog(1024, val) FILTER (WHERE keep) FROM freq_bug GROUP BY g;
SELECT g, stats_agg(val)         FILTER (WHERE keep) FROM freq_bug GROUP BY g;

Note: the sketch must be finalized/returned for the crash to surface. Wrapping the
crashing query so the sketch column is discarded — e.g.
SELECT count(*) FROM (<query #1>) q; — does not error, which points at the
combine/finalize/serialize path for an empty (None) partial state.

Workarounds

Any of these avoids the panic:

• Restrict with WHERE in a subquery instead of FILTER, so no group is ever empty:

  SELECT g, toolkit_experimental.freq_agg(0.01, val::text)
  FROM freq_bug WHERE keep GROUP BY g;   -- only non-empty groups exist

  (LEFT JOIN this back to the full group list if rows for the filtered-out groups are still needed.)
• Disable parallel aggregation for the statement: SET max_parallel_workers_per_gather = 0;
• Avoid FILTER directly on freq_agg.

Impact

Hit in production when computing a per-group freq_agg with a FILTER (to restrict the
sketch to a subset of groups) over a large table that the planner parallelizes; groups
outside the filtered subset produce empty partial states and abort the whole query.

TimescaleDB version affected

2.27.1 Toolkit 1.22.0

PostgreSQL version used

17.10

What operating system did you use?

Ubuntu 22.04

What installation method did you use?

Docker

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

ERROR:  called `Option::unwrap()` on a `None` value

How can we reproduce the bug?

CREATE EXTENSION IF NOT EXISTS timescaledb;
CREATE EXTENSION IF NOT EXISTS timescaledb_toolkit;

CREATE TABLE freq_bug (g int, keep boolean, val double precision);

-- 100 groups across 1,000,000 rows:
--   groups 0..49   -> keep = true   (freq_agg receives rows)
--   groups 50..99  -> keep = false  (FILTER removes ALL their rows -> empty partial state)
INSERT INTO freq_bug
SELECT (i % 100), ((i % 100) < 50), (i % 1000)::float8
FROM generate_series(1, 1000000) i;
ANALYZE freq_bug;

-- Force a parallel aggregation plan
SET max_parallel_workers_per_gather = 4;
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET min_parallel_table_scan_size = 0;

-- >>> PANICS: ERROR: called `Option::unwrap()` on a `None` value
SELECT g, toolkit_experimental.freq_agg(0.01, val::text) FILTER (WHERE keep)
FROM freq_bug
GROUP BY g;

DROP TABLE freq_bug;

[Dvdandrades]

As this bug originates in the TimescaleDB Toolkit, I recommend transferring this issue to the corresponding repository.

This looks like a recurring pattern in the toolkit where the aggregate's internal state is never initialized if no rows are passed to it. The code calls unwrap() on the state, which results in a panic and aborts the query because the state is None instead of being NULL.

I'd be happy to handle this since I recently implemented a similar fix in another aggregate. I can locate the problematic unwrap() call and submit a PR with the fix.