Skip to content

SelectGatewayEnvelopesByOriginators is still too slow #1523

@mkysel

Description

@mkysel
Image

an average of 584ms is not acceptable.

Here is an example standalone query

WITH cursors AS (
  SELECT
    x.node_id AS cursor_node_id,
    y.seq_id  AS cursor_sequence_id
  FROM unnest(ARRAY[200, 0, 1, 100]::INT[]) WITH ORDINALITY AS x(node_id, ord)
  JOIN unnest(ARRAY[737081, 223047, 1244704, 643862]::BIGINT[]) WITH ORDINALITY AS y(seq_id, ord)
    USING (ord)
),
filtered AS (
  SELECT
    m.originator_node_id,
    m.originator_sequence_id,
    m.gateway_time,
    m.topic
  FROM gateway_envelopes_meta AS m
  LEFT JOIN cursors AS c
    ON m.originator_node_id = c.cursor_node_id
  WHERE m.originator_node_id = ANY (ARRAY[100]::INT[])
    AND m.originator_sequence_id > COALESCE(c.cursor_sequence_id, 0)
  ORDER BY m.originator_node_id, m.originator_sequence_id
  LIMIT 1000
)
SELECT
  f.originator_node_id,
  f.originator_sequence_id,
  f.gateway_time,
  f.topic,
  b.originator_envelope
FROM filtered AS f
JOIN gateway_envelope_blobs AS b
  ON b.originator_node_id = f.originator_node_id
 AND b.originator_sequence_id = f.originator_sequence_id
ORDER BY f.originator_node_id, f.originator_sequence_id;

The plan will query all subpartitions and then throw everything away.

A simpler plan executes in 200ms.

SELECT
  m.originator_node_id,
  m.originator_sequence_id,
  m.gateway_time,
  m.topic,
  b.originator_envelope
FROM gateway_envelopes_meta AS m
JOIN gateway_envelope_blobs AS b
  ON b.originator_node_id = m.originator_node_id
 AND b.originator_sequence_id = m.originator_sequence_id
WHERE m.originator_node_id = 100
  AND m.originator_sequence_id >= 643862
ORDER BY m.originator_node_id, m.originator_sequence_id
LIMIT 1000;

Even 200ms feels too slow for an index scan that returns a few rows

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Status

    Todo

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions