Skip to content

Per-message HashMap allocs in session sender #1385

@muscariello

Description

@muscariello

Files:

  • core/session/src/session_sender.rson_publish_message()
  • core/session/src/producer_buffer.rspush()

Problem:
In reliable-session mode, every outbound message triggers:

  1. producer_buffer.push()HashMap::insert(msg_id, slot).
  2. pending_acks.insert(msg_id, …) — another HashMap insert.
  3. ack_notifiers.insert(msg_id, oneshot_tx) — a third HashMap insert.
  4. A per-message retransmission timer (tokio::spawn of a timer task).

Three HashMap inserts + a spawned task per message is significant at high
rates.

Mitigation plan:

  1. Replace the three per-message HashMaps with a single indexed structure
    (slab or ring buffer keyed by msg_id % window).
  2. Use a timer wheel (e.g., tokio_util::time::DelayQueue) instead of
    spawning individual timer tasks. One task manages all pending timeouts.
  3. Batch ACK processing: coalesce multiple ACKs before modifying data
    structures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status

    No status

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions