Skip to content

docs: overlay hashes vs payloads (pull by Hash) #5443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
drQedwards wants to merge 2 commits into from
Closed

docs: overlay hashes vs payloads (pull by Hash) #5443

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b715e40
docs: spell out overlay hash-first pull vs payload flood
Sep 2, 2026
d3d793b
Potential fix for pull request finding
drQedwards Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions docs/overlay.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,36 @@ title: Overlay
- peers request new data when they're ready to process it. This is done to prevent network congestion. Applying back-pressure on the receiver side also allows the sender to prioritize accumulated messages in the queue, and shed load that becomes obsolete.

* Versioning: The overlay subsystem has a version number, which is the latest version of the protocol that the node supports. It also maintains a minimum supported overlay version. Any connection that doesn't support the minimum version is rejected.

## Hashes versus payloads

The overlay does not flood full objects by default. Peers first exchange
**identifiers** (32-byte hashes). A node only **pulls** the matching payload if
it does not already have it. Flooding is keyed by `Hash` in `FloodGate`
(`src/overlay/Floodgate.h`).

`StellarMessage` (see `Stellar-overlay.x`) splits into three kinds, documented
in `src/overlay/OverlayManager.h`:

- **Peer-directed:** `HELLO`, `PEERS`, `DONT_HAVE`, `ERROR_MSG`.
- **Broadcast:** `TRANSACTION`, `SCP_MESSAGE`. Consensus votes are **pushed**
because they are small and latency-sensitive.
- **Anycast by hash:** `GET_TX_SET` / `TX_SET`, `GET_SCP_QUORUMSET` /
`SCP_QUORUMSET`, `GET_SCP_STATE`. `ItemFetcher` asks connected peers, in
sequence, for the body of a hash. These messages are not flooded.

Transaction dissemination in pull mode uses the same split:

- `FLOOD_ADVERT` carries a `FloodAdvert` of `txHashes` (up to
`TX_ADVERT_VECTOR_MAX_SIZE`).
- `FLOOD_DEMAND` carries a `FloodDemand` of hashes this node is missing.
- The transaction body is sent only in response to a demand.

So a peer that already holds a given hash never downloads the envelope again.
That is the same rule as `broadcastMessage(..., std::optional<Hash>)`: when a
transaction is flooded, its envelope hash is what overlay uses to decide
whether the message is new.

History catchup and long-term ledger archives stay **off** the overlay (see
`docs/architecture.md` and `docs/history.md`). Overlay is for live consensus
and mempool, not for shipping the full payload of the network's past.
5 changes: 5 additions & 0 deletions src/overlay/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,8 @@ the network any transactions injected from public API servers.

Good reading entry points are [`OverlayManager.h`](./OverlayManager.h), as well as the implementation of
`OverlayManagerImpl::tick`, and `OverlayManagerImpl::broadcastMessage`.

Flooding is hash-keyed (`Floodgate`). Transaction envelopes are advertised and
requested with `FLOOD_ADVERT` / `FLOOD_DEMAND`; transaction sets and quorum
sets are fetched by hash through `ItemFetcher` using `GET_TX_SET` and
`GET_SCP_QUORUMSET`. See [`docs/overlay.md`](../../docs/overlay.md).