Skip to content

docs: overlay hashes vs payloads (pull by Hash) - #5443

Closed
drQedwards wants to merge 2 commits into
stellar:masterfrom
drQedwards:docs/overlay-hashes-vs-payloads
Closed

drQedwards wants to merge 2 commits into
stellar:masterfrom
drQedwards:docs/overlay-hashes-vs-payloads

Conversation

@drQedwards

Copy link
Copy Markdown

Summary

docs/overlay.md already says block/tx flooding is pull-based and SCP is push-based. This PR names the actual machinery so a first-time reader of overlay does not have to reverse-engineer OverlayManager.h and Stellar-overlay.x.

No protocol or consensus change.

What

Document that:

  • FloodGate is keyed by Hash.
  • Transaction and quorum-set bodies are anycast by hash (GET_TX_SET / GET_SCP_QUORUMSET via ItemFetcher).
  • Pull-mode tx gossip advertises hashes (FLOOD_ADVERT) and demands missing ones (FLOOD_DEMAND); the envelope is sent only if the peer does not already have that hash.
  • History / long-term ledger archives stay off the overlay (docs/architecture.md).

A short pointer is added in src/overlay/README.md.

Why

Peers on this network already treat a 32-byte hash as the thing that is flooded, and the payload as something you pull iff you lack it. That is easy to miss from the current overlay glossary. Naming FLOOD_ADVERT / FLOOD_DEMAND and ItemFetcher matches the headers and XDR.

Test plan

  • Docs only; no C++ change.
  • Overlay maintainers: wording matches current pull-mode behavior.

Document that FloodGate is keyed by Hash, ItemFetcher anycasts
GET_TX_SET / GET_SCP_QUORUMSET by hash, and pull-mode tx
dissemination uses FLOOD_ADVERT / FLOOD_DEMAND so a peer that
already has a hash never re-downloads the envelope. History
archives stay off the overlay.

No consensus or protocol change.
Copilot AI balanced review requested due to automatic review settings September 2, 2026 00:18
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documents hash-based overlay dissemination and links subsystem guidance.

Changes:

  • Explains push, pull, and anycast message flows.
  • Adds an overlay README pointer.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
docs/overlay.md Documents hash and payload handling.
src/overlay/README.md Summarizes and links overlay documentation.
Suppressed comments (2)

docs/overlay.md:38

  • GET_SCP_STATE is not an anycast-by-hash operation handled by ItemFetcher: it carries a ledger sequence (Peer::sendGetScpState) and directly asks that peer to send SCP envelopes. Only transaction-set and quorum-set fetches use the two ItemFetcher instances.
- **Anycast by hash:** `GET_TX_SET` / `TX_SET`, `GET_SCP_QUORUMSET` /
  `SCP_QUORUMSET`, `GET_SCP_STATE`. `ItemFetcher` asks connected peers, in
  sequence, for the body of a hash. These messages are not flooded.

docs/overlay.md:50

  • These lines conflate two distinct hashes. Floodgate::broadcast indexes mFloodMap with xdrBlake2(*msg), while the optional argument is the transaction-envelope hash passed to sendAdvert; it is not the key used to decide whether the Floodgate broadcast record is new.
So a peer that already holds a given hash never downloads the envelope again.
That is the same rule as `broadcastMessage(..., std::optional<Hash>)`: when a
transaction is flooded, its envelope hash is what overlay uses to decide
whether the message is new.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/overlay.md
Comment thread src/overlay/README.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 2, 2026 00:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

docs/overlay.md:28

  • This opening incorrectly generalizes transaction pull mode to the whole overlay: SCP_MESSAGE and survey traffic are sent as full pushed/broadcast messages. Narrow the statement to the hash-pulled object types; the replacement also uses the actual class name, Floodgate.

This issue also appears in the following locations of the same file:

  • line 30
  • line 36
  • line 47

docs/overlay.md:31

  • This sounds exhaustive, but the union has additional message families such as authentication, flow-control, advert/demand, and survey messages. Present these three groups as representative rather than claiming that all StellarMessage variants split this way.
`StellarMessage` (see `Stellar-overlay.x`) splits into three kinds, documented
in `src/overlay/OverlayManager.h`:

docs/overlay.md:38

  • GET_SCP_STATE is neither keyed by a hash nor driven by ItemFetcher: it carries a ledger sequence and directly causes the selected peer to send stored SCP envelopes. Also, current tx-set replies can be GENERALIZED_TX_SET, so the documented request/reply pair is incomplete.
- **Anycast by hash:** `GET_TX_SET` / `TX_SET`, `GET_SCP_QUORUMSET` /
  `SCP_QUORUMSET`, `GET_SCP_STATE`. `ItemFetcher` asks connected peers, in
  sequence, for the body of a hash. These messages are not flooded.

docs/overlay.md:50

  • The optional transaction hash is not the key Floodgate uses to decide whether a message is new. Floodgate::broadcast always keys its record with xdrBlake2(*msg); the optional full transaction hash is instead supplied to sendAdvert, so these are separate hash domains.
So a peer that already holds a given hash never downloads the envelope again.
That is the same rule as `broadcastMessage(..., std::optional<Hash>)`: when a
transaction is flooded, its envelope hash is what overlay uses to decide
whether the message is new.

@drQedwards drQedwards left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these functions need to be C coded into a file or a new file made according to this?

@graydon graydon closed this Sep 2, 2026
@drQedwards

drQedwards commented Sep 4, 2026

Copy link
Copy Markdown
Author

It's because I did that separate joke PR wasn't it?

All well, you're absolutely right, (Says ChatGPT me), those contracts aren't fully a Soroban smart contract address as primitives for the interchainer tool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants