Skip to content

graph/db: cross-version graph Store#10656

Open
ellemouton wants to merge 9 commits intolightningnetwork:masterfrom
ellemouton:g175-graph-db-cross-version
Open

graph/db: cross-version graph Store#10656
ellemouton wants to merge 9 commits intolightningnetwork:masterfrom
ellemouton:g175-graph-db-cross-version

Conversation

@ellemouton
Copy link
Collaborator

@ellemouton ellemouton commented Mar 23, 2026

Summary

Make the graph Store interface cross-version so that query methods work
across gossip v1 and v2, enabling callers to interact with the graph without
needing to know which gossip version announced a given node or channel.

This is part of the Gossip 1.75 epic.

Key changes

  • Version-aware horizon queries: NodeUpdatesInHorizon and
    ChanUpdatesInHorizon now take a gossip version and a typed range
    (NodeUpdateRange/ChanUpdateRange) that enforces v1=timestamps vs
    v2=block-heights at the type level. Full SQL implementations for both
    versions.

  • Cross-version iteration: ForEachNode and ForEachChannel iterate
    across all gossip versions, yielding each unique node/channel exactly once
    (highest version preferred).

  • PreferHighest fetch helpers: FetchChannelEdgesByIDPreferHighest,
    FetchChannelEdgesByOutpointPreferHighest return the highest available
    gossip version for a channel without the caller specifying one.

  • GetVersions queries: GetVersionsBySCID and GetVersionsByOutpoint
    report which gossip versions exist for a given channel.

  • Version-aware FilterKnownChanIDs: Now takes a gossip version parameter.
    Zombie-revival logic moved from ChannelGraph into VersionedGraph so
    callers through that type don't need to pass a version explicitly.

  • Prefer-highest node traversal: ForEachNodeDirectedChannel and
    FetchNodeFeatures on ChannelGraph iterate across all versions (v2
    first), so pathfinding prefers v2-announced channels and features.

@gemini-code-assist
Copy link

Warning

Gemini encountered an error creating the summary. You can try again by commenting /gemini summary.

@ellemouton ellemouton force-pushed the g175-graph-db-cross-version branch from 5337e61 to cbe9436 Compare March 23, 2026 12:07
@ellemouton ellemouton self-assigned this Mar 23, 2026
@ellemouton ellemouton force-pushed the g175-graph-db-cross-version branch from cbe9436 to 670b11f Compare March 23, 2026 12:44
Add version-aware range types for channel and node update horizon
queries. V1 gossip uses unix timestamps for ordering while v2 uses
block heights, so each range type validates that the correct bound
type is provided for the requested gossip version.

These types will be used in follow-up commits to version the
NodeUpdatesInHorizon and ChanUpdatesInHorizon Store methods.
Replace the (startTime, endTime time.Time) parameters on
NodeUpdatesInHorizon and ChanUpdatesInHorizon with
(v GossipVersion, r NodeUpdateRange/ChanUpdateRange). The range
types enforce version-correct bounds at the type level: v1 uses unix
timestamps, v2 uses block heights.

The KV store rejects non-v1 versions since it only stores v1 data.
The SQL store handles v1 queries; v2 block-height range queries are
added in the following commits.

VersionedGraph wrappers supply the version from the embedded field,
so callers only pass the range.
@ellemouton ellemouton force-pushed the g175-graph-db-cross-version branch 2 times, most recently from f689002 to f6937d1 Compare March 23, 2026 13:07
Add GetNodesByBlockHeightRange and GetChannelsByPolicyBlockRange
SQL queries for v2 gossip horizon lookups which use block heights
instead of unix timestamps. Hardcode version=1 in
GetNodesByLastUpdateRange and GetChannelsByPolicyLastUpdateRange
since only v1 gossip uses unix-timestamp-based ordering.

Implement the v2 gossip cases in the SQL store's
NodeUpdatesInHorizon and ChanUpdatesInHorizon using the new queries,
replacing the placeholder TODO stubs. Add extractMaxBlockHeight
helper for v2 channel pagination cursor tracking.

Also add a TestV2HorizonQueries integration test that exercises both
node and channel horizon lookups with block-height ranges on the SQL
backend.
Add a gossip version parameter to FilterKnownChanIDs on the Store
interface so that it can filter channel IDs for the correct gossip
version. The KV store rejects non-v1 versions.

Move the zombie-revival logic from ChannelGraph into VersionedGraph
so that FilterKnownChanIDs is only exposed on VersionedGraph (which
supplies the version from its embedded field). This means callers
no longer need to pass a version explicitly.
Add four new Store interface methods:

- FetchChannelEdgesByIDPreferHighest: version-agnostic channel lookup
  that returns the highest available gossip version.
- FetchChannelEdgesByOutpointPreferHighest: same but keyed by outpoint.
- GetVersionsBySCID: returns which gossip versions exist for a SCID.
- GetVersionsByOutpoint: same but keyed by outpoint.

The KV store implementations delegate to v1 since that's the only
version it supports. The SQL store implementations iterate over
versions from highest to lowest (for PreferHighest) or lowest to
highest (for GetVersions).

These will be used in follow-up commits to make ChannelGraph callers
version-agnostic.
Remove the gossip version parameter from ForEachNode and
ForEachChannel on the Store interface. Both methods now iterate
across all gossip versions, yielding each unique node/channel
exactly once. A versionsMask (uint32) is passed to the callback
where bit 0 indicates a v1 entry exists and bit 1 indicates v2.

The KV store always passes versionsMask=1 since it only stores v1
data. The SQL store collects entries across both versions, deduping
by pub key (nodes) or channel ID (channels), preferring the highest
version's data when both exist.

VersionedGraph wrappers discard the versionsMask so that existing
callers through that type don't need to change their callbacks.
…rappers

Switch FetchChannelEdgesByID and FetchChannelEdgesByOutpoint on
ChannelGraph to use the PreferHighest store variants so that callers
get the highest available gossip version without needing to specify
one.

Also add GetVersionsBySCID and GetVersionsByOutpoint convenience
wrappers on ChannelGraph that delegate to the corresponding Store
methods.
Update ForEachNodeDirectedChannel and FetchNodeFeatures on
ChannelGraph to iterate across all gossip versions (highest first)
instead of hardcoding v1. This ensures that channels announced via
v2 are preferred over v1 and that v2 features are used when
available.
Copy link
Collaborator

@bhandras bhandras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🥇


// TestValidateForVersion verifies that ChanUpdateRange and NodeUpdateRange
// reject invalid field combinations for each gossip version.
func TestValidateForVersion(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the naming of this test could be more specific so it reflects the purpose better.


hasMore = len(rows) == batchSize

//nolint:ll
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this block seems to be within the line limit.

},
)
}, func() {
batch = []*models.Node{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could just be = nil.

}

queryV2 := func(db SQLQueries) ([]sqlc.GraphNode, error) {
//nolint:ll
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: these nolints don't seem to have any effect as the block is within limits.

sqlc.GetChannelsByPolicyBlockRangeParams{
Version: int16(v),
StartHeight: sqldb.SQLInt64(
int64(r.StartHeight.UnwrapOr(0)), //nolint:ll
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra nolints :)

0, len(blockRows),
)
for _, br := range blockRows {
//nolint:ll
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: nolint 🪓

return cb(channel)
},
func() {
seen = make(map[uint64]struct{})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call the reset here?

// v2 channel with no policies from
// hiding a v1 channel that had
// valid policy data.
hasPolicies := p1 != nil || p2 != nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codex: Medium: the new “prefer highest” fetch helpers can hide a usable v1 edge behind an incomplete v2 shell. In graph/db/sql_store.go:1904, ForEachChannel explicitly avoids letting a v2 row with no policies replace a v1 row that has valid policy data. But graph/db/sql_store.go:2587 and graph/db/sql_store.go:2613 return the first v2 hit unconditionally, and graph/db/graph.go:730 routes the unversioned ChannelGraph lookups through those helpers. So callers can now get nil policies or otherwise less-useful edge data even when a complete v1 record exists for the same channel. I’d either mirror the ForEachChannel safeguard here or make the API/documentation explicit that “prefer highest” does not mean “prefer highest usable edge state.”

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants