graph/db: cross-version graph Store#10656
graph/db: cross-version graph Store#10656ellemouton wants to merge 9 commits intolightningnetwork:masterfrom
Conversation
|
Warning Gemini encountered an error creating the summary. You can try again by commenting |
5337e61 to
cbe9436
Compare
cbe9436 to
670b11f
Compare
Add version-aware range types for channel and node update horizon queries. V1 gossip uses unix timestamps for ordering while v2 uses block heights, so each range type validates that the correct bound type is provided for the requested gossip version. These types will be used in follow-up commits to version the NodeUpdatesInHorizon and ChanUpdatesInHorizon Store methods.
Replace the (startTime, endTime time.Time) parameters on NodeUpdatesInHorizon and ChanUpdatesInHorizon with (v GossipVersion, r NodeUpdateRange/ChanUpdateRange). The range types enforce version-correct bounds at the type level: v1 uses unix timestamps, v2 uses block heights. The KV store rejects non-v1 versions since it only stores v1 data. The SQL store handles v1 queries; v2 block-height range queries are added in the following commits. VersionedGraph wrappers supply the version from the embedded field, so callers only pass the range.
f689002 to
f6937d1
Compare
Add GetNodesByBlockHeightRange and GetChannelsByPolicyBlockRange SQL queries for v2 gossip horizon lookups which use block heights instead of unix timestamps. Hardcode version=1 in GetNodesByLastUpdateRange and GetChannelsByPolicyLastUpdateRange since only v1 gossip uses unix-timestamp-based ordering. Implement the v2 gossip cases in the SQL store's NodeUpdatesInHorizon and ChanUpdatesInHorizon using the new queries, replacing the placeholder TODO stubs. Add extractMaxBlockHeight helper for v2 channel pagination cursor tracking. Also add a TestV2HorizonQueries integration test that exercises both node and channel horizon lookups with block-height ranges on the SQL backend.
Add a gossip version parameter to FilterKnownChanIDs on the Store interface so that it can filter channel IDs for the correct gossip version. The KV store rejects non-v1 versions. Move the zombie-revival logic from ChannelGraph into VersionedGraph so that FilterKnownChanIDs is only exposed on VersionedGraph (which supplies the version from its embedded field). This means callers no longer need to pass a version explicitly.
Add four new Store interface methods: - FetchChannelEdgesByIDPreferHighest: version-agnostic channel lookup that returns the highest available gossip version. - FetchChannelEdgesByOutpointPreferHighest: same but keyed by outpoint. - GetVersionsBySCID: returns which gossip versions exist for a SCID. - GetVersionsByOutpoint: same but keyed by outpoint. The KV store implementations delegate to v1 since that's the only version it supports. The SQL store implementations iterate over versions from highest to lowest (for PreferHighest) or lowest to highest (for GetVersions). These will be used in follow-up commits to make ChannelGraph callers version-agnostic.
Remove the gossip version parameter from ForEachNode and ForEachChannel on the Store interface. Both methods now iterate across all gossip versions, yielding each unique node/channel exactly once. A versionsMask (uint32) is passed to the callback where bit 0 indicates a v1 entry exists and bit 1 indicates v2. The KV store always passes versionsMask=1 since it only stores v1 data. The SQL store collects entries across both versions, deduping by pub key (nodes) or channel ID (channels), preferring the highest version's data when both exist. VersionedGraph wrappers discard the versionsMask so that existing callers through that type don't need to change their callbacks.
…rappers Switch FetchChannelEdgesByID and FetchChannelEdgesByOutpoint on ChannelGraph to use the PreferHighest store variants so that callers get the highest available gossip version without needing to specify one. Also add GetVersionsBySCID and GetVersionsByOutpoint convenience wrappers on ChannelGraph that delegate to the corresponding Store methods.
Update ForEachNodeDirectedChannel and FetchNodeFeatures on ChannelGraph to iterate across all gossip versions (highest first) instead of hardcoding v1. This ensures that channels announced via v2 are preferred over v1 and that v2 features are used when available.
f6937d1 to
fe7a97b
Compare
|
|
||
| // TestValidateForVersion verifies that ChanUpdateRange and NodeUpdateRange | ||
| // reject invalid field combinations for each gossip version. | ||
| func TestValidateForVersion(t *testing.T) { |
There was a problem hiding this comment.
nit: the naming of this test could be more specific so it reflects the purpose better.
|
|
||
| hasMore = len(rows) == batchSize | ||
|
|
||
| //nolint:ll |
There was a problem hiding this comment.
nit: this block seems to be within the line limit.
| }, | ||
| ) | ||
| }, func() { | ||
| batch = []*models.Node{} |
There was a problem hiding this comment.
nit: could just be = nil.
| } | ||
|
|
||
| queryV2 := func(db SQLQueries) ([]sqlc.GraphNode, error) { | ||
| //nolint:ll |
There was a problem hiding this comment.
nit: these nolints don't seem to have any effect as the block is within limits.
| sqlc.GetChannelsByPolicyBlockRangeParams{ | ||
| Version: int16(v), | ||
| StartHeight: sqldb.SQLInt64( | ||
| int64(r.StartHeight.UnwrapOr(0)), //nolint:ll |
| 0, len(blockRows), | ||
| ) | ||
| for _, br := range blockRows { | ||
| //nolint:ll |
| return cb(channel) | ||
| }, | ||
| func() { | ||
| seen = make(map[uint64]struct{}) |
There was a problem hiding this comment.
Should we call the reset here?
| // v2 channel with no policies from | ||
| // hiding a v1 channel that had | ||
| // valid policy data. | ||
| hasPolicies := p1 != nil || p2 != nil |
There was a problem hiding this comment.
codex: Medium: the new “prefer highest” fetch helpers can hide a usable v1 edge behind an incomplete v2 shell. In graph/db/sql_store.go:1904, ForEachChannel explicitly avoids letting a v2 row with no policies replace a v1 row that has valid policy data. But graph/db/sql_store.go:2587 and graph/db/sql_store.go:2613 return the first v2 hit unconditionally, and graph/db/graph.go:730 routes the unversioned ChannelGraph lookups through those helpers. So callers can now get nil policies or otherwise less-useful edge data even when a complete v1 record exists for the same channel. I’d either mirror the ForEachChannel safeguard here or make the API/documentation explicit that “prefer highest” does not mean “prefer highest usable edge state.”
Summary
Make the graph
Storeinterface cross-version so that query methods workacross gossip v1 and v2, enabling callers to interact with the graph without
needing to know which gossip version announced a given node or channel.
This is part of the Gossip 1.75 epic.
Key changes
Version-aware horizon queries:
NodeUpdatesInHorizonandChanUpdatesInHorizonnow take a gossip version and a typed range(
NodeUpdateRange/ChanUpdateRange) that enforces v1=timestamps vsv2=block-heights at the type level. Full SQL implementations for both
versions.
Cross-version iteration:
ForEachNodeandForEachChanneliterateacross all gossip versions, yielding each unique node/channel exactly once
(highest version preferred).
PreferHighest fetch helpers:
FetchChannelEdgesByIDPreferHighest,FetchChannelEdgesByOutpointPreferHighestreturn the highest availablegossip version for a channel without the caller specifying one.
GetVersions queries:
GetVersionsBySCIDandGetVersionsByOutpointreport which gossip versions exist for a given channel.
Version-aware FilterKnownChanIDs: Now takes a gossip version parameter.
Zombie-revival logic moved from
ChannelGraphintoVersionedGraphsocallers through that type don't need to pass a version explicitly.
Prefer-highest node traversal:
ForEachNodeDirectedChannelandFetchNodeFeaturesonChannelGraphiterate across all versions (v2first), so pathfinding prefers v2-announced channels and features.