Problem
Archive.UpsertProofLeafBatch logs a NEW_PROOF event for every item it was
handed, not for the leaves that were actually new:
// universe/archive.go:609 (main @ aea668c7)
// Log a sync event for the newly inserted leaf in the background as an
// async goroutine.
ids := fn.Map(items, func(item *Item) Identifier {
return item.ID
})
Archive.UpsertProofLeaf does the same unconditionally at archive.go:364.
Re-registering a leaf the node already holds is a no-op upsert, but it still
writes an event row.
Two consequences:
UniverseStats.num_total_proofs overcounts. The universe_stats view
derives total_asset_proofs as COUNT(*) over
universe_events WHERE event_type = 'NEW_PROOF', so the reported figure is
insert attempts, not proofs held.
universe_events has no retention and is never pruned, so the table grows
with sync traffic rather than with universe size, and the view aggregates
the whole table on every stats read.
Data
From a long-running regtest federation (postgres backend, ~6 days, tapd
v0.8.1):
universe_events (NEW_PROOF) 1,324,186 rows
universe_leaves 11,126 rows
worst single root 460,720 events / 2,502 leaves
tapcli universe stats on that node:
{"num_total_proofs":"1324096","num_total_assets":"10"}
Caveat on the magnitude: most of this amplification comes from the
pointer-identity diff bug fixed in bd69208 (#2187, not in v0.8.1 or the
v0.8.2 RCs, so released versions still re-fetch a peer's whole leaf set on
every root mismatch). With that fix in place the rate drops sharply. The
residual is still real: any peer pushing a leaf we already hold inflates the
stat, and nothing bounds the table over time.
Suggested fix
Have UpsertProofLeaf/UpsertProofLeafBatch report which leaves were newly
inserted, and log events only for those. Retention for universe_events is a
separate question but worth considering for long-lived universe servers.
Related: #1633 covers query-plan optimization for QueryUniverseAssetStats,
which is a different concern from the row growth and the double counting here.
Problem
Archive.UpsertProofLeafBatchlogs aNEW_PROOFevent for every item it washanded, not for the leaves that were actually new:
Archive.UpsertProofLeafdoes the same unconditionally atarchive.go:364.Re-registering a leaf the node already holds is a no-op upsert, but it still
writes an event row.
Two consequences:
UniverseStats.num_total_proofsovercounts. Theuniverse_statsviewderives
total_asset_proofsasCOUNT(*)overuniverse_events WHERE event_type = 'NEW_PROOF', so the reported figure isinsert attempts, not proofs held.
universe_eventshas no retention and is never pruned, so the table growswith sync traffic rather than with universe size, and the view aggregates
the whole table on every stats read.
Data
From a long-running regtest federation (postgres backend, ~6 days, tapd
v0.8.1):
tapcli universe statson that node:{"num_total_proofs":"1324096","num_total_assets":"10"}Caveat on the magnitude: most of this amplification comes from the
pointer-identity diff bug fixed in bd69208 (#2187, not in v0.8.1 or the
v0.8.2 RCs, so released versions still re-fetch a peer's whole leaf set on
every root mismatch). With that fix in place the rate drops sharply. The
residual is still real: any peer pushing a leaf we already hold inflates the
stat, and nothing bounds the table over time.
Suggested fix
Have
UpsertProofLeaf/UpsertProofLeafBatchreport which leaves were newlyinserted, and log events only for those. Retention for
universe_eventsis aseparate question but worth considering for long-lived universe servers.
Related: #1633 covers query-plan optimization for
QueryUniverseAssetStats,which is a different concern from the row growth and the double counting here.