Skip to content

[bug]: duplicate proof inserts are logged as NEW_PROOF, inflating universe stats #2253

Description

@GeorgeTsagk

Problem

Archive.UpsertProofLeafBatch logs a NEW_PROOF event for every item it was
handed, not for the leaves that were actually new:

// universe/archive.go:609 (main @ aea668c7)
// Log a sync event for the newly inserted leaf in the background as an
// async goroutine.
ids := fn.Map(items, func(item *Item) Identifier {
	return item.ID
})

Archive.UpsertProofLeaf does the same unconditionally at archive.go:364.
Re-registering a leaf the node already holds is a no-op upsert, but it still
writes an event row.

Two consequences:

  1. UniverseStats.num_total_proofs overcounts. The universe_stats view
    derives total_asset_proofs as COUNT(*) over
    universe_events WHERE event_type = 'NEW_PROOF', so the reported figure is
    insert attempts, not proofs held.
  2. universe_events has no retention and is never pruned, so the table grows
    with sync traffic rather than with universe size, and the view aggregates
    the whole table on every stats read.

Data

From a long-running regtest federation (postgres backend, ~6 days, tapd
v0.8.1):

universe_events (NEW_PROOF)   1,324,186 rows
universe_leaves                  11,126 rows
worst single root               460,720 events / 2,502 leaves

tapcli universe stats on that node:

{"num_total_proofs":"1324096","num_total_assets":"10"}

Caveat on the magnitude: most of this amplification comes from the
pointer-identity diff bug fixed in bd69208 (#2187, not in v0.8.1 or the
v0.8.2 RCs, so released versions still re-fetch a peer's whole leaf set on
every root mismatch). With that fix in place the rate drops sharply. The
residual is still real: any peer pushing a leaf we already hold inflates the
stat, and nothing bounds the table over time.

Suggested fix

Have UpsertProofLeaf/UpsertProofLeafBatch report which leaves were newly
inserted, and log events only for those. Retention for universe_events is a
separate question but worth considering for long-lived universe servers.

Related: #1633 covers query-plan optimization for QueryUniverseAssetStats,
which is a different concern from the row growth and the double counting here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinguniverse

Type

No type

Projects

Status
🆕 New

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions