[bug]: full recovery (seed + backup import) resets address key derivation — new addresses reuse historical script keys, breaking receives

[satscribe]

Background

The wallet asset backup (tapcli assets backup export/import) does not preserve the address key derivation state. After a node loses its ~/.tapd directory (LND wallet intact) and restores via assets backup import, every new tapcli addrs new re-derives script keys that were already used by pre-loss addresses/outputs. The custodian logs make this visible:

GRDN: Taproot addr <p2tr> was already added to wallet before, skipping

A counterparty that holds proofs for those historical script keys (e.g. the node that originally sent assets to this wallet) then fails to send to any new address of the restored node:

unable to fund address send: error funding packet:
error extracting allocations: output amount exceeds input sum

The same sender pays a fresh third node without issues, and the restored node's addresses were confirmed to carry script keys identical to keys that currently anchor existing assets (we observed a new address re-using the exact script key that holds the wallet's own current balance).

Net effect: a node restored from the native backup cannot receive from its previous counterparties — in practice its main peers — and on-chain address/key reuse is silently reintroduced. A file-level backup of ~/.tapd (which preserves the key counters) does not exhibit the problem.

This compounds with #2220 (grouped assets restored unspendable): together they mean the native backup path currently cannot fully recover a wallet.

Your environment

• tapd v0.8.0-alpha (commit v0.8.0), integrated mode inside litd v0.17.0-alpha
• lnd v0.21.1-beta (integrated)
• regtest, bitcoind 27.1 backend, Docker (official lightninglabs/lightning-terminal:v0.17.0-alpha image)

Steps to reproduce

1. Node A mints an asset and sends 100 units to node B via a V1 address (universe proof courier); do a few back-and-forth transfers so several script keys have been used.
2. Node B: tapcli assets backup export --mode raw --output_file b.bin.
3. Stop node B; move ~/.tapd away (keep ~/.lnd untouched); start node B; tapcli assets backup import --backup_file b.bin.
4. Node B: tapcli addrs new --asset_id <id> --amt 9 → tapd logs "already added to wallet before, skipping"; the new address's script key equals a historical one.
5. Node A: tapcli assets send --addr <new B address> → fails with "output amount exceeds input sum". Repeating with additional fresh addresses from B fails the same way; sending to a clean node C succeeds.

Expected behaviour

Either the backup carries the address/script-key derivation state, or the restore path re-syncs the key index (or addrs new skips keys that already have proofs/outputs), so that a restored wallet can safely receive again without key reuse.

Actual behaviour

Restored wallets re-derive already-used script keys; senders holding proofs for those keys fail with allocation errors, and address reuse is reintroduced silently.

[GeorgeTsagk]

I’m unable to reproduce the key-reuse behavior under the stated condition that the LND wallet remains intact.

Taproot Assets does not maintain the authoritative key-derivation counter used for new asset addresses.

tapd requests new keys through LND’s WalletKit.DeriveNextKey, and LND persists and advances the derivation index in its own wallet state. Deleting ~/.tapd while retaining the same ~/.lnd wallet should therefore not reset key derivation or cause previously used script keys to be returned again.

[jtobin]

It seems that the problematic condition described here could only occur if .lnd were also wiped and the wallet basically restored from seed (so lnd would have no a priori information about indexes used). This is somewhat problematic, but definitely a more extreme situation.

Fable claims a fix could exist wholly on tapd's side, which seems reasonable:

The backup format already contains what's needed for the fix. Each entry carries ScriptKeyInfo.RawKey and AnchorInternalKeyInfo with full key locators (family + index) — that's what ImportBackup upserts into tapd's DB. Import could compute the max index per family across the blob and fast-forward lnd by looping DeriveNextKey until past it. That's essentially the OP's "restore re-syncs the key index" expected-behavior option.

[satscribe]

Thanks for checking — you're right, and I owe a correction on the reproduction conditions. Re-reading my lab notes: the key-reuse behaviour appeared after the LND wallet itself had been re-created from its aezeed seed (initwallet with recovery_window=250), i.e. the full disaster-recovery flow (seed + tapd backup blob), not the narrower case where ~/.lnd survives intact. With ~/.lnd genuinely untouched I cannot reproduce it either — apologies for the imprecision in the original report.

Corrected repro:

1. Node A mints an asset, sends 100 units to node B (V1 address, universe courier); a few back-and-forth transfers so several script keys from tapd's key family have been consumed.
2. Node B: tapcli assets backup export --mode raw --output_file b.bin.
3. Destroy node B completely (~/.lnd and ~/.tapd).
4. Recreate node B's wallet from the same aezeed seed (initwallet, recovery_window=250); on-chain funds are recovered fine.
5. tapcli assets backup import --backup_file b.bin → assets visible.
6. tapcli addrs new → GRDN logs Taproot addr <p2tr> was already added to wallet before, skipping; the new address carries a script key that already anchors assets from the imported proofs.
7. Node A: tapcli assets send --addr <new B address> → output amount exceeds input sum. Sending to a clean node C works.

The mechanism seems to be: LND's recovery scan advances on-chain address indexes, but has no way to learn how many keys the tapd key family had consumed, so DeriveNextKey restarts at low indexes and tapd re-derives script keys that are already present in the imported proof set.

Since seed + backup blob is the canonical full-recovery path for this feature, would it make sense for the backup to carry the highest used key index per family (or for restore to advance derivation past the highest script-key index found in the backup)? Happy to re-run the lab against any patch — same environment as #2220 (which, thank you for the immediate fix in #2222!).

[GeorgeTsagk]

Reproduced locally, and Bob does re-use the derivation paths. One difference is that on my end Alice's sends seem to succeed when sending to the re-used Bob addresses. Can you share a bit more on the setup / amounts involved?

Regardless of that detail, we should indeed add some derivation index in the backup. Thanks for reporting this.

[satscribe]

Thanks George — glad it reproduces, and agreed that carrying a per-family derivation index in the backup (or fast-forwarding DeriveNextKey past the max key locator index found in the imported blob, per Fable's note above) is the right fix.

On the difference you saw — Alice's sends succeeding to the reused Bob addresses on your end while ours failed — the most likely factor is that our asset is grouped. Exact setup and amounts:

1. Alice mints a grouped normal asset: assets mint --type normal --name PROOFRES --supply 10000 --new_grouped_asset.

2. Alice sends to Bob over a V1 address (universe courier): 100 units, then a few back-and-forth transfers so several script keys from tapd's key family get consumed. Bob nets 85 units across the transfers.

3. assets backup export --mode raw on Bob.

4. Destroy Bob completely (~/.lnd and ~/.tapd), recreate from the same aezeed (initwallet, recovery_window=250), then assets backup import.

5. addrs new returns a reused script key (Taproot addr <p2tr> was already added to wallet before, skipping); the reused key already anchors the 85 units from the imported proofs.

6. Alice sends to that new Bob address and gets output amount exceeds input sum. A clean node C receives fine from the same Alice.

So on our side the reused script key already anchors an asset UTXO that Alice's proof set knows about, which seems to be what pushes output past input during allocation. Was your repro using a grouped or a plain asset, and roughly what amounts? Our failure is on a grouped asset — the same asset type hit by #2220 — so it's possible the send-side break only surfaces for grouped, or when the reused key already anchors a known UTXO.

Happy to share the raw backup blob and the tapd/lnd logs from the failing send, or to re-run the lab against any patch (same environment as #2220).