Skip to content

Commit e92f49e

Browse files
gmelodiegmelodie
committed
feat: provider record spillover
1 parent 6b6203e commit e92f49e

1 file changed

Lines changed: 223 additions & 0 deletions

File tree

‎kad-dht/provider-record-spillover.md‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# Provider Record Spillover
2+
3+
| Lifecycle Stage | Maturity | Status | Latest Revision |
4+
|-----------------|---------------|--------|-----------------|
5+
| 1A | Working Draft | Active | r0, 2026-05-11 |
6+
7+
Authors: [@gmelodie]
8+
9+
Interest Group: []
10+
11+
[@gmelodie]: https://github.com/gmelodie
12+
13+
See the [lifecycle document][lifecycle-spec] for context about the maturity level
14+
and spec status.
15+
16+
[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md
17+
18+
---
19+
20+
## Overview
21+
22+
This document specifies an optional extension to the [libp2p Kademlia DHT
23+
specification][kad-spec] that addresses provider record hotspots. When a key is
24+
popular, the `k` closest nodes concentrate all `ADD_PROVIDER` traffic for it
25+
and can become overloaded. This extension lets nodes enforce per-key provider
26+
limits and signal rejection to advertisers, and lets advertisers spill over to
27+
progressively farther peers from their lookup path rather than repeatedly
28+
hammering the same overloaded nodes.
29+
30+
The extension is opt-in and backward compatible. Nodes that have not enabled it
31+
behave exactly as before.
32+
33+
## Definitions
34+
35+
**Provider record capacity**: the maximum number of distinct providers a node
36+
is willing to store for a single key.
37+
38+
**Rejection**: a node signalling to an advertiser that it will not store the
39+
provider record for the given key.
40+
41+
**Spillover**: the behavior of an advertiser that, upon receiving rejections
42+
from the closest peers, continues advertising to progressively farther peers
43+
discovered during the iterative lookup.
44+
45+
**Spillover round**: one batch of `ADD_PROVIDER` requests sent to the next
46+
group of candidates during a spillover.
47+
48+
## Motivation
49+
50+
In the base Kademlia DHT protocol, provider advertisement targets the `k`
51+
closest peers to the content key unconditionally. For popular keys, these peers
52+
accumulate unbounded provider records and act as a permanent bottleneck. The
53+
base spec provides no mechanism for a peer to decline an `ADD_PROVIDER` or for
54+
the advertiser to store the record elsewhere.
55+
56+
This extension solves both sides of the problem:
57+
58+
1. **Server-side protection**: nodes can enforce a per-key limit and reject
59+
`ADD_PROVIDER` requests once the limit is reached.
60+
2. **Client-side resilience**: advertisers react to rejections by backtracking
61+
through the lookup path, widening the set of peers that store the record
62+
until the replication target is met.
63+
64+
## Provider Record Limits
65+
66+
A node MAY enforce a maximum number of distinct providers per key,
67+
`maxProvidersPerKey`. When set, the node counts the distinct provider peer IDs
68+
already stored for the key. If that count is at or above `maxProvidersPerKey`
69+
and the `ADD_PROVIDER` sender is not already a known provider for that key
70+
(i.e., it is not a re-advertisement), the node MUST reject the request.
71+
72+
Re-advertisements from a provider that is already stored for the key are always
73+
accepted, regardless of the limit, so that existing providers can refresh their
74+
records.
75+
76+
Nodes SHOULD also enforce coarser bounds such as total provider records stored
77+
(`providerRecordCapacity`) and total distinct keys for which records are held
78+
(`providedKeyCapacity`), but those are outside the scope of the rejection
79+
signalling defined here.
80+
81+
## ADD_PROVIDER Rejection
82+
83+
### Response signalling
84+
85+
Support for this extension is optional. A node that supports it MUST include a
86+
`providerStatus` field (field 11, see [Protobuf](#protobuf)) in its
87+
`ADD_PROVIDER` response:
88+
89+
- `accepted (0)` — the record was stored.
90+
- `rejected (1)` — the record was not stored due to capacity limits.
91+
92+
An absent `providerStatus` field — whether because the responding node does not
93+
support this extension or due to a timeout — MUST be interpreted as `accepted`
94+
by the advertiser.
95+
96+
## Spillover Algorithm
97+
98+
### Overview
99+
100+
When an advertiser sends `ADD_PROVIDER` to a batch of peers and all peers in
101+
that batch reject, it performs a **spillover round**: it moves to the next
102+
group of peers that are farther from the key, as discovered during the initial
103+
iterative lookup. This continues until either:
104+
105+
- the replication target `k` accepting peers has been reached, or
106+
- the advertiser has exhausted all peers discovered during the lookup.
107+
108+
### Lookup phase
109+
110+
Before advertising, the advertiser performs a full iterative lookup (using
111+
`FIND_NODE`) for the content key as usual, collecting all peers encountered.
112+
The resulting candidate set is sorted in ascending order of XOR distance to
113+
the key.
114+
115+
### Advertisement phase
116+
117+
The advertiser splits the sorted candidate list into chunks of size `α` (the
118+
concurrency parameter). It then iterates over these chunks from closest to
119+
farthest:
120+
121+
1. Send `ADD_PROVIDER` to all peers in the current chunk concurrently.
122+
2. Collect responses. Count accepting peers (absent `providerStatus` or
123+
`providerStatus = accepted`) towards the replication target.
124+
3. If the replication target `k` accepting peers has been reached, stop.
125+
4. If **all** peers in the chunk rejected (a unanimous rejection), continue to
126+
the next chunk (spillover round). Otherwise stop, whether the target was
127+
met or not, to avoid unnecessary advertisement.
128+
5. If no more chunks remain, stop.
129+
130+
**Note:** The timeout per peer in a spillover round SHOULD be slightly larger
131+
than the base timeout to account for dial overhead to less-familiar peers.
132+
133+
### Relationship to base advertisement
134+
135+
This algorithm is a generalisation of the base `ADD_PROVIDER` procedure. When
136+
no peer rejects, only the closest chunk (or the closest `k` peers) is used and
137+
behaviour is identical to the base spec. Spillover only occurs when unanimous
138+
rejections are encountered.
139+
140+
## Protobuf
141+
142+
The following changes extend the `Message` type defined in the [kad-dht
143+
spec][kad-spec]:
144+
145+
```protobuf
146+
syntax = "proto2";
147+
148+
message Record {
149+
bytes key = 1;
150+
bytes value = 2;
151+
string timeReceived = 5;
152+
}
153+
154+
message Message {
155+
enum MessageType {
156+
PUT_VALUE = 0;
157+
GET_VALUE = 1;
158+
ADD_PROVIDER = 2;
159+
GET_PROVIDERS = 3;
160+
FIND_NODE = 4;
161+
PING = 5;
162+
}
163+
164+
enum ConnectionType {
165+
NOT_CONNECTED = 0;
166+
CONNECTED = 1;
167+
CAN_CONNECT = 2;
168+
CANNOT_CONNECT = 3;
169+
}
170+
171+
// Added by this extension.
172+
enum AddProviderStatus {
173+
accepted = 0;
174+
rejected = 1;
175+
}
176+
177+
message Peer {
178+
bytes id = 1;
179+
repeated bytes addrs = 2;
180+
ConnectionType connection = 3;
181+
}
182+
183+
MessageType type = 1;
184+
bytes key = 2;
185+
Record record = 3;
186+
repeated Peer closerPeers = 8;
187+
repeated Peer providerPeers = 9;
188+
int32 clusterLevelRaw = 10; // NOT USED
189+
190+
// Added by this extension. Absent field MUST be treated as accepted.
191+
optional AddProviderStatus providerStatus = 11;
192+
}
193+
```
194+
195+
Field 11 is optional; older implementations that do not know about it ignore
196+
it, maintaining full wire-level backward compatibility.
197+
198+
## Backward Compatibility
199+
200+
- Nodes that have not enabled `providerRejection` never write `providerStatus`
201+
and are unaffected by receiving it.
202+
- Advertisers that do not implement spillover continue to advertise to the `k`
203+
closest peers; absent `providerStatus` is treated as acceptance.
204+
- No change is made to `GET_PROVIDERS` or any lookup message.
205+
206+
## Security Considerations
207+
208+
**False rejections**: an adversary controlling the closest peers to a key could
209+
reject all `ADD_PROVIDER` requests to suppress its advertisement. Spillover
210+
defeats this: unanimous rejection causes the advertiser to route around those
211+
peers and store the record on nodes beyond the adversary's controlled set.
212+
However, an adversary that mixes accepts and rejects strategically can stay
213+
below the unanimous-rejection threshold and reduce replication without
214+
triggering spillover. This is a known limitation of the unanimous-rejection
215+
trigger.
216+
217+
---
218+
219+
## References
220+
221+
[0]: libp2p Kademlia DHT specification – https://github.com/libp2p/specs/blob/master/kad-dht/README.md
222+
223+
[kad-spec]: https://github.com/libp2p/specs/blob/master/kad-dht/README.md

0 commit comments

Comments
 (0)