Skip to content

Commit 4d8dbbb

Browse files
committed
docs: update packet-format.md to the 128-bit NodeId wire layout (closes the 2.4 doc gap)
The NodeId widening (2.4) doubled every on-the-wire NodeId from 8 to 16 bytes but packet-format.md still documented the u64/8-byte layout. Updated throughout: - NodeId is 16 raw bytes (not a u64); header is 17 B (was 9); sender/originator/ event.creator and breadcrumb NodeIds are 16 B each. - Recomputed every size: clock notification 89->97 B; chain request base 89->113 (+16.h breadcrumb); chain response 21->37 + 16.h + EventChain; EventReference 44->52 B; Event 64+d+44r -> 72+d+52r; ClockEvent 76+44r -> 84+52r; and the hash preimage's ref-creator is now 16 B. These match the empirical sim run (packet bytes +~14%, e.g. every clock notification +8 B).
1 parent 57270c6 commit 4d8dbbb

1 file changed

Lines changed: 50 additions & 49 deletions

File tree

doc/packet-format.md

Lines changed: 50 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,8 @@ All multi-byte integers are **big-endian (network byte order)**.
4848
| --- | --- | --- |
4949
| `u8` | 1 byte | Used for the message type tag. |
5050
| `u32` | 4 bytes, big-endian | Used only as a length / count prefix. |
51-
| `u64` | 8 bytes, big-endian | All identifiers, salts, timestamps. |
51+
| `u64` | 8 bytes, big-endian | Salts and timestamps. |
52+
| `NodeId` | 16 raw bytes (fixed) | A node's 128-bit identity — written as 16 bytes, not an integer. |
5253
| `blob` | `u32 length` then *length* raw bytes | Any variable-length byte string (hashes, content, signatures). A zero-length blob is 4 bytes (`length = 0`). |
5354
| `array<T>` | `u32 count` then *count* × `T` | Repeated elements, each encoded in turn. |
5455

@@ -62,7 +63,7 @@ How each logical field type maps onto the primitives above:
6263

6364
| Field type | Wire encoding | Typical size | Meaning |
6465
| --- | --- | --- | --- |
65-
| `NodeId` | `u64` | 8 B | A node's identity. |
66+
| `NodeId` | 16 raw bytes | 16 B | A node's **128-bit** identity — the first 16 bytes of `SHA-256(pubkey)`. |
6667
| `Salt` | `u64` | 8 B | Random salt folded into a hash. |
6768
| `Timestamp` | `u64` | 8 B | A raw clock tick (nanoseconds in production), signed 64-bit reinterpreted as `u64`. |
6869
| `TimeRange` | two `u64` (`lo`, `hi`) | 16 B | A closed `[lo, hi]` window of `Timestamp` ticks — a discovery's estimated time window. |
@@ -78,24 +79,24 @@ How each logical field type maps onto the primitives above:
7879

7980
## Datagram header
8081

81-
**Every** datagram begins with the same 9-byte header (`write_header` in
82+
**Every** datagram begins with the same 17-byte header (`write_header` in
8283
[`packets.cpp`](../src/core/wire/packets.cpp)):
8384

8485
| Offset | Size | Field | Encoding | Meaning |
8586
| --- | --- | --- | --- | --- |
8687
| 0 | 1 B | `type` | `u8` | Message type: `0` = clock notification, `1` = chain request, `2` = chain response. Selects how the rest of the datagram is parsed. |
87-
| 1 | 8 B | `sender` | `u64` | The **sender's** own `NodeId`. Lets the receiver identify the neighbor; the payload that follows never repeats it. |
88+
| 1 | 16 B | `sender` | `NodeId` | The **sender's** own `NodeId`. Lets the receiver identify the neighbor; the payload that follows never repeats it. |
8889

8990
```
90-
0 1 9
91-
+--------+-----------------------------------+
92-
| type | sender |
93-
| u8 | u64 |
94-
+--------+-----------------------------------+
95-
1 B 8 B
91+
0 1 17
92+
+--------+------------------------------------------+
93+
| type | sender |
94+
| u8 | NodeId (16 B) |
95+
+--------+------------------------------------------+
96+
1 B 16 B
9697
```
9798

98-
**Header size: 9 bytes** (fixed, on every datagram).
99+
**Header size: 17 bytes** (fixed, on every datagram).
99100

100101
The three `type` values and their payloads:
101102

@@ -115,7 +116,7 @@ node's clock DAG. A node runs several independent, geometrically-spaced clock ch
115116
[implementation.md](implementation.md#multi-resolution-clock-chains)), so the notification
116117
carries a `chain` id alongside the two hashes — never the clock event's contents.
117118

118-
Payload after the 9-byte header:
119+
Payload after the 17-byte header:
119120

120121
| Field | Encoding | Meaning |
121122
| --- | --- | --- |
@@ -124,13 +125,13 @@ Payload after the 9-byte header:
124125
| `neighbor_last_clock_event_hash` | `blob` (EventHash) | Hash of the newest clock event the sender has heard **from this neighbor on that chain** — the back-reference that stitches the two DAGs together. |
125126

126127
```
127-
+----------- header 9 B -----------+---- u64 ----+------ blob ------+------ blob ------+
128-
| type=0 | sender u64 | chain | last_clock_hash | neighbor_last… |
129-
+--------+-------------------------+-------------+------------------+------------------+
130-
8 B \--- 4 + 32 B ---/ \--- 4 + 32 B ---/
128+
+---------- header 17 B ----------+---- u64 ----+------ blob ------+------ blob ------+
129+
| type=0 | sender NodeId | chain | last_clock_hash | neighbor_last… |
130+
+--------+------------------------+-------------+------------------+------------------+
131+
16 B 8 B \--- 4 + 32 B ---/ \--- 4 + 32 B ---/
131132
```
132133

133-
**Size (SHA-256 hashes): 9 + 8 + 36 + 36 = 89 bytes** (fixed).
134+
**Size (SHA-256 hashes): 17 + 8 + 36 + 36 = 97 bytes** (fixed).
134135

135136
On receipt, the node records the neighbor's newest hash for that chain, and, if it recognizes
136137
`neighbor_last_clock_event_hash` as one of its own clock events, records the reverse cross-link.
@@ -143,12 +144,12 @@ On receipt, the node records the neighbor's newest hash for that chain, and, if
143144
create, or an intermediate node forwards such a request one hop closer to the event's creator.
144145
Routed toward the creator via each node's next-hop table.
145146

146-
Payload after the 9-byte header:
147+
Payload after the 17-byte header:
147148

148149
| Field | Encoding | Meaning |
149150
| --- | --- | --- |
150-
| `originator` | `u64` (NodeId) | The node that started the discovery; the response is routed back to it. Unchanged across every hop. |
151-
| `event.creator` | `u64` (NodeId) | Creator of the target event — the routing destination. |
151+
| `originator` | `NodeId` (16 B) | The node that started the discovery; the response is routed back to it. Unchanged across every hop. |
152+
| `event.creator` | `NodeId` (16 B) | Creator of the target event — the routing destination. |
152153
| `event.hash` | `blob` (EventHash) | Hash of the target event. |
153154
| `range` | `TimeRange` (16 B) | The querying party's estimated time window for the event; forwarding routes over the overlay as it was within it. |
154155
| `hop_limit` | `u64` | Maximum forward hops (`0` = unlimited) — bounds a flood's depth. |
@@ -157,15 +158,15 @@ Payload after the 9-byte header:
157158
(`event.creator` + `event.hash` together are an `EventReference`, inlined directly into the datagram.)
158159

159160
```
160-
+--- header 9 B ---+-- u64 --+-- u64 --+- blob -+- TimeRange -+-- u64 --+- array<NodeId> -+
161-
| type=1 | sender | orig. | creator | e.hash | lo | hi | hop_lim | u32 n · n·u64 |
162-
+--------+---------+---------+---------+--------+-------------+---------+-----------------+
163-
8 B 8 B 4 + 32 B 16 B 8 B 4 + 8·n (breadcrumb)
161+
+-- header 17 B --+-- NodeId --+-- NodeId --+- blob -+- TimeRange -+-- u64 --+- array<NodeId> --+
162+
| type=1 | sender | orig. 16 B | creat. 16 B| e.hash | lo | hi | hop_lim | u32 n · n·16 B |
163+
+--------+--------+------------+------------+--------+-------------+---------+------------------+
164+
16 B 16 B 4 + 32 B 16 B 8 B 4 + 16·n (breadcrumb)
164165
```
165166

166-
**Size (SHA-256 hash): 9 + 8 + 8 + 36 + 16 + 8 + (4 + 8·*h*) = 89 + 8·*h* bytes**, where *h* is the
167-
breadcrumb length (one entry per hop travelled so far). *Now variable* — it was 61 B fixed before
168-
the time-dependent routing / breadcrumb fields were added.
167+
**Size (SHA-256 hash): 17 + 16 + 16 + 36 + 16 + 8 + (4 + 16·*h*) = 113 + 16·*h* bytes**, where *h*
168+
is the breadcrumb length (one entry per hop travelled so far). Variable — one 16-byte `NodeId` is
169+
appended to the breadcrumb at each hop.
169170

170171
Routing: if `event.creator` is this node, it builds the enclosing chain locally and replies with a
171172
chain response. Otherwise it appends itself to `path` and forwards a copy to **each** next hop its
@@ -182,22 +183,22 @@ way back, **extends** the chain with its own clock-event bounds before forwardin
182183
to the originator — **along the reverse-path breadcrumb the request recorded**, not via a routing
183184
table (the response never re-routes).
184185

185-
Payload after the 9-byte header:
186+
Payload after the 17-byte header:
186187

187188
| Field | Encoding | Meaning |
188189
| --- | --- | --- |
189-
| `originator` | `u64` (NodeId) | Copied from the request; identifies the discovery. The response is *home* when it reaches this node. |
190+
| `originator` | `NodeId` (16 B) | Copied from the request; identifies the discovery. The response is *home* when it reaches this node. |
190191
| `chain` | `EventChain` (see below) | The accreting proof: the target event plus a lower- and upper-bound run of clock events. Grows at each hop. |
191192
| `path` | `array<NodeId>` | The **remaining** reverse-path breadcrumb toward the originator: the next hop is `path.back()`, popped at each hop; empty once the response reaches the originator. |
192193

193194
```
194-
+----------- header 9 B -----------+-- u64 --+-- EventChain (var) --+- array<NodeId> -+
195-
| type=2 | sender u64 | orig. | event·lower[]·upper[]| u32 n · n·u64 |
196-
+--------+-------------------------+---------+----------------------+-----------------+
197-
8 B (variable) 4 + 8·n (remaining)
195+
+---------- header 17 B ----------+-- NodeId --+-- EventChain (var) --+- array<NodeId> --+
196+
| type=2 | sender NodeId | orig. 16 B | event·lower[]·upper[]| u32 n · n·16 B |
197+
+--------+------------------------+------------+----------------------+------------------+
198+
16 B 16 B (variable) 4 + 16·n (remaining)
198199
```
199200

200-
**Size: 9 + 8 + `sizeof(EventChain)` + (4 + 8·*h*) bytes** (variable — the largest datagram),
201+
**Size: 17 + 16 + `sizeof(EventChain)` + (4 + 16·*h*) bytes** (variable — the largest datagram),
201202
where *h* is the remaining breadcrumb length. The size is dominated by the number of clock events
202203
accreted into the chain: one per hop on each side, each further inflated by its referenced-event
203204
list and (if present) its signature.
@@ -209,27 +210,27 @@ list and (if present) its signature.
209210
These structures appear only within `EventChain`. Field order below is the exact byte order the
210211
codec writes ([`codec.hpp`](../src/core/wire/codec.hpp)).
211212

212-
### `EventReference`44 B (SHA-256, fixed)
213+
### `EventReference`52 B (SHA-256, fixed)
213214

214215
A typed pointer to an event or clock event.
215216

216217
| Field | Encoding | Size |
217218
| --- | --- | --- |
218-
| `creator` | `u64` | 8 B |
219+
| `creator` | `NodeId` | 16 B |
219220
| `hash` | `blob` (EventHash) | 4 + 32 B |
220221

221-
### `Event`64 + *d* + 44·*r* B (unsigned, SHA-256)
222+
### `Event`72 + *d* + 52·*r* B (unsigned, SHA-256)
222223

223224
The target content event at the center of a chain. *d* = content length, *r* = referenced-event
224225
count.
225226

226227
| Field | Encoding | Size |
227228
| --- | --- | --- |
228-
| `creator` | `u64` | 8 B |
229+
| `creator` | `NodeId` | 16 B |
229230
| `hash` | `blob` (EventHash) | 4 + 32 B |
230231
| `data` | `blob` | 4 + *d* B |
231232
| `salt` | `u64` | 8 B |
232-
| `referenced_events` | `array<EventReference>` | 4 + 44·*r* B |
233+
| `referenced_events` | `array<EventReference>` | 4 + 52·*r* B |
233234
| `signature` | `blob` (Signature) | 4 B unsigned · 68 B signed |
234235

235236
> **The event content `data` is on the wire.** A chain response transmits the target event's
@@ -238,20 +239,20 @@ count.
238239
> deliberately **excludes** `data` — that is a modeling choice for size statistics, not a claim
239240
> about the encoded bytes. The actual datagram includes it.)
240241
241-
### `ClockEvent`76 + 44·*r* B (unsigned, SHA-256)
242+
### `ClockEvent`84 + 52·*r* B (unsigned, SHA-256)
242243

243244
A timestamped node of the clock DAG; a chain carries many of them. *r* = referenced-event count.
244245
`chain` — which of the node's multi-resolution clock chains this event belongs to (0 = fastest)
245246
— is part of the hashed content, so a clock event cannot lie about its chain.
246247

247248
| Field | Encoding | Size |
248249
| --- | --- | --- |
249-
| `creator` | `u64` | 8 B |
250+
| `creator` | `NodeId` | 16 B |
250251
| `hash` | `blob` (EventHash) | 4 + 32 B |
251252
| `chain` | `u64` | 8 B |
252253
| `timestamp` | `u64` | 8 B |
253254
| `salt` | `u64` | 8 B |
254-
| `referenced_events` | `array<EventReference>` | 4 + 44·*r* B |
255+
| `referenced_events` | `array<EventReference>` | 4 + 52·*r* B |
255256
| `signature` | `blob` (Signature) | 4 B unsigned · 68 B signed |
256257

257258
### `EventChain` — variable
@@ -281,9 +282,9 @@ adds 64 bytes per signed `Event`/`ClockEvent`.
281282

282283
| Datagram | `type` | Size | Fixed? |
283284
| --- | --- | --- | --- |
284-
| Clock notification | 0 | 89 B | ✔ fixed |
285-
| Chain request | 1 | 89 + 8·*h* B (*h* = breadcrumb hops) | ✘ variable |
286-
| Chain response | 2 | 21 + 8·*h* B + `EventChain` | ✘ variable |
285+
| Clock notification | 0 | 97 B | ✔ fixed |
286+
| Chain request | 1 | 113 + 16·*h* B (*h* = breadcrumb hops) | ✘ variable |
287+
| Chain response | 2 | 37 + 16·*h* B + `EventChain` | ✘ variable |
287288

288289
(UDP + IPv4 headers are added by the OS network stack and are not part of these figures.)
289290

@@ -296,8 +297,8 @@ hangs on. Importantly, the hash is computed over a **fixed canonical layout that
296297
the transport encoding above** (defined in [`src/core/hash/hashing.hpp`](../src/core/hash/hashing.hpp)) —
297298
so a node can recompute and verify any hash it receives:
298299

299-
- **Event hash** = `SHA-256( data ‖ salt(u64 BE) ‖ for each referenced event: creator(u64 BE) ‖ hash bytes )`.
300-
- **Clock-event hash** = `SHA-256( chain(u64 BE) ‖ timestamp(u64 BE) ‖ salt(u64 BE) ‖ for each referenced event: creator(u64 BE) ‖ hash bytes )`.
300+
- **Event hash** = `SHA-256( data ‖ salt(u64 BE) ‖ for each referenced event: creator(16 B) ‖ hash bytes )`.
301+
- **Clock-event hash** = `SHA-256( chain(u64 BE) ‖ timestamp(u64 BE) ‖ salt(u64 BE) ‖ for each referenced event: creator(16 B) ‖ hash bytes )`.
301302

302303
A node's own `creator`/`hash` fields and its `signature` are **not** part of the preimage — the
303304
signature signs the hash, so signing never changes a hash. This is why the `data` bytes must
@@ -349,8 +350,8 @@ Originator O Intermediate I Creator C
349350
```
350351

351352
Notes:
352-
- The request carries only an event reference (fixed 61 B); the response carries the full,
353-
growing chain (variable).
353+
- The request carries only an event reference plus the breadcrumb (`113 + 16·h` B); the response
354+
carries the full, growing chain (variable, the largest datagram).
354355
- If the originator itself created the event, the chain is built entirely locally with **no
355356
datagrams** at all.
356357
- *Event bounds* and *event order* discoveries send **no new datagram types** — they are built on

0 commit comments

Comments
 (0)