feat(foundations): migrate DHT and RLDP pages from the old documentation

docs.json

@@ -719,6 +719,13 @@
           "foundations/shards",
           "foundations/limits",
           "foundations/config",
+          {
+            "group": "Network protocols",
+            "pages": [
+              "foundations/network/rldp",
+              "foundations/network/dht"
+            ]
+          },
           {
             "group": "Web3 services",
             "pages": [
@@ -1410,17 +1417,17 @@
     },
     {
       "source": "/v3/documentation/network/protocols/dht/overview",
-      "destination": "https://old-docs.ton.org/v3/documentation/network/protocols/dht/overview",
+      "destination": "/foundations/network/dht",
       "permanent": true
     },
     {
       "source": "/v3/documentation/network/protocols/dht/deep-dive",
-      "destination": "https://old-docs.ton.org/v3/documentation/network/protocols/dht/deep-dive",
+      "destination": "/foundations/network/dht",
       "permanent": true
     },
     {
       "source": "/v3/documentation/network/protocols/rldp",
-      "destination": "https://old-docs.ton.org/v3/documentation/network/protocols/rldp",
+      "destination": "/foundations/network/rldp",
       "permanent": true
     },
     {

foundations/network/dht.mdx

@@ -0,0 +1,179 @@
+---
+title: "Distributed Hash Table"
+description: "Kademlia-like DHT for node discovery, address resolution, and overlay network management in TON"
+sidebarTitle: "DHT"
+---
+
+import { Aside } from "/snippets/aside.jsx";
+
+The TON Distributed Hash Table (DHT) is a Kademlia-like distributed key-value database used for discovering nodes in the network. Any network participant can operate a DHT node, generate keys, and store data.
+
+The DHT is used to store IP addresses of [ADNL](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/overview) nodes, torrent holder lists for [TON Storage](/foundations/web3/ton-storage), overlay subnetwork node addresses, and ADNL addresses of TON services and blockchain accounts.
+
+## Keys and values
+
+Keys are 256-bit integers, typically derived from SHA-256 of a TL-serialized object. Values are arbitrary byte strings of limited length, with meaning determined by the key's pre-image.
+
+In the simplest case, the key represents an ADNL address and the value is an IP address and port.
+
+## DHT nodes
+
+Each DHT node has a 256-bit address that should remain relatively stable. If an address changes too frequently, other nodes cannot locate the keys it stores.
+
+The value of key `K` is stored on the `S` nearest nodes to `K` (typically `S = 7`). This redundancy ensures that the value survives individual nodes going offline.
+
+## Kademlia distance
+
+Distance between a key and a node is computed via 256-bit XOR:
+
+```text
+distance(key, node) = key XOR node_address
+```
+
+This distance is purely computational and unrelated to geographic location. A smaller XOR result means the node is "closer" to the key.
+
+## Routing table
+
+Each DHT node maintains a Kademlia routing table of 256 buckets (numbered 0 to 255). Bucket `i` contains nodes at Kademlia distance `2^i` to `2^(i+1) - 1` from the node's own address.
+
+Each bucket stores a fixed number of "best" nodes plus additional candidates. Stored information includes DHT addresses, IP addresses, UDP ports, and availability data (last ping time, latency).
+
+When a query discovers a new node, it is placed in the appropriate bucket as a candidate. Unresponsive "best" nodes are eventually replaced by candidates, keeping the routing table populated.
+
+## Key-value update rules
+
+Update rules vary by key type:
+
+- **Signature-based**: The new value must be signed by the key owner. The signature is retained for later verification.
+- **Sequence-number-based**: The new value must contain a higher sequence number than the old value, preventing replay attacks.
+- **Unrestricted**: Anyone can update the value.
+
+## Finding a value by key
+
+To look up a value, [connect to any DHT node via ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp) and send a `dht.findValue` query.
+
+### Constructing the key ID
+
+First, build the DHT key using:
+
+```tlb
+dht.key id:int256 name:bytes idx:int = dht.Key
+```
+
+- `id` — the ADNL address or other identifier being searched.
+- `name` — the key type as a byte string (for example, `"address"` for ADNL address lookups, `"nodes"` for shard node lookups).
+- `idx` — 0 when there is only one key.
+
+Serialize this structure and compute its SHA-256 hash — this is the key ID.
+
+### Sending the query
+
+```tlb
+dht.findValue key:int256 k:int = dht.ValueResult
+```
+
+- `key` — the key ID from the previous step.
+- `k` — search width. Smaller values produce more precise results but fewer candidate nodes.
+
+The response is either [`dht.valueFound`](#dhtvaluefound) or [`dht.valueNotFound`](#dhtvaluenotfound).
+
+### `dht.valueNotFound`
+
+The response includes a list of nodes that are close to the requested key:
+
+```tlb
+dht.node id:PublicKey addr_list:adnl.addressList version:int signature:bytes = dht.Node;
+dht.nodes nodes:(vector dht.node) = dht.Nodes;
+dht.valueNotFound nodes:dht.nodes = dht.ValueResult;
+```
+
+For each node:
+
+1. Verify the signature: read it, set the field to empty bytes, serialize the `dht.node`, and verify the signature against the `id` public key.
+1. Connect via [ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp) using the address from `addr_list` and the `id` as the server key.
+1. Compute the XOR distance between the node's [key ID](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/tcp) and the target key. If the distance is small enough, repeat the query.
+
+Continue until a value is found or no new nodes remain.
+
+### `dht.valueFound`
+
+```tlb
+dht.keyDescription key:dht.key id:PublicKey update_rule:dht.UpdateRule signature:bytes = dht.KeyDescription;
+dht.value key:dht.keyDescription value:bytes ttl:int signature:bytes = dht.Value;
+dht.valueFound value:dht.Value = dht.ValueResult;
+```
+
+The `key:dht.keyDescription` provides full key metadata:
+
+- `key:dht.key` — must match the searched key.
+- `id:PublicKey` — the record owner's public key.
+- `update_rule:dht.UpdateRule` — determines who can modify this record:
+
+| Rule                          | Description                                                                       |
+| ----------------------------- | --------------------------------------------------------------------------------- |
+| `dht.updateRule.signature`    | Only the private key owner can update; both key and value signatures are verified |
+| `dht.updateRule.anybody`      | Anyone can update; signatures are not checked                                     |
+| `dht.updateRule.overlayNodes` | Nodes from the same overlay can update                                            |
+
+### Verifying signature-based values
+
+For `dht.updateRule.signature` (used in ADNL address lookups):
+
+1. Set the `signature` field of `dht.keyDescription` to empty bytes, serialize, and verify against the `id` public key.
+1. Restore the key signature, then set the `signature` field of `dht.value` to empty bytes, serialize, and verify.
+
+[Implementation example](https://github.com/xssnick/tonutils-go/blob/46dbf5f820af066ab10c5639a508b4295e5aa0fb/adnl/dht/client.go#L331).
+
+### Verifying overlay node values
+
+For `dht.updateRule.overlayNodes`, the value contains:
+
+```tlb
+overlay.node id:PublicKey overlay:int256 version:int signature:bytes = overlay.Node;
+overlay.nodes nodes:(vector overlay.node) = overlay.Nodes;
+```
+
+For each node, verify the `signature` by serializing:
+
+```tlb
+overlay.node.toSign id:adnl.id.short overlay:int256 version:int = overlay.node.ToSign;
+```
+
+Replace `id` with `adnl.id.short` (the key ID hash of the original `id` field) and verify the signature.
+
+### Using the value
+
+Once verified and the `ttl` has not expired, extract `value:bytes`. For ADNL address lookups, this contains an `adnl.addressList` with IP addresses and ports. Use the `id:PublicKey` from the key description as the server key when connecting.
+
+## Finding blockchain nodes
+
+The DHT is also used to discover nodes storing blockchain data for specific workchains and shards.
+
+### Building the overlay key
+
+1. Fill in the TL structure:
+
+   ```tlb
+   tonNode.shardPublicOverlayId workchain:int shard:long zero_state_file_hash:int256 = tonNode.ShardPublicOverlayId;
+   ```
+
+   For the masterchain: `workchain = -1`, `shard = -9223372036854775808` (`0x8000000000000000`). The `zero_state_file_hash` comes from the `validator > zero_state` section of the global config.
+
+1. Serialize and hash to get the key ID.
+
+1. Wrap this key ID in `pub.overlay name:bytes = PublicKey`, serialize, and hash again to get the final key.
+
+1. Use this key with `dht.findValue`, setting `name` to `"nodes"` in the `dht.key`.
+
+The result uses `dht.updateRule.overlayNodes`. After validation, the returned public keys identify nodes with blockchain data. Hash these keys to get ADNL addresses, then look up each address in the DHT to obtain connection details.
+
+To discover additional nodes in the same shard, use [`overlay.getRandomPeers`](https://github.com/ton-blockchain/ton/blob/ad736c6bc3c06ad54dc6e40d62acbaf5dae41584/tl/generate/scheme/ton_api.tl#L237).
+
+## See also
+
+- [ADNL specification](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/overview)
+- [ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp)
+- [DHT implementation](https://github.com/ton-blockchain/ton/tree/master/dht)
+- [DHT server implementation](https://github.com/ton-blockchain/ton/tree/master/dht-server)
+- [The Open Network Whitepaper, Chapter 3.2](https://ton.org/whitepaper.pdf)
+- [Original article by Oleg Baranov](https://github.com/xssnick/ton-deep-doc/blob/master/DHT.md)

foundations/network/rldp.mdx

@@ -0,0 +1,149 @@
+---
+title: "Reliable Large Datagram Protocol"
+description: "Protocol for transferring large data blocks over ADNL UDP using Forward Error Correction"
+sidebarTitle: "RLDP"
+---
+
+import { Aside } from "/snippets/aside.jsx";
+
+The Reliable Large Datagram Protocol (RLDP) operates on top of [ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp) and is designed for transferring large data blocks. It uses Forward Error Correction (FEC) instead of acknowledgment packets, enabling more efficient data transfer at the cost of increased traffic.
+
+RLDP is used throughout the TON infrastructure: downloading blocks from other nodes, transferring data, and accessing [TON Sites](/foundations/web3/ton-sites) and [TON Storage](/foundations/web3/ton-storage).
+
+## Protocol
+
+RLDP uses the following TL structures:
+
+```tlb
+fec.raptorQ data_size:int symbol_size:int symbols_count:int = fec.Type;
+fec.roundRobin data_size:int symbol_size:int symbols_count:int = fec.Type;
+fec.online data_size:int symbol_size:int symbols_count:int = fec.Type;
+
+rldp.messagePart transfer_id:int256 fec_type:fec.Type part:int total_size:long seqno:int data:bytes = rldp.MessagePart;
+rldp.confirm transfer_id:int256 part:int seqno:int = rldp.MessagePart;
+rldp.complete transfer_id:int256 part:int = rldp.MessagePart;
+
+rldp.message id:int256 data:bytes = rldp.Message;
+rldp.query query_id:int256 max_answer_size:long timeout:int data:bytes = rldp.Message;
+rldp.answer query_id:int256 data:bytes = rldp.Message;
+```
+
+All structures are serialized, wrapped in `adnl.message.custom`, and transmitted over [ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp).
+
+## Transfer mechanism
+
+1. Generate a random `transfer_id`.
+1. Process the data through the FEC algorithm to produce encoded symbols.
+1. Wrap each symbol in `rldp.messagePart` and send to the peer.
+1. Continue sending until the peer responds with `rldp.complete` or a timeout occurs.
+
+The receiver collects `rldp.messagePart` messages, and once enough symbols are gathered, decodes them via FEC and deserializes the result into `rldp.query` or `rldp.answer`.
+
+## Forward Error Correction
+
+Three FEC algorithms are supported: RoundRobin, Online, and RaptorQ. Currently, [RaptorQ](https://www.qualcomm.com/media/documents/files/raptorq-technical-overview.pdf) is the standard choice.
+
+### RaptorQ
+
+RaptorQ divides data into fixed-size symbols (blocks). These symbols are organized into matrices where discrete mathematical operations produce an effectively unlimited number of encoded symbols from the same source data.
+
+This allows the receiver to recover lost packets without requesting retransmission — fewer packets are needed than if raw data segments were simply resent.
+
+Symbols are transmitted until the peer confirms that all data has been received and successfully decoded.
+
+[RaptorQ implementation in Go](https://github.com/xssnick/tonutils-go/tree/46dbf5f820af066ab10c5639a508b4295e5aa0fb/adnl/rldp/raptorq).
+
+## RLDP-HTTP
+
+[TON Sites](/foundations/web3/ton-sites) use RLDP to wrap HTTP traffic. The host runs a standard HTTP web server with `rldp-http-proxy` alongside it. Incoming requests from the TON network arrive via RLDP, the proxy converts them to HTTP and calls the local web server. On the client side, a local proxy (such as [Tonutils Proxy](https://github.com/xssnick/TonUtils-Proxy)) converts outgoing HTTP requests into RLDP.
+
+HTTP over RLDP uses these TL structures:
+
+```tlb
+http.header name:string value:string = http.Header;
+http.payloadPart data:bytes trailer:(vector http.header) last:Bool = http.PayloadPart;
+http.response http_version:string status_code:int reason:string headers:(vector http.header) no_payload:Bool = http.Response;
+
+http.request id:int256 method:string url:string http_version:string headers:(vector http.header) = http.Response;
+http.getNextPayloadPart id:int256 seqno:int max_chunk_size:int = http.PayloadPart;
+```
+
+The request-response flow:
+
+1. Client sends `http.request` via `rldp.query`.
+1. Server checks the `Content-Length` header:
+   - If non-zero, server sends `http.getNextPayloadPart` requests to the client.
+   - Client responds with `http.payloadPart` chunks, incrementing `seqno` until `last` is true.
+1. Server sends `http.response`.
+1. Client checks `Content-Length`:
+   - If non-zero (`no_payload` is false), client sends `http.getNextPayloadPart` requests to the server using the same chunked mechanism.
+
+## Example: requesting a TON Site
+
+This example demonstrates requesting `foundation.ton`, assuming the ADNL address has been resolved via DNS and a connection has been established over [ADNL UDP](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/udp).
+
+### Sending a GET request
+
+Serialize `http.request`:
+
+```text
+e191b161                                                           -- TL ID http.request
+116505dac8a9a3cdb464f9b5dd9af78594f23f1c295099a9b50c8245de471194   -- id (random)
+03 474554                                                          -- method = "GET"
+16 687474703a2f2f666f756e646174696f6e2e746f6e2f 00                 -- url = "http://foundation.ton/"
+08 485454502f312e31 000000                                         -- http_version = "HTTP/1.1"
+01000000                                                           -- headers (1 entry)
+   04 486f7374 000000                                              -- name = "Host"
+   0e 666f756e646174696f6e2e746f6e 00                              -- value = "foundation.ton"
+```
+
+Wrap in `rldp.query`:
+
+```text
+694d798a                                                              -- TL ID rldp.query
+184c01cb1a1e4dc9322e5cabe8aa2d2a0a4dd82011edaf59eb66f3d4d15b1c5c      -- query_id (random)
+0004040000000000                                                      -- max_answer_size = 257 KB
+258f9063                                                              -- timeout (unix timestamp)
+34 ...                                                                -- data (serialized http.request)
+```
+
+### Encoding with RaptorQ
+
+1. Choose a symbol size (typically 768 bytes).
+1. Pad the data to a multiple of the symbol size with zero bytes. For 156 bytes of data, add 612 zero bytes to reach 768.
+1. Create a RaptorQ encoder ([example](https://github.com/xssnick/tonutils-go/blob/46dbf5f820af066ab10c5639a508b4295e5aa0fb/adnl/rldp/raptorq/encoder.go#L15)).
+
+### Sending packets
+
+Send encoded symbols in `rldp.messagePart`:
+
+- `transfer_id` — random int256, same for all parts of this transfer.
+- `fec_type` — `fec.raptorQ` with `data_size = 156`, `symbol_size = 768`, `symbols_count = 1`.
+- `part` — 0 (used for very large transfers that exceed size limits).
+- `total_size` — 156 (actual data size).
+- `seqno` — starts at 0, incremented for each packet.
+- `data` — encoded symbol (768 bytes).
+
+Wrap each in `adnl.message.custom` and send over ADNL UDP. Send symbols in round-robin order, continuing until `rldp.complete` is received. After sending one full round of symbols, slow down to one packet per 10 milliseconds — extra packets serve as FEC recovery data.
+
+[Implementation example](https://github.com/xssnick/tonutils-go/blob/be3411cf412f23e6889bf0b648904306a15936e7/adnl/rldp/rldp.go#L249).
+
+### Processing the response
+
+The response arrives as `rldp.messagePart` messages inside `adnl.message.custom`. The `transfer_id` is the original transfer ID with each byte XORed with `0xFF`.
+
+1. Extract FEC parameters (`data_size`, `symbol_size`, `symbols_count`) from the first received message.
+1. Initialize a RaptorQ decoder ([example](https://github.com/xssnick/tonutils-go/blob/be3411cf412f23e6889bf0b648904306a15936e7/adnl/rldp/rldp.go#L137)).
+1. Feed received symbols with their `seqno` into the decoder.
+1. Once `symbols_count` symbols are collected, attempt decoding. On success, send `rldp.complete` ([example](https://github.com/xssnick/tonutils-go/blob/be3411cf412f23e6889bf0b648904306a15936e7/adnl/rldp/rldp.go#L168)).
+
+The decoded result is `rldp.answer` containing `http.response`. If `no_payload` is false, retrieve the body by sending `http.getNextPayloadPart` requests with incrementing `seqno` and `max_chunk_size` of 128 KiB (131,072 bytes) until `last` is true.
+
+## See also
+
+- [ADNL specification](https://old-docs.ton.org/v3/documentation/network/protocols/adnl/overview)
+- [TON Proxy](/foundations/web3/ton-proxy)
+- [TON Sites](/foundations/web3/ton-sites)
+- [RLDP implementation](https://github.com/ton-blockchain/ton/tree/master/rldp)
+- [RLDP v2 implementation](https://github.com/ton-blockchain/ton/tree/master/rldp2)
+- [Original article by Oleg Baranov](https://github.com/xssnick/ton-deep-doc/blob/master/RLDP.md)