Skip to content

Commit f354a64

Browse files
chaitanyapremchaitanyaprem
authored
Merge branch 'main' into mix-rev1-security
2 parents a9c1bb3 + 9d11a22 commit f354a64

19 files changed

+6039
-62
lines changed

‎codex/raw/codex-block-exchange.md‎

Lines changed: 485 additions & 0 deletions
Large diffs are not rendered by default.

‎codex/raw/codex-marketplace.md‎

Lines changed: 802 additions & 0 deletions
Large diffs are not rendered by default.

‎nomos/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22

33
Nomos is building a secure, flexible, and
44
scalable infrastructure for developers creating applications for the network state.
5-
To learn more about Nomos current protocols under discussion,
6-
head over to [Nomos Specs](https://github.com/logos-co/nomos-specs).
5+
Published Specifications are currently available here,
6+
[Nomos Specifications](https://nomos-tech.notion.site/project).

‎nomos/raw/nomosda-encoding.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
---
2+
title: NOMOSDA-ENCODING
3+
name: NomosDA Encoding Protocol
4+
status: raw
5+
category:
6+
tags: data-availability
7+
editor: Daniel Sanchez-Quiros <[email protected]>
8+
contributors:
9+
- Daniel Kashepava <[email protected]>
10+
- Álvaro Castro-Castilla <[email protected]>
11+
- Filip Dimitrijevic <[email protected]>
12+
- Thomas Lavaur <[email protected]>
13+
- Mehmet Gonen <[email protected]>
14+
---
15+
16+
## Introduction
17+
18+
This document describes the encoding and verification processes of NomosDA, which is the data availability (DA) solution used by the Nomos blockchain. NomosDA provides an assurance that all data from Nomos blobs are accessible and verifiable by every network participant.
19+
20+
This document presents an implementation specification describing how:
21+
22+
- Encoders encode blobs they want to upload to the Data Availability layer.
23+
- Other nodes implement the verification of blobs that were already uploaded to DA.
24+
25+
## Definitions
26+
27+
- **Encoder**: An encoder is any actor who performs the encoding process described in this document. This involves committing to the data, generating proofs, and submitting the result to the DA layer.
28+
29+
In the Nomos architecture, the rollup sequencer typically acts as the encoder, but the role is not exclusive and any actor in the DA layer can also act as encoders.
30+
- **Verifier**: Verifies its portion of the distributed blob data as per the verification protocol. In the Nomos architecture, the DA nodes act as the verifiers.
31+
32+
## Overview
33+
34+
In the encoding stage, the encoder takes the DA parameters and the padded blob data and creates an initial matrix of data chunks. This matrix is expanded using Reed-Solomon coding and various commitments and proofs are created for the data.
35+
36+
When a verifier receives a sample, it verifies the data it receives from the encoder and broadcasts the information if the data is verified. Finally, the verifier stores the sample data for the required length of time.
37+
38+
## Construction
39+
40+
The encoder and verifier use the [NomosDA cryptographic protocol](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21) to carry out their respective functions. These functions are implemented as abstracted and configurable software entities that allow the original data to be encoded and verified via high-level operations.
41+
42+
### Glossary
43+
44+
| Name | Description | Representation |
45+
| --- | --- | --- |
46+
| `Commitment` | Commitment as per the [NomosDA Cryptographic Protocol](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21) | `bytes` |
47+
| `Proof` | Proof as per the [NomosDA Cryptographic Protocol](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21) | `bytes` |
48+
| `ChunksMatrix` | Matrix of chunked data. Each chunk is **31 bytes.** Row and Column sizes depend on the encoding necessities. | `List[List[bytes]]` |
49+
50+
### Encoder
51+
52+
An encoder takes a set of parameters and the blob data, and creates a matrix of chunks that it uses to compute the necessary cryptographic data. It produces the set of Reed-Solomon (RS) encoded data, the commitments, and the proofs that are needed prior to [dispersal](https://www.notion.so/NomosDA-Dispersal-1fd261aa09df815288c9caf45ed72c95?pvs=21).
53+
54+
```mermaid
55+
flowchart LR
56+
A[DaEncoderParams] -->|Input| B(Encoder)
57+
I[31bytes-padded-input] -->|Input| B
58+
B -->|Creates| D[Chunks matrix]
59+
D --> |Input| C[NomosDA encoding]
60+
C --> E{Encoded data📄}
61+
```
62+
63+
#### Encoding Process
64+
65+
The encoder executes the encoding process as follows:
66+
67+
1. The encoder takes the following input parameters:
68+
69+
```python
70+
class DAEncoderParams:
71+
column_count: usize
72+
bytes_per_field_element: usize
73+
```
74+
75+
| Name | Description | Representation |
76+
| --- | --- | --- |
77+
| `column_count` | The number of subnets available for dispersal in the system | `usize`, `int` in Python |
78+
| `bytes_per_field_element` | The amount of bytes per data chunk. This is set to 31 bytes. Each chunk has 31 bytes rather than 32 to ensure that the chunk value does not exceed the maximum value on the [BLS12-381 elliptic curve](https://electriccoin.co/blog/new-snark-curve/). | `usize`, `int` in Python |
79+
80+
2. The encoder also includes the blob data to be encoded, which must be of a size that is a multiple of `bytes_per_field_element` bytes. Clients are responsible for padding the data so it fits this constraint.
81+
3. The encoder splits the data into `bytes_per_field_element`-sized chunks. It also arranges these chunks into rows and columns, creating a matrix.
82+
a. The amount of columns of the matrix needs to fit with the `column_count` parameter, taking into account the `rs_expansion_factor` (currently fixed to 2).
83+
i. This means that the size of each row in this matrix is `(bytes_per_field_element*column_count)/rs_expansion_factor`.
84+
b. The amount of rows depends on the size of the data.
85+
4. The data is encoded as per [the cryptographic details](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21).
86+
5. The encoder provides the encoded data set:
87+
88+
| Name | Description | Representation |
89+
| --- | --- | --- |
90+
| `data` | Original data | `bytes` |
91+
| `chunked_data` | Matrix before RS expansion | `ChunksMatrix` |
92+
| `extended_matrix` | Matrix after RS expansion | `ChunksMatrix` |
93+
| `row_commitments` | Commitments for each matrix row | `List[Commitment]` |
94+
| `combined_column_proofs` | Proofs for each matrix column | `List[Proof]` |
95+
96+
```python
97+
class EncodedData:
98+
data: bytes
99+
chunked_data: ChunksMatrix
100+
extended_matrix: ChunksMatrix
101+
row_commitments: List[Commitment]
102+
combined_column_proofs: List[Proof]
103+
```
104+
105+
#### Encoder Limits
106+
107+
NomosDA does not impose a fixed limit on blob size at the encoding level. However, protocols that involve resource-intensive operations must include upper bounds to prevent abuse. In the case of NomosDA, blob size limits are expected to be enforced, as part of the protocol's broader responsibility for resource management and fairness.
108+
109+
Larger blobs naturally result in higher computational and bandwidth costs, particularly for the encoder, who must compute a proof for each column. Without size limits, malicious clients could exploit the system by attempting to stream unbounded data to DA nodes. Since payment is provided before blob dispersal, DA nodes are protected from performing unnecessary work. This enables the protocol to safely accept very large blobs, as the primary computational cost falls on the encoder. The protocol can accommodate generous blob sizes in practice, while rejecting only absurdly large blobs, such as those exceeding 1 GB, to prevent denial-of-service attacks and ensure network stability.
110+
111+
To mitigate this, the protocol define acceptable blob size limits, and DA implementations enforce local mitigation strategies, such as flagging or blacklisting clients that violate these constraints.
112+
113+
### Verifier
114+
115+
A verifier checks the proper encoding of data blobs it receives. A verifier executes the verification process as follows:
116+
117+
1. The verifier receives a `DAShare` with the required verification data:
118+
119+
| Name | Description | Representation |
120+
| --- | --- | --- |
121+
| `column` | Column chunks (31 bytes) from the encoded matrix | `List[bytes]` |
122+
| `column_idx` | Column id (`0..2047`). It is directly related to the `subnetworks` in the [network specification](https://www.notion.so/NomosDA-Network-Specification-1fd261aa09df81188e76cb083791252d?pvs=21). | `u16`, unsigned int of 16 bits. `int` in Python |
123+
| `combined_column_proof` | Proof of the random linear combination of the column elements. | `Proof` |
124+
| `row_commitments` | Commitments for each matrix row | `List[Commitment]` |
125+
| `blob_id` | This is computed as the hash (**blake2b**) of `row_commitments` | `bytes` |
126+
127+
2. Upon receiving the above data it verifies the column data as per the [cryptographic details](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21). If the verification is successful, the node triggers the [replication protocol](https://www.notion.so/NomosDA-Subnetwork-Replication-1fd261aa09df811d93f8c6280136bfbb?pvs=21) and stores the blob.
128+
129+
```python
130+
class DAShare:
131+
column: Column
132+
column_idx: u16
133+
combined_column_proof: Proof
134+
row_commitments: List[Commitment]
135+
136+
def blob_id(self) -> BlobId:
137+
hasher = blake2b(digest_size=32)
138+
for c in self.row_commitments:
139+
hasher.update(bytes(c))
140+
return hasher.digest()
141+
```
142+
143+
### Verification Logic
144+
145+
```mermaid
146+
sequenceDiagram
147+
participant N as Node
148+
participant S as Subnetwork Column N
149+
loop For each incoming blob column
150+
N-->>N: If blob is valid
151+
N-->>S: Replication
152+
N->>N: Stores blob
153+
end
154+
```
155+
156+
## Details
157+
158+
The encoder and verifier processes described above make use of a variety of cryptographic functions to facilitate the correct verification of column data by verifiers. These functions rely on primitives such as polynomial commitments and Reed-Solomon erasure codes, the details of which are outside the scope of this document. These details, as well as introductions to the cryptographic primitives being used, can be found in the NomosDA Cryptographic Protocol:
159+
160+
[NomosDA Cryptographic Protocol](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21)
161+
162+
## References
163+
164+
- Encoder Specification: [GitHub/encoder.py](https://github.com/logos-co/nomos-specs/blob/master/da/encoder.py)
165+
- Verifier Specification: [GitHub/verifier.py](https://github.com/logos-co/nomos-specs/blob/master/da/verifier.py)
166+
- Cryptographic protocol: [NomosDA Cryptographic Protocol](https://www.notion.so/NomosDA-Cryptographic-Protocol-1fd261aa09df816fa97ac81304732e77?pvs=21)
167+
168+
## Copyright
169+
170+
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

0 commit comments

Comments
 (0)