Skip to content
This repository was archived by the owner on Jan 31, 2025. It is now read-only.

Commit c76f1a9

Browse files
technicallytytechnicallyty
authored
docs (#575)
1 parent f1cde2a commit c76f1a9

File tree

9 files changed

+1535
-23
lines changed

9 files changed

+1535
-23
lines changed

‎docs/0-integrate-the-sdk.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Integrate the Block SDK
2+
3+
The Block SDK is **open-source software** licensed under MIT. It is free to use, and has existing plug-and-play Lanes that work immediately!
4+
5+
Visit the GitHub repo [here](https://github.com/skip-mev/block-sdk).
6+
7+
We strive to be responsive to questions and issues within 1-2 weeks - please open a GitHub issue or join us on [**discord**](skip.build/discord). Note, we are not currently providing hands-on support for new integrations.
8+
9+
10+
## ⚙️ Architecture [15 mins]
11+
12+
This is a high-level overview of the architecture, please reference [this page](2-how-it-works.md) or the [`Block-SDK` repo](https://github.com/skip-mev/block-sdk) for detailed and up to date info. For those eager to code, feel free to skip this and start down the page at **Set Up**!
13+
14+
15+
### How Were Blocks Constructed pre-Block-SDK?
16+
17+
There are 3 relevant stages of consensus (these are all ABCI++ methods)
18+
19+
- **PrepareProposal**
20+
- In this step, the consensus-engine (CometBFT, etc.) gives the application all of the transactions it has seen thus far.
21+
- The app looks over these, performs some app-specific logic, and then gives them back to the consensus-engine. The consensus-engine then creates and broadcasts a proposal containing the transactions sent back from the app.
22+
- **ProcessProposal**
23+
- In this step, all validators check that the transactions in the proposal are valid, and that the proposal (as a whole) satisfies validity conditions determined by the application
24+
- If the proposal fails, validators will not vote on the block, and the network will be forced to another round of consensus
25+
- if the proposal passes, valdiators vote on the block, and the block will become canonical (barring unforeseen events)
26+
27+
### **Application Mempools**
28+
29+
In `v0.47.0` of the cosmos-sdk, **app-side mempools** were added to the SDK. With app-side mempools, validators no longer need to rely on the consensus-engine to keep track of and order all available transactions. Now applications can define their own mempool implementations, that
30+
31+
1. Store all pending (not finalized in a block) transactions
32+
2. Order the set of pending transactions
33+
34+
#### **How does block-building change?**
35+
36+
Now in **PrepareProposal** instead of getting transactions from the consensus-engine, validators can pull transactions from their application-state aware mempools, and prioritize those transactions instead of the consensus-engine's transactions.
37+
38+
**Why is this better?**
39+
40+
- Mempools that are not app-state aware will not have the ability to make state-aware ordering rules. Like
41+
42+
1. All staker transactions are placed at the top of the block
43+
2. All IBC `LightClientUpdate` messages are placed at the top of the block
44+
3. Anything you can think of!!
45+
46+
- The consensus engine's mempool is generally in-efficient.
47+
- The consensus-engine's mempool does not know when to remove transactions from its own mempool
48+
- The consensus-engine spends most of its time re-broadcasting transactions between peers, hogging network bandwidth
49+
50+
## Block-SDK!!
51+
52+
The `Block-SDK` defines its own custom implementation of an **app-side mempool**, a `LaneMempool`. The `LaneMempool` is composed of `Lanes`, and handles transaction ingress, ordering, and cleaning.
53+
54+
**transaction ingress**
55+
56+
- The `LanedMempool` constructor defines an ordering of lanes. When a transaction is received by the app, it iterates through all lanes in order and inserts the transaction into the first `Lane` that it belongs in.
57+
**ordering**
58+
- Each `Lane` of the `LanedMempool` maintains its own ordering of transactions. When the `LanedMempool` routes a transaction to its corresponding `Lane` the `Lane` then inserts the transaction at its designated position with respect to all other transactions in the lane
59+
60+
### PrepareProposal
61+
62+
When the application is instructed to `PrepareProposal` it iterates through its `Lane`s in order, and calls each `Lane`'s `PrepareLane` method. The `Lane.PrepareLane` method collects transactions from a `Lane` and appends those transactions to the set of transactions from previous `Lane`'s `PrepareLane` calls. In other words, each block-proposal is now a collection of the transactions from the `LanedMempool`'s constituent lanes.
63+
64+
### ProcessProposal
65+
66+
When the application receives a proposal, and calls `ProcessProposal`, the app delegates the validation to the `LaneMempool.ProcessLanes` method. Remember, the proposal is composed of transactions from the sub-lanes of the `LaneMempool`, as such, the `LaneMempool` can route each `Lane`'s contribution to the Proposal to that `Lane` for validation. The proposal passes iff all `Lane`'s contributions are valid.
67+
68+
#### ⚠️ NOTE ⚠️
69+
70+
A block constructed from a `LaneMempool`'s `PrepareLanes` method must always pass that `LaneMempool`'s `ProcessLanes` method, otherwise, the chain will fail to produce blocks!! These functions are consensus critical, so practice caution when implementing them!!
71+
72+
73+
## 📖 Set Up [20 mins]
74+
75+
To get set up, we're going to implement the `Default Lane`, which is the **most general and least restrictive** that accepts all transactions. This will cause **no changes** to your chain functionality, but will prepare you to add `lanes` with more functionality afterwards!
76+
77+
The default lane mirrors how CometBFT creates proposals today.
78+
79+
- It does a basic check to ensure that the transaction is valid.
80+
- Orders the transactions based on tx fee amount (highest to lowest).
81+
- The `PrepareLane` handler will reap transactions from the lane up to the `MaxBlockSpace` limit
82+
- The `ProcessLane` handler will ensure that the transactions are ordered based on their fee amount and pass the same checks done in `PrepareLane`.
83+
84+
<!-- TODO: create script -->
85+
86+
# 🏗️ Default Lane Setup
87+
88+
## 📦 Dependencies
89+
90+
The Block SDK is built on top of the Cosmos SDK. The Block SDK is currently
91+
compatible with Cosmos SDK versions greater than or equal to `v0.47.0`.
92+
93+
### Release Compatibility Matrix
94+
95+
| Block SDK Version | Cosmos SDK |
96+
| :---------------: | :--------: |
97+
| `v1.x.x` | `v0.47.x` |
98+
| `v2.x.x` | `v0.50.x` |
99+
100+
## 📥 Adding the Block SDK to Your Project
101+
102+
```bash
103+
$ go get github.com/skip-mev/block-sdk
104+
```
105+
106+
## 📚 Usage
107+
108+
1. First determine the set of lanes that you want to use in your application. This guide only sets up the `default lane`
109+
110+
```golang
111+
import (
112+
"github.com/skip-mev/block-sdk/abci"
113+
"github.com/skip-mev/block-sdk/block/base"
114+
defaultlane "github.com/skip-mev/block-sdk/lanes/base"
115+
)
116+
117+
// 1. Create the lanes.
118+
//
119+
// NOTE: The lanes are ordered by priority. The first lane is the highest priority
120+
// lane and the last lane is the lowest priority lane. Top of block lane allows
121+
// transactions to bid for inclusion at the top of the next block.
122+
//
123+
// For more information on how to utilize the LaneConfig please
124+
// visit the README in docs.skip.money/chains/lanes/build-your-own-lane#-lane-config.
125+
//
126+
// Default lane accepts all transactions.
127+
128+
func NewApp() {
129+
...
130+
defaultConfig := base.LaneConfig{
131+
Logger: app.Logger(),
132+
TxEncoder: app.txConfig.TxEncoder(),
133+
TxDecoder: app.txConfig.TxDecoder(),
134+
MaxBlockSpace: math.LegacyZeroDec(),
135+
MaxTxs: 0,
136+
}
137+
defaultLane := defaultlane.NewDefaultLane(defaultConfig)
138+
// TODO(you): Add more Lanes!!!
139+
```
140+
141+
2. In your base application, you will need to create a `LanedMempool` composed
142+
of the `lanes` you want to use.
143+
144+
```golang
145+
// 2. Set up the relative priority of lanes
146+
lanes := []block.Lane{
147+
defaultLane,
148+
}
149+
mempool := block.NewLanedMempool(app.Logger(), true, lanes...)
150+
app.App.SetMempool(mempool)
151+
```
152+
153+
3. Next, order the lanes by priority. The first lane is the highest priority lane
154+
and the last lane is the lowest priority lane. **It is recommended that the last
155+
lane is the default lane.**
156+
157+
```golang
158+
// 3. Set up the ante handler.
159+
anteDecorators := []sdk.AnteDecorator{
160+
ante.NewSetUpContextDecorator(),
161+
...
162+
utils.NewIgnoreDecorator(
163+
ante.NewDeductFeeDecorator(
164+
options.BaseOptions.AccountKeeper,
165+
options.BaseOptions.BankKeeper,
166+
options.BaseOptions.FeegrantKeeper,
167+
options.BaseOptions.TxFeeChecker,
168+
),
169+
options.FreeLane,
170+
),
171+
...
172+
}
173+
174+
anteHandler := sdk.ChainAnteDecorators(anteDecorators...)
175+
176+
// Set the lane ante handlers on the lanes.
177+
//
178+
// NOTE: This step is very important. Without the antehandlers, lanes will not
179+
// be able to verify transactions.
180+
for _, lane := range lanes {
181+
lane.SetAnteHandler(anteHandler)
182+
}
183+
app.App.SetAnteHandler(anteHandler)
184+
```
185+
186+
4. You will also need to create a `PrepareProposalHandler` and a
187+
`ProcessProposalHandler` that will be responsible for preparing and processing
188+
proposals respectively. Configure the order of the lanes in the
189+
`PrepareProposalHandler` and `ProcessProposalHandler` to match the order of the
190+
lanes in the `LanedMempool`.
191+
192+
```golang
193+
// 4. Set the abci handlers on base app
194+
// Create the LanedMempool's ProposalHandler
195+
proposalHandler := abci.NewProposalHandler(
196+
app.Logger(),
197+
app.TxConfig().TxDecoder(),
198+
mempool,
199+
)
200+
201+
// set the Prepare / ProcessProposal Handlers on the app to be the `LanedMempool`'s
202+
app.App.SetPrepareProposal(proposalHandler.PrepareProposalHandler())
203+
app.App.SetProcessProposal(proposalHandler.ProcessProposalHandler())
204+
```
205+
206+
### 💅 Next step: implement other `lanes`
207+
208+
See the [Mev Lane](lanes/existing-lanes/1-mev.md) and select the `lanes` you want, or [Build Your Own](lanes/1-build-your-own-lane.md).

‎docs/1-overview.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Overview
2+
3+
### 🤔What is the Block SDK?
4+
5+
**the Block SDK is a toolkit for building customized blocks**
6+
7+
The Block SDK is a set of Cosmos SDK and ABCI++ primitives that allow chains to fully customize blocks to specific use cases. It turns your chain's blocks into a **`highway`** consisting of individual **`lanes`** with their own special functionality.
8+
9+
10+
Skip has built out a number of plug-and-play `lanes` on the SDK that your protocol can use, including in-protocol MEV recapture and Oracles! Additionally, the Block SDK can be extended to add **your own custom `lanes`** to configure your blocks to exactly fit your application needs.
11+
12+
🚦 **Blocks are like highways**
13+
14+
Let's say you're the designer of a 4 lane highway. You'd want a paid lane, for fast drivers who'd like to be separated from other lanes. You'd like a lane for large vehicles, you can configure this lane to be wider, require more space between vehicles, etc. The other two lanes are for the rest of traffic. The beauty here, is that as the owner of the highway, you get to decide what vehicles (transactions) you'll allow, and how they can behave (ordering)!!
15+
16+
17+
#### If you've been here before
18+
19+
##### [Integrate Block-SDK](0-integrate-the-sdk.md)
20+
21+
##### [Building your own Lane](lanes/1-build-your-own-lane.md)
22+
23+
##### [Searcher docs for MEV Lane](3-searcher-docs.md)
24+
25+
### ❌ Problems: Blocks are not Customizable
26+
27+
Most Cosmos chains today utilize traditional block construction - which is too limited.
28+
29+
- Traditional block building is susceptible to MEV-related issues, such as front-running and sandwich attacks, since proposers have monopolistic rights on ordering and no verification of good behavior. MEV that is created cannot be redistributed to the protocol.
30+
- Traditional block building uses a one-size-fits-all approach, which can result in inefficient transaction processing for specific applications or use cases and sub-optimal fee markets.
31+
- Transactions tailored for specific applications may need custom prioritization, ordering or validation rules that the mempool is otherwise unaware of because transactions within a block are currently in-differentiable when a blockchain might want them to be.
32+
33+
### ✅ Solution: The Block SDK
34+
35+
You can think of the Block SDK as a **transaction `highway` system**, where each
36+
`lane` on the highway serves a specific purpose and has its own set of rules and
37+
traffic flow.
38+
39+
In the Block SDK, each `lane` has its own set of rules and transaction flow management systems.
40+
41+
- A `lane` is what we might traditionally consider to be a standard mempool
42+
where transaction **_validation_**, **_ordering_** and **_prioritization_** for
43+
contained transactions are shared.
44+
- `lanes` implement a **standard interface** that allows each individual `lane` to
45+
propose and validate a portion of a block.
46+
- `lanes` are ordered with each other, configurable by developers. All `lanes`
47+
together define the desired block structure of a chain.
48+
49+
### ✨ Block SDK Use Cases
50+
51+
A block with separate `lanes` can be used for:
52+
53+
1. **MEV mitigation**: a top of block lane could be designed to create an in-protocol top-of-block [auction](lanes/existing-lanes/1-mev.md) to recapture MEV in a transparent and governable way.
54+
2. **Free/reduced fee txs**: transactions with certain properties (e.g. from trusted accounts or performing encouraged actions) could leverage a free lane to facilitate _good_ behavior.
55+
3. **Dedicated oracle space** Oracles could be included before other kinds of transactions to ensure that price updates occur first, and are not able to be sandwiched or manipulated.
56+
4. **Orderflow auctions**: an OFA lane could be constructed such that order flow providers can have their submitted transactions bundled with specific backrunners, to guarantee MEV rewards are attributed back to users
57+
5. **Enhanced and customizable privacy**: privacy-enhancing features could be introduced, such as threshold encrypted lanes, to protect user data and maintain privacy for specific use cases.
58+
6. **Fee market improvements**: one or many fee markets - such as EIP-1559 - could be easily adopted for different lanes (potentially custom for certain dApps). Each smart contract/exchange could have its own fee market or auction for transaction ordering.
59+
7. **Congestion management**: segmentation of transactions to lanes can help mitigate network congestion by capping usage of certain applications and tailoring fee markets.
60+
61+
### 🎆 Chains Currently Using the Block-SDK
62+
63+
#### Mainnets
64+
65+
| Chain Name | Chain-ID | Block-SDK Version |
66+
| ----------- | --------------- | ----------------- |
67+
| Juno | `juno-1` | `v1.0.2` |
68+
| Persistence | `persistence-1` | `v1.0.2` |
69+
| Initia | `NA` | `v1.0.2` |
70+
| Prism | `NA` | `v1.0.2` |
71+
| Terra | `phoenix-1` | `v1.0.2` |
72+
73+
#### Testnets
74+
75+
| Chain Name | Chain-ID | Block-SDK Version |
76+
| ---------- | -------- | ----------------- |
77+
| Juno | `uni-6` | `v1.0.2` |

‎docs/2-how-it-works.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# How it Works
2+
3+
<!-- TODO: add images to this -->
4+
5+
### Summary
6+
7+
With the Block SDK, blocks are broken up into smaller partial blocks called `lanes`.
8+
9+
- Each `lane` has its own custom block building logic and stores distinct types of transactions.
10+
- Each lane can only consume a portion of the block as defined on the `lane`'s configuration (`MaxBlockSpace`).
11+
- When a block proposal is requested, a block will **fill** with transactions from each `lane`, iteratively, in the order in which the `lanes` are defined in the application.
12+
- When a block proposal is processed, each `lane` will **verify** its portion of the block, iteratively, in the order in which the `lanes` are defined in the application.
13+
- **Transactions in blocks MUST respect the ordering of lanes.**
14+
15+
### 🔁 Background: Transaction Lifecycle
16+
17+
Knowledge of the general transaction lifecycle is important to understand how `lanes` work.
18+
19+
- A transaction begins when it is signed and broadcasted to a node on a chain.
20+
- It will be then be verified by the application on the node.
21+
- If it is valid, it will be inserted into the node's `mempool`, which is a storage area for transactions before inclusion in a block.
22+
- If the node happens to be a `validator`, and is proposing a block, the application will call `PrepareProposal` to create a new block proposal.
23+
- The proposer will look at what transactions they have in their mempool, iteratively select transactions until the block is full, and share the proposal with other validators.
24+
- When a different validator receives a proposal, the validator will verify its contents via `ProcessProposal` before signing it.
25+
- If the proposal is valid, the validator will sign the proposal and broadcast their vote to the network.
26+
- If the block is invalid, the validator will reject the proposal.
27+
- Once a proposal is accepted by the network, it is committed as a block and the transactions that were included are removed from every validator's mempool.
28+
29+
### 🛣️ Lane Lifecycle
30+
31+
`Lanes` introduce new steps in the transaction lifecycle outlined above.
32+
33+
A `LanedMempool` is composed of several distinct `lanes` that store their own transactions. The `LanedMempool` will insert the transaction into all `lanes` that accept it
34+
35+
- After the base application accepts a transaction, the transaction will be checked to see if it can go into any `lanes`, as defined by the lane's `MatchHandler`.
36+
- `Lane`'s can be configured to only accept transactions that match a certain criteria. For example, a `lane` could be configured to only accept transactions that are staking related (such as a free-transaction lane).
37+
- When a new block is proposed, the `PrepareProposalHandler` of the application will iteratively call `PrepareLane` on each `lane` (in the order in which they are defined in the application). The `PrepareLane` method is similar to `PrepareProposal`.
38+
- Calling `PrepareLane` on a `lane` will trigger the lane to reap transactions from its mempool and add them to the proposal (if they respect the verification rules of the `lane`).
39+
- When proposals are verified in `ProcessProposal` by other validators, the `ProcessProposalHandler` defined in `abci/abci.go` will call `ProcessLane` on each `lane` in the same order as they were called in the `PrepareProposalHandler`.
40+
- Each subsequent call to `ProcessLane` will filter out transactions that belong to previous lanes. **A given lane's ProcessLane will only verify transactions that belong to that lane.**
41+
42+
**Scenario**
43+
44+
Let's say we have a `LanedMempool` composed of two lanes: `LaneA` and `LaneB`.
45+
46+
`LaneA` is defined first in the `LanedMempool` and `LaneB` is defined second.
47+
48+
`LaneA` contains transactions Tx1 and Tx2 and `LaneB` contains transactions
49+
Tx3 and Tx4.
50+
51+
52+
When a new block needs to be proposed, the `PrepareProposalHandler` will call `PrepareLane` on `LaneA` first and `LaneB` second.
53+
54+
When `PrepareLane` is called on `LaneA`, `LaneA` will reap transactions from its mempool and add them to the proposal. The same applies for `LaneB`. Say `LaneA` reaps transactions Tx1 and Tx2 and `LaneB` reaps transactions Tx3 and Tx4. This gives us a proposal composed of the following:
55+
56+
- `Tx1`, `Tx2`, `Tx3`, `Tx4`
57+
58+
When the `ProcessProposalHandler` is called, it will call `ProcessLane` on `LaneA` with the proposal composed of Tx1, Tx2, Tx3, and Tx4. `LaneA` will then verify Tx1 and Tx2 and return the remaining transactions - Tx3 and Tx4. The `ProcessProposalHandler` will then call `ProcessLane` on `LaneB` with the remaining transactions - Tx3 and Tx4. `LaneB` will then verify Tx3 and Tx4 and return no remaining transactions.

0 commit comments

Comments
 (0)