Skip to content

Commit 9ccab29

Browse files
DeshErBojhaagjermundgaraba
DeshErBojhaa
and
gjermundgaraba
authored
feat: Packet forward middleware (#8285)
Co-authored-by: Gjermund Garaba <[email protected]>
1 parent 33d2d18 commit 9ccab29

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+4335
-87
lines changed

‎.github/workflows/e2e-test-workflow-call.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,16 @@ on:
3636
description: 'The tag to use for chain B'
3737
required: true
3838
type: string
39+
chain-c-tag:
40+
description: 'The tag to use for chain C'
41+
required: true
42+
type: string
43+
default: main
44+
chain-d-tag:
45+
default: main
46+
description: 'The tag to use for chain D'
47+
required: true
48+
type: string
3949
# upgrade-plan-name is only required during upgrade tests, and is otherwise ignored.
4050
upgrade-plan-name:
4151
default: ''
@@ -78,6 +88,8 @@ jobs:
7888
echo "Chain Image: ${{ inputs.chain-image }}"
7989
echo "Chain A Tag: ${{ inputs.chain-a-tag }}"
8090
echo "Chain B Tag: ${{ inputs.chain-b-tag }}"
91+
echo "Chain C Tag: ${{ inputs.chain-c-tag }}"
92+
echo "Chain D Tag: ${{ inputs.chain-d-tag }}"
8193
echo "Upgrade Plan Name: ${{ inputs.upgrade-plan-name }}"
8294
echo "Test Entry Point: ${{ inputs.test-entry-point }}"
8395
echo "Test: ${{ inputs.test }}"
@@ -205,6 +217,8 @@ jobs:
205217
CHAIN_UPGRADE_PLAN: '${{ inputs.upgrade-plan-name }}'
206218
CHAIN_A_TAG: '${{ inputs.chain-a-tag }}'
207219
CHAIN_B_TAG: '${{ inputs.chain-b-tag }}'
220+
CHAIN_C_TAG: '${{ inputs.chain-c-tag }}'
221+
CHAIN_D_TAG: '${{ inputs.chain-d-tag }}'
208222
E2E_CONFIG_PATH: '${{ inputs.e2e-config-path }}'
209223
strategy:
210224
fail-fast: false
@@ -243,6 +257,8 @@ jobs:
243257
CHAIN_IMAGE: '${{ inputs.chain-image }}'
244258
CHAIN_A_TAG: '${{ inputs.chain-a-tag }}'
245259
CHAIN_B_TAG: '${{ inputs.chain-b-tag }}'
260+
CHAIN_C_TAG: '${{ inputs.chain-c-tag }}'
261+
CHAIN_D_TAG: '${{ inputs.chain-d-tag }}'
246262
E2E_CONFIG_PATH: '${{ inputs.e2e-config-path }}'
247263
strategy:
248264
fail-fast: false
@@ -256,6 +272,7 @@ jobs:
256272
- entrypoint: TestTransferLocalhostTestSuite
257273
- entrypoint: TestConnectionTestSuite
258274
- entrypoint: TestInterchainAccountsGovTestSuite
275+
- entrypoint: TestForwardTransferSuite
259276
steps:
260277
- uses: actions/checkout@v4
261278
with:

‎.github/workflows/e2e-upgrade.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@ jobs:
6868
CHAIN_IMAGE: '${{ env.DOCKER_IMAGE_NAME }}'
6969
CHAIN_A_TAG: '${{ matrix.test-config.tag }}'
7070
CHAIN_B_TAG: '${{ matrix.test-config.tag }}'
71+
CHAIN_C_TAG: '${{ matrix.test-config.tag }}'
72+
CHAIN_D_TAG: '${{ matrix.test-config.tag }}'
7173
CHAIN_UPGRADE_PLAN: '${{ matrix.test-config.upgrade-plan }}'
7274
E2E_CONFIG_PATH: 'ci-e2e-config.yaml'
7375
run: |

‎.github/workflows/e2e.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,8 @@ jobs:
104104
CHAIN_IMAGE: '${{ env.DOCKER_IMAGE_NAME }}'
105105
CHAIN_A_TAG: '${{ needs.determine-image-tag.outputs.simd-tag }}'
106106
CHAIN_B_TAG: '${{ needs.determine-image-tag.outputs.simd-tag }}'
107+
CHAIN_C_TAG: '${{ needs.determine-image-tag.outputs.simd-tag }}'
108+
CHAIN_D_TAG: '${{ needs.determine-image-tag.outputs.simd-tag }}'
107109
E2E_CONFIG_PATH: 'ci-e2e-config.yaml'
108110
run: |
109111
cd e2e

‎CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ Ref: https://keepachangelog.com/en/1.0.0/
3838

3939
### Features
4040

41+
* [\#8285](https://github.com/cosmos/ibc-go/pull/8285) Packet forward middleware.
42+
4143
### Dependencies
4244

4345
* [\#8369](https://github.com/cosmos/ibc-go/pull/8369) Bump **github.com/CosmWasm/wasmvm** to **2.2.4**

‎docs/docs/02-apps/03-packet-forward-middleware/integration.md

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# Integration
2+
3+
This document provides instructions on integrating and configuring the Packet Forward Middleware (PFM) within your
4+
existing chain implementation. This document is *NOT* a guide on developing with the Cosmos SDK or ibc-go and makes
5+
the assumption that you have some existing codebase for your chain with IBC already enabled.
6+
7+
The integration steps include the following:
8+
9+
1. Import the PFM, initialize the PFM Module & Keeper, initialize the store keys and module params, and initialize the Begin/End Block logic and InitGenesis order.
10+
2. Configure the IBC application stack including the transfer module.
11+
3. Configuration of additional options such as timeout period, number of retries on timeout, refund timeout period, and fee percentage.
12+
13+
Integration of the PFM should take approximately 20 minutes.
14+
15+
## Example integration of the Packet Forward Middleware
16+
17+
```go
18+
// app.go
19+
20+
// Import the packet forward middleware
21+
import (
22+
"github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v10/packetforward"
23+
packetforwardkeeper "github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v10/packetforward/keeper"
24+
packetforwardtypes "github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v10/packetforward/types"
25+
)
26+
27+
...
28+
29+
// Register the AppModule for the packet forward middleware module
30+
ModuleBasics = module.NewBasicManager(
31+
...
32+
packetforward.AppModuleBasic{},
33+
...
34+
)
35+
36+
...
37+
38+
// Add packet forward middleware Keeper
39+
type App struct {
40+
...
41+
PacketForwardKeeper *packetforwardkeeper.Keeper
42+
...
43+
}
44+
45+
...
46+
47+
// Create store keys
48+
keys := sdk.NewKVStoreKeys(
49+
...
50+
packetforwardtypes.StoreKey,
51+
...
52+
)
53+
54+
...
55+
56+
// Initialize the packet forward middleware Keeper
57+
// It's important to note that the PFM Keeper must be initialized before the Transfer Keeper
58+
app.PacketForwardKeeper = packetforwardkeeper.NewKeeper(
59+
appCodec,
60+
keys[packetforwardtypes.StoreKey],
61+
nil, // will be zero-value here, reference is set later on with SetTransferKeeper.
62+
app.IBCKeeper.ChannelKeeper,
63+
appKeepers.DistrKeeper,
64+
app.BankKeeper,
65+
app.IBCKeeper.ChannelKeeper,
66+
authtypes.NewModuleAddress(govtypes.ModuleName).String(),
67+
)
68+
69+
// Initialize the transfer module Keeper
70+
app.TransferKeeper = ibctransferkeeper.NewKeeper(
71+
appCodec,
72+
keys[ibctransfertypes.StoreKey],
73+
app.GetSubspace(ibctransfertypes.ModuleName),
74+
app.PacketForwardKeeper,
75+
app.IBCKeeper.ChannelKeeper,
76+
&app.IBCKeeper.PortKeeper,
77+
app.AccountKeeper,
78+
app.BankKeeper,
79+
scopedTransferKeeper,
80+
)
81+
82+
app.PacketForwardKeeper.SetTransferKeeper(app.TransferKeeper)
83+
84+
// See the section below for configuring an application stack with the packet forward middleware
85+
86+
...
87+
88+
// Register packet forward middleware AppModule
89+
app.moduleManager = module.NewManager(
90+
...
91+
packetforward.NewAppModule(app.PacketForwardKeeper, app.GetSubspace(packetforwardtypes.ModuleName)),
92+
)
93+
94+
...
95+
96+
// Add packet forward middleware to begin blocker logic
97+
app.moduleManager.SetOrderBeginBlockers(
98+
...
99+
packetforwardtypes.ModuleName,
100+
...
101+
)
102+
103+
// Add packet forward middleware to end blocker logic
104+
app.moduleManager.SetOrderEndBlockers(
105+
...
106+
packetforwardtypes.ModuleName,
107+
...
108+
)
109+
110+
// Add packet forward middleware to init genesis logic
111+
app.moduleManager.SetOrderInitGenesis(
112+
...
113+
packetforwardtypes.ModuleName,
114+
...
115+
)
116+
117+
// Add packet forward middleware to init params keeper
118+
func initParamsKeeper(appCodec codec.BinaryCodec, legacyAmino *codec.LegacyAmino, key, tkey storetypes.StoreKey) paramskeeper.Keeper {
119+
...
120+
paramsKeeper.Subspace(packetforwardtypes.ModuleName).WithKeyTable(packetforwardtypes.ParamKeyTable())
121+
...
122+
}
123+
```
124+
125+
## Configuring the transfer application stack with Packet Forward Middleware
126+
127+
Here is an example of how to create an application stack using `transfer` and `packet-forward-middleware`.
128+
The following `transferStack` is configured in `app/app.go` and added to the IBC `Router`.
129+
The in-line comments describe the execution flow of packets between the application stack and IBC core.
130+
131+
For more information on configuring an IBC application stack see the ibc-go docs [here](https://github.com/cosmos/ibc-go/blob/e69a833de764fa0f5bdf0338d9452fd6e579a675/docs/docs/04-middleware/01-ics29-fee/02-integration.md#configuring-an-application-stack-with-fee-middleware).
132+
133+
```go
134+
// Create Transfer Stack
135+
// SendPacket, since it is originating from the application to core IBC:
136+
// transferKeeper.SendPacket -> packetforward.SendPacket -> channel.SendPacket
137+
138+
// RecvPacket, message that originates from core IBC and goes down to app, the flow is the other way
139+
// channel.RecvPacket -> packetforward.OnRecvPacket -> transfer.OnRecvPacket
140+
141+
// transfer stack contains (from top to bottom):
142+
// - Packet Forward Middleware
143+
// - Transfer
144+
var transferStack ibcporttypes.IBCModule
145+
transferStack = transfer.NewIBCModule(app.TransferKeeper)
146+
transferStack = packetforward.NewIBCMiddleware(
147+
transferStack,
148+
app.PacketForwardKeeper,
149+
0, // retries on timeout
150+
packetforwardkeeper.DefaultForwardTransferPacketTimeoutTimestamp, // forward timeout
151+
)
152+
153+
// Add transfer stack to IBC Router
154+
ibcRouter.AddRoute(ibctransfertypes.ModuleName, transferStack)
155+
```
156+
157+
## Configurable options in the Packet Forward Middleware
158+
159+
The Packet Forward Middleware has several configurable options available when initializing the IBC application stack.
160+
You can see these passed in as arguments to `packetforward.NewIBCMiddleware` and they include the number of retries that
161+
will be performed on a forward timeout, the timeout period that will be used for a forward, and the timeout period that
162+
will be used for performing refunds in the case that a forward is taking too long.
163+
164+
Additionally, there is a fee percentage parameter that can be set in `InitGenesis`, this is an optional parameter that
165+
can be used to take a fee from each forwarded packet which will then be distributed to the community pool. In the
166+
`OnRecvPacket` callback `ForwardTransferPacket` is invoked which will attempt to subtract a fee from the forwarded
167+
packet amount if the fee percentage is non-zero.
168+
169+
- Retries On Timeout - how many times will a forward be re-attempted in the case of a timeout.
170+
- Timeout Period - how long can a forward be in progress before giving up.
171+
- Refund Timeout - how long can a forward be in progress before issuing a refund back to the original source chain.
172+
- Fee Percentage - % of the forwarded packet amount which will be subtracted and distributed to the community pool.

‎e2e/go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,7 @@ require (
152152
github.com/hdevalence/ed25519consensus v0.2.0 // indirect
153153
github.com/holiman/uint256 v1.3.2 // indirect
154154
github.com/huandu/skiplist v1.2.1 // indirect
155+
github.com/iancoleman/orderedmap v0.3.0 // indirect
155156
github.com/iancoleman/strcase v0.3.0 // indirect
156157
github.com/icza/dyno v0.0.0-20230330125955-09f820a8d9c0 // indirect
157158
github.com/improbable-eng/grpc-web v0.15.0 // indirect

‎e2e/go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1213,6 +1213,8 @@ github.com/huandu/go-assert v1.1.5/go.mod h1:yOLvuqZwmcHIC5rIzrBhT7D3Q9c3GFnd0Jr
12131213
github.com/huandu/skiplist v1.2.1 h1:dTi93MgjwErA/8idWTzIw4Y1kZsMWx35fmI2c8Rij7w=
12141214
github.com/huandu/skiplist v1.2.1/go.mod h1:7v3iFjLcSAzO4fN5B8dvebvo/qsfumiLiDXMrPiHF9w=
12151215
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
1216+
github.com/iancoleman/orderedmap v0.3.0 h1:5cbR2grmZR/DiVt+VJopEhtVs9YGInGIxAoMJn+Ichc=
1217+
github.com/iancoleman/orderedmap v0.3.0/go.mod h1:XuLcCUkdL5owUCQeF2Ue9uuw1EptkJDkXXS7VoV7XGE=
12161218
github.com/iancoleman/strcase v0.2.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho=
12171219
github.com/iancoleman/strcase v0.3.0 h1:nTXanmYxhfFAMjZL34Ov6gkzEsSJZ5DbhxWjvSASxEI=
12181220
github.com/iancoleman/strcase v0.3.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho=

‎e2e/sample.config.extended.yaml

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
# | CHAIN_IMAGE | The image that will be used for the chain | ghcr.io/cosmos/ibc-go-simd |
88
# | CHAIN_A_TAG | The tag used for chain A | N/A (must be set) |
99
# | CHAIN_B_TAG | The tag used for chain B | N/A (must be set) |
10+
# | CHAIN_C_TAG | The tag used for chain C | Optional (fallback to A) |
11+
# | CHAIN_D_TAG | The tag used for chain D | Optional (fallback to A) |
1012
# | CHAIN_BINARY | The binary used in the container | simd |
1113
# | RELAYER_TAG | The tag used for the relayer | 1.10.4 |
1214
# | RELAYER_ID | The type of relayer to use (rly/hermes) | hermes |
@@ -23,13 +25,29 @@ chains:
2325
tag: main # override with CHAIN_A_TAG
2426
binary: simd # override with CHAIN_BINARY
2527

26-
# the entry at index 1 corresponds to CHAIN_B
28+
# the entry at index 1 corresponds to CHAIN_B
2729
- chainId: chainB-1
2830
numValidators: 4
2931
numFullNodes: 1
3032
image: ghcr.io/cosmos/ibc-go-simd # override with CHAIN_IMAGE
3133
tag: main # override with CHAIN_B_TAG
3234
binary: simd # override with CHAIN_BINARY
35+
36+
# the entry at index 2 corresponds to CHAIN_C
37+
- chainId: chainC-1
38+
numValidators: 4
39+
numFullNodes: 1
40+
image: ghcr.io/cosmos/ibc-go-simd # override with CHAIN_IMAGE
41+
tag: main # override with CHAIN_C_TAG
42+
binary: simd # override with CHAIN_BINARY
43+
44+
# the entry at index 3 corresponds to CHAIN_D
45+
- chainId: chainD-1
46+
numValidators: 4
47+
numFullNodes: 1
48+
image: ghcr.io/cosmos/ibc-go-simd # override with CHAIN_IMAGE
49+
tag: main # override with CHAIN_D_TAG
50+
binary: simd # override with CHAIN_BINARY
3351

3452
# activeRelayer must match the id of a relayer specified in the relayers list below.
3553
activeRelayer: hermes # override with RELAYER_ID

‎e2e/sample.config.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,7 @@ chains:
66
chainId: chainA-1
77
- tag: main # override with CHAIN_B_TAG
88
chainId: chainB-1
9+
- tag: main # override with CHAIN_C_TAG
10+
chainId: chainC-1
11+
- tag: main # override with CHAIN_D_TAG
12+
chainId: chainD-1

‎e2e/tests/core/03-connection/connection_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ func (s *ConnectionTestSuite) SetupSuite() {
4040
}
4141

4242
func (s *ConnectionTestSuite) CreateConnectionTestPath(testName string) (ibc.Relayer, ibc.ChannelOutput) {
43-
return s.CreatePaths(ibc.DefaultClientOpts(), s.TransferChannelOptions(), testName), s.GetChainAChannelForTest(testName)
43+
return s.CreatePaths(ibc.DefaultClientOpts(), s.TransferChannelOptions(), testName), s.GetChainAToChainBChannel(testName)
4444
}
4545

4646
// QueryMaxExpectedTimePerBlockParam queries the on-chain max expected time per block param for 03-connection

0 commit comments

Comments
 (0)