Skip to content

Commit 7b45d74

Browse files
authored
network service, fabricx: query-service based key lookup #1386 (#1387)
Signed-off-by: Angelo De Caro <adc@zurich.ibm.com>
1 parent 3181bb2 commit 7b45d74

38 files changed

Lines changed: 1603 additions & 263 deletions

AGENTS.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,3 +152,13 @@ This directory contains the **Fabric Token SDK**, a project under Hyperledger La
152152
* **Docker errors:** Ensure `make testing-docker-images` has been run.
153153
* **Linting errors on commit:** Run `make lint-auto-fix`.
154154
* **Test timeouts:** Integration tests can be slow. Ensure you have allocated enough resources to Docker.
155+
156+
## Workflow Rules
157+
158+
- Before implementing any task, create a `plan.md` file in the project root containing:
159+
- A clear description of the goal
160+
- A numbered list of implementation steps
161+
- An "Implementation Progress" section with each step marked as `[ ] Pending`
162+
- As you complete each step, update `plan.md` immediately, marking the step as `[x] Done` and adding a brief note about what was changed
163+
- If you encounter a blocker or make a significant decision, log it under a `## Notes & Decisions` section in `plan.md`
164+
- Mark the plan as `✅ COMPLETE` once all steps are done

Makefile

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,3 +197,8 @@ lint-auto-fix:
197197
install-linter-tool:
198198
@echo "Installing golangci Linter"
199199
@curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(HOME)/go/bin v2.10.1
200+
201+
.PHONY: fmt
202+
fmt: ## Run gofmt on the entire project
203+
@echo "Running gofmt..."
204+
@gofmt -l -s -w .

docs/core-token.md

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,22 @@ token:
9595
workers: 10
9696
# queueSize is the size of the event buffer. Defaults to 1000.
9797
queueSize: 1000
98+
99+
# fabricx configuration for FabricX-specific settings
100+
fabricx:
101+
# lookup configuration for the lookup service
102+
lookup:
103+
# permanent lookup configuration
104+
permanent:
105+
# interval is the polling interval for permanent lookups. Defaults to 1m.
106+
interval: 1m
107+
# one-time lookup configuration
108+
once:
109+
# deadline is the maximum time to wait for a one-time lookup. Defaults to 5m.
110+
deadline: 5m
111+
# interval is the polling interval for one-time lookups. Defaults to 2s.
112+
interval: 2s
113+
98114
tms:
99115
mytms: # unique name of this token management system
100116
network: default # the name of the network this TMS refers to (Fabric, etc.)
@@ -239,4 +255,27 @@ Default values:
239255
- delivery.blockProcessParallelism: 10
240256
- delivery.lruSize: 30
241257
- delivery.lruBuffer: 15
242-
- delivery.listenerTimeout: 10s
258+
- delivery.listenerTimeout: 10s
259+
260+
---
261+
262+
## Optional: token.fabricx.lookup
263+
264+
If not specified, the default configuration is:
265+
266+
```yaml
267+
token:
268+
fabricx:
269+
lookup:
270+
permanent:
271+
interval: 1m
272+
once:
273+
deadline: 5m
274+
interval: 2s
275+
```
276+
277+
Default values:
278+
279+
- permanent.interval: 1m
280+
- once.deadline: 5m
281+
- once.interval: 2s

docs/services/network.md

Lines changed: 126 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -7,33 +7,132 @@ The network service architecture is depicted below:
77

88
![network_service.png](../imgs/network_service.png)
99

10+
## Overview
11+
12+
The Network Service is a critical component of the Fabric Token SDK that abstracts the complexities of the underlying Distributed Ledger Technology (DLT). It serves several key purposes:
13+
14+
- **Unified Interface**: Regardless of the backend (Fabric, Fabric-X, etc.), it provides a common set of APIs for transaction management and ledger interaction.
15+
- **Transaction Lifecycle Management**: Handles the submission (broadcasting) of transactions and provides mechanisms to wait for their finality (commitment and validity).
16+
- **Ledger Querying (QE)**: Allows services to retrieve the current state of tokens and other ledger entries through a specialized Query Engine.
17+
- **Public Parameters (PP) Management**: Monitors the ledger for updates to the system's public parameters and ensures the SDK is always using the latest version to maintain cryptographic integrity.
18+
- **Identity & Membership**: Interfaces with the local Membership Service Provider (MSP) to provide identities for signing and transaction creation.
19+
20+
The service uses a **Driver-based architecture**, allowing for different implementations to be plugged in based on the specific requirements and features of the target network.
21+
1022
## Fabric
1123

12-
The Fabric-based network implementation utilizes the Fabric Smart Client for configuration and operations, including chaincode queries and transaction broadcasting.
13-
14-
During bootstrap, the Token SDK processes the TMS defined in the configuration.
15-
For each TMS, the network provider retrieves the network instance corresponding to the `network` and `channel` specified in the TMS ID.
16-
Failure to retrieve the network instance results in a bootstrap failure.
17-
Upon success, the `Connect` function on the `Network` instance is invoked with the target namespace.
18-
This function establishes a connection to the backend, enabling the Token SDK to receive updates on public parameters and transaction finality.
19-
20-
When `Connect` is called, the Fabric network implementation establishes two `Fabric Delivery` streams to receive committed blocks:
21-
- One stream is used to analyze transactions that update the public parameters.
22-
More specifically, for each transaction in a block, the parser checks if the RW set contains a write whose key is the `setup key`.
23-
The setup key is set as `\x00seU+0000`. If such a key is found in a valid transaction, then the listener added upon calling `Connect` does the following:
24-
- It invokes the `Update` function of the TMS provider passing the TMS ID and the byte representation of the new public parameters.
25-
The function works as follows: if a TMS instance with the passed ID does not exist, it creates one.
26-
If a TMS with that ID already exists, then:
27-
- A new instance of TMS is created with the new public parameters.
28-
- If the previous step succeeds, then the `Done` function on the old TMS instance is invoked to release all allocated resources.
29-
- If the above step succeeds, then the public parameters are appended to the `PublicParameters` table.
30-
- The other stream is dedicated to transaction finality. Services can add listeners to the `Network` instance to listen for the finality of specific transactions.
31-
The `ttx` service and the `audit` service add a listener when a transaction has reached the point of being ready to be submitted to the ordering service.
32-
(For more information, look at the sections dedicated to these services). Both services use the same listener.
33-
This listener performs the following actions upon notification of the finality of a transaction:
34-
- If the transaction's status is valid, then the token request's hash contained in the transaction is matched against the hash of the token request stored in the database.
35-
If they match, then the `Tokens` table is updated by inserting the new tokens and marking the spent tokens as deleted.
36-
The corresponding token request in the `Requests` table is marked as `Valid` with a change of the status field.
37-
- If the transaction's status is invalid, then the corresponding token request in the `Requests` table is marked as `Invalid` or `Deleted`.
38-
In all other cases, an error is returned.
24+
The Fabric-based network implementation utilizes the Fabric Smart Client (FSC) to interact with the underlying Hyperledger Fabric network. It leverages FSC's configuration, transaction management, and communication layers to provide a robust backend for the Token SDK.
25+
26+
### Lifecycle and Bootstrap
27+
During the Token SDK bootstrap process, the system initializes a `Network` instance for each TMS (Token Management Service) defined in the configuration. The mapping is determined by the `network` and `channel` fields in the TMS identifier. If the specified network cannot be initialized (e.g., due to missing FSC configuration for that network), the bootstrap process will fail.
28+
29+
Upon successful initialization, the `Connect` function is invoked for the target namespace. This step is crucial as it:
30+
- Registers listeners for **Public Parameters** updates.
31+
- Initializes the **Endorsement Service** for the specific namespace.
32+
- Sets up the **Finality** and **Lookup** managers.
33+
34+
### Public Parameters Monitoring
35+
The Fabric driver monitors the ledger for updates to a specific "setup key" (usually `\x00seU+0000`). It uses a `PermanentLookupListener` that triggers whenever a valid transaction writes to this key. When an update is detected:
36+
1. The **TMS Provider** is updated with the new parameters. If a TMS instance already exists, it is replaced by a new one initialized with the updated cryptographic material, and the old instance is gracefully decommissioned.
37+
2. The new public parameters are persisted in the local **Tokens Database** to ensure consistency across restarts.
38+
39+
### Finality Management
40+
The Fabric driver supports two primary modes for monitoring transaction finality, configurable via `token.finality.type`:
41+
42+
- **Delivery Mode (`delivery`)**: This is the default mode for Fabric. It establishes a dedicated block delivery stream from the peer. The driver parses incoming blocks, processes read-write sets, and notifies registered listeners when a specific transaction ID is committed and validated. It includes advanced features like:
43+
- **Parallel Processing**: Blocks and transactions can be processed in parallel to improve throughput.
44+
- **LRU Caching**: Uses a Least Recently Used cache to track recently processed blocks and prevent redundant work.
45+
- **Notification Mode (`notification`)**: In this mode, the driver relies on event notifications from the underlying network service rather than pulling the entire block stream.
46+
47+
Regardless of the mode, the `ttx` and `audit` services utilize these listeners to update the local token vault and request status (e.g., marking a request as `Valid` or `Invalid`) once a transaction reaches finality on the ledger.
48+
49+
## FabricX
50+
51+
The `fabricx` driver is a specialized implementation designed for the Fabric-X network. It shares the same overall goals as the standard Fabric driver but introduces several implementation-specific optimizations and behaviors.
52+
53+
### Async Finality Processing
54+
FabricX handles transaction finality notifications asynchronously using an internal `EventQueue`. This queue is serviced by a pool of workers (by default, 10 workers with a queue size of 1000). This decoupled architecture ensures that the main network event loop remains non-blocking even when processing a high volume of finality notifications or performing complex transaction checks.
55+
56+
### Robust Transaction Submission
57+
The transaction submission process in FabricX involves a multi-step preparation phase:
58+
1. **Transaction ID Calculation**: Computes a unique ID based on a nonce and the creator's identity.
59+
2. **Namespace Marshaling**: Uses ASN1 marshaling for the target namespace (`TxNamespace`) before signing. This ensures the transaction structure meets the specific requirements of the Fabric-X MSP and ledger.
60+
3. **Broadcasting & Confirmation**: Once signed, the transaction is broadcast to the network. The broadcaster includes retry logic specifically for `io.EOF` errors, which often occur during network startup or transient connectivity issues.
61+
62+
### Public Parameters Versioning
63+
Unlike the standard Fabric driver, FabricX employs a `VersionKeeper` to manage the lifecycle of public parameters.
64+
- **Initialization**: The first time public parameters are updated, the version is initialized (the counter does not increment).
65+
- **Updates**: Subsequent updates to the public parameters increment an atomic version counter.
66+
- **Setup**: The TMS deployment process writes both the raw public parameters and their SHA256 hash to the ledger using specific setup keys defined by the translator.
67+
68+
### Query Engine (QE) and Token Detection
69+
The Query Engine in FabricX is responsible for retrieving the state of tokens from the ledger. For non-graph-hiding drivers, it determines if a token is spent by checking for the absence of its key in the ledger (a `nil` raw value). It supports batch retrieval of states to minimize network round-trips.
70+
71+
### Finality Retries
72+
During the initial connection phase, FabricX implements a specific retry strategy for retrieving finality information. If block 0 is not yet committed (a common scenario during network cold-starts), the driver will retry the operation (up to 5 times with a 2-second delay) to ensure a stable connection is established.
73+
74+
## Configuration
75+
76+
The Network Service and its drivers can be fine-tuned through the application configuration. Below are the key configuration parameters and examples for both Fabric and FabricX.
77+
78+
### TMS Configuration
79+
Each Token Management Service must be mapped to a network and channel.
80+
81+
```yaml
82+
token:
83+
enabled: true
84+
tms:
85+
my-tms-id:
86+
network: fabric-network-name # Matches fsc.networks configuration
87+
channel: my-channel
88+
namespace: my-chaincode-id
89+
```
90+
91+
### Fabric Finality Configuration
92+
These settings control the behavior of the Fabric driver's finality manager.
93+
94+
```yaml
95+
token:
96+
finality:
97+
# Mode: "delivery" (default) or "notification"
98+
type: delivery
99+
committer:
100+
maxRetries: 3
101+
retryWaitDuration: 5s
102+
delivery:
103+
# Number of parallel workers for mapping transactions
104+
mapperParallelism: 10
105+
# Number of parallel workers for processing blocks
106+
blockProcessParallelism: 10
107+
# Size of the LRU cache for block tracking
108+
lruSize: 30
109+
# Wait duration before timing out a delivery listener
110+
listenerTimeout: 10s
111+
```
112+
113+
### FabricX Specific Configuration
114+
FabricX introduces additional settings for its asynchronous event queue and lookup service.
115+
116+
```yaml
117+
token:
118+
finality:
119+
# FabricX defaults to "notification" mode
120+
type: notification
121+
notification:
122+
# Number of worker goroutines for the async queue
123+
workers: 10
124+
# Size of the event buffer
125+
queueSize: 1000
126+
127+
fabricx:
128+
lookup:
129+
permanent:
130+
# Polling interval for permanent lookups (e.g., public params)
131+
interval: 1m
132+
once:
133+
# Max time allowed for a one-time lookup
134+
deadline: 5m
135+
# Polling interval for one-time lookups
136+
interval: 2s
137+
```
39138

go.mod

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,10 @@ require (
1111
github.com/IBM/mathlib v0.0.3-0.20260204160318-5a4d1a385dfa
1212
github.com/dgraph-io/ristretto/v2 v2.4.0
1313
github.com/gin-gonic/gin v1.11.0
14+
github.com/go-co-op/gocron/v2 v2.19.1
1415
github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef
1516
github.com/hashicorp/go-uuid v1.0.3
16-
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260219175226-8f512e10d1a0
17+
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260302105910-adde99d4bf54
1718
github.com/hyperledger/fabric-chaincode-go/v2 v2.3.0
1819
github.com/hyperledger/fabric-lib-go v1.1.3-0.20240523144151-25edd1eaf5f5
1920
github.com/hyperledger/fabric-protos-go-apiv2 v0.3.7
@@ -145,6 +146,7 @@ require (
145146
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
146147
github.com/jaegertracing/jaeger-idl v0.6.0 // indirect
147148
github.com/jbenet/go-temp-err-catcher v0.1.0 // indirect
149+
github.com/jonboulle/clockwork v0.5.0 // indirect
148150
github.com/json-iterator/go v1.1.12 // indirect
149151
github.com/kilic/bls12-381 v0.1.0 // indirect
150152
github.com/klauspost/compress v1.18.0 // indirect
@@ -231,6 +233,7 @@ require (
231233
github.com/quic-go/quic-go v0.59.0 // indirect
232234
github.com/quic-go/webtransport-go v0.10.0 // indirect
233235
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
236+
github.com/robfig/cron/v3 v3.0.1 // indirect
234237
github.com/rogpeppe/go-internal v1.14.1 // indirect
235238
github.com/sagikazarmark/locafero v0.11.0 // indirect
236239
github.com/sirupsen/logrus v1.9.3 // indirect

go.sum

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -822,6 +822,8 @@ github.com/gkampitakis/go-diff v1.3.2 h1:Qyn0J9XJSDTgnsgHRdz9Zp24RaJeKMUHg2+PDZZ
822822
github.com/gkampitakis/go-diff v1.3.2/go.mod h1:LLgOrpqleQe26cte8s36HTWcTmMEur6OPYerdAAS9tk=
823823
github.com/gkampitakis/go-snaps v0.5.15 h1:amyJrvM1D33cPHwVrjo9jQxX8g/7E2wYdZ+01KS3zGE=
824824
github.com/gkampitakis/go-snaps v0.5.15/go.mod h1:HNpx/9GoKisdhw9AFOBT1N7DBs9DiHo/hGheFGBZ+mc=
825+
github.com/go-co-op/gocron/v2 v2.19.1 h1:B4iLeA0NB/2iO3EKQ7NfKn5KsQgZfjb2fkvoZJU3yBI=
826+
github.com/go-co-op/gocron/v2 v2.19.1/go.mod h1:5lEiCKk1oVJV39Zg7/YG10OnaVrDAV5GGR6O0663k6U=
825827
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
826828
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
827829
github.com/go-fonts/dejavu v0.1.0/go.mod h1:4Wt4I4OU2Nq9asgDCteaAaWZOV24E+0/Pwo0gppep4g=
@@ -1025,8 +1027,8 @@ github.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc=
10251027
github.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8=
10261028
github.com/hyperledger-labs/SmartBFT v0.0.0-20250503203013-eb005eef8866 h1:Mu/6NJsfl9g3wM15Ue7hqPq4LtgYDoABh8MO4u8aW4g=
10271029
github.com/hyperledger-labs/SmartBFT v0.0.0-20250503203013-eb005eef8866/go.mod h1:9aNHNXsCVy/leGz2gpTC1eOL5QecxbSAGjqsLh4T1LM=
1028-
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260219175226-8f512e10d1a0 h1:iXfHzs9I08/ok4XuAbcxEcz5qa7nQF6+oImB5VOMo/0=
1029-
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260219175226-8f512e10d1a0/go.mod h1:DHm6Q7lmQC6TIEy++a/hmDNRi6q05bAfRt0v2wfzFGQ=
1030+
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260302105910-adde99d4bf54 h1:a/bkgpkgkKJvW57SDM8RuKEZrygqsguJztNw7pOo0rg=
1031+
github.com/hyperledger-labs/fabric-smart-client v0.8.3-0.20260302105910-adde99d4bf54/go.mod h1:e0xYtdh2E4DgaSlS0sQ7xWjIVD1NA2y+11x3llSLzGY=
10301032
github.com/hyperledger/aries-bbs-go v0.0.0-20240528084656-761671ea73bc h1:3Ykk6MtyfnlzMOQry9zkxsoLWpCWZwDPqehO/BJwArM=
10311033
github.com/hyperledger/aries-bbs-go v0.0.0-20240528084656-761671ea73bc/go.mod h1:Kofn6A6WWea1ZM8Rys5aBW9dszwJ7Ywa0kyyYL0TPYw=
10321034
github.com/hyperledger/fabric-amcl v0.0.0-20230602173724-9e02669dceb2 h1:B1Nt8hKb//KvgGRprk0h1t4lCnwhE9/ryb1WqfZbV+M=
@@ -1092,6 +1094,8 @@ github.com/jaegertracing/jaeger-idl v0.6.0 h1:LOVQfVby9ywdMPI9n3hMwKbyLVV3BL1XH2
10921094
github.com/jaegertracing/jaeger-idl v0.6.0/go.mod h1:mpW0lZfG907/+o5w5OlnNnig7nHJGT3SfKmRqC42HGQ=
10931095
github.com/jbenet/go-temp-err-catcher v0.1.0 h1:zpb3ZH6wIE8Shj2sKS+khgRvf7T7RABoLk/+KKHggpk=
10941096
github.com/jbenet/go-temp-err-catcher v0.1.0/go.mod h1:0kJRvmDZXNMIiJirNPEYfhpPwbGVtZVWC34vc5WLsDk=
1097+
github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I=
1098+
github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60=
10951099
github.com/joshdk/go-junit v1.0.0 h1:S86cUKIdwBHWwA6xCmFlf3RTLfVXYQfvanM5Uh+K6GE=
10961100
github.com/joshdk/go-junit v1.0.0/go.mod h1:TiiV0PqkaNfFXjEiyjWM3XXrhVyCa1K4Zfga6W52ung=
10971101
github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=
@@ -1386,6 +1390,8 @@ github.com/quic-go/webtransport-go v0.10.0/go.mod h1:LeGIXr5BQKE3UsynwVBeQrU1TPr
13861390
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
13871391
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
13881392
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
1393+
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
1394+
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
13891395
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
13901396
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
13911397
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=

integration/token/fungible/support.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1046,7 +1046,7 @@ func UpdatePublicParamsAndWait(network *integration.Infrastructure, publicParams
10461046
if node.Id() == "custodian" {
10471047
continue
10481048
}
1049-
gomega.Eventually(GetPublicParams).WithArguments(network, node).WithTimeout(30 * time.Second).WithPolling(15 * time.Second).Should(gomega.Equal(publicParams))
1049+
gomega.Eventually(GetPublicParams).WithArguments(network, node).WithTimeout(60 * time.Second).WithPolling(5 * time.Second).Should(gomega.Equal(publicParams))
10501050
}
10511051
}
10521052

0 commit comments

Comments
 (0)