You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/coordinator.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@ parallel processing while ensuring deterministic outcomes.
30
30
31
31
## 2. Core Responsibilities
32
32
33
-
The [Coordinator](https://github.com/hyperledger/fabric-x-committer/tree/main/service/coordinator) service performs five main tasks:
33
+
The [Coordinator](/service/coordinator) service performs five main tasks:
34
34
35
35
1.**Receive Blocks:** Ingests blocks of transactions from the Sidecar service.
36
36
2.**Manage Dependencies:** Utilizes a `Dependency Graph Manager` to construct and maintain a directed acyclic graph (DAG) of transaction dependencies.
@@ -44,7 +44,7 @@ to update the graph, and relays the status for each transaction back to the Side
44
44
45
45
## 3. Configuration
46
46
47
-
The Coordinator service requires the following configuration settings, provided in a standard YAML [configuration file](https://github.com/hyperledger/fabric-x-committer/blob/main/cmd/config/samples/coordinator.yaml):
47
+
The Coordinator service requires the following configuration settings, provided in a standard YAML [configuration file](/cmd/config/samples/coordinator.yaml):
48
48
49
49
-*Signature Verifier Endpoints:* Address(es) of the signature verifier service node(s).
50
50
-*Validator-Committer Endpoints:* Address(es) of the validator-committer service node(s).
@@ -91,14 +91,14 @@ onto the `coordinatorToDepGraphTxs` channel.
The [`Dependency Graph Manager`](https://github.com/hyperledger/fabric-x-committer/tree/main/service/coordinator/dependencygraph)
94
+
The [`Dependency Graph Manager`](/service/coordinator/dependencygraph)
95
95
consumes transactions from the `coordinatorToDepGraphTxs` channel and builds a dependency graph. This process
96
96
identifies all transactions that are "dependency-free" and can be immediately scheduled for validation. These dependency-free transaction
97
97
nodes are sent to the `depGraphToSigVerifierFreeTxs` channel.
98
98
99
99
### Step 3. Signature Verification
100
100
101
-
The [`Signature Verifier Manager`](https://github.com/hyperledger/fabric-x-committer/blob/main/service/coordinator/signature_verifier_manager.go) receives dependency-free nodes from the `depGraphToSigVerifierFreeTxs` channel and acts as a load balancer,
101
+
The [`Signature Verifier Manager`](/service/coordinator/signature_verifier_manager.go) receives dependency-free nodes from the `depGraphToSigVerifierFreeTxs` channel and acts as a load balancer,
102
102
distributing the transactions across a pool of available `Signature Verifiers`. The manager also keeps track of all requests sent to each
103
103
verifier service. If a service fails, the manager forwards any pending requests to another available service to ensure no transactions are lost.
104
104
These services perform preliminary checks, including signature
@@ -109,7 +109,7 @@ are passed along to the next stage via the `sigVerifierToVCServiceValidatedTxs`
109
109
110
110
### Step 4. Final Validation and Commit
111
111
112
-
The [`Validator-Committer Manager`](https://github.com/hyperledger/fabric-x-committer/blob/main/service/coordinator/validator_committer_manager.go) receives verified transactions from the `sigVerifierToVCServiceValidatedTxs` channel and routes them to
112
+
The [`Validator-Committer Manager`](/service/coordinator/validator_committer_manager.go) receives verified transactions from the `sigVerifierToVCServiceValidatedTxs` channel and routes them to
113
113
available `Validator-Committer` services. The manager also keeps track of all requests sent to each verifier service. If a service fails, the manager
114
114
forwards any pending requests to another available service to ensure no transactions are lost. The `Validator-Committer` services execute their own
115
115
three-phase pipelined process (Prepare, Validate, Commit) to perform final MVCC checks against the database and commit the results.
@@ -125,7 +125,7 @@ to the Sidecar for final aggregation and delivery to clients.
125
125
126
126
### Step 6. Post-Commit Processing
127
127
If a committed transaction creates or updates a namespace, a post-commit process is triggered to update the system's policies. The `Validator-Committer Manager`,
128
-
in conjunction with a [`Policy Manager`](https://github.com/hyperledger/fabric-x-committer/blob/main/service/coordinator/policy_manager.go), ensures that all `Signature Verifier` services are updated with the new endorsement policy for that namespace.
128
+
in conjunction with a [`Policy Manager`](/service/coordinator/policy_manager.go), ensures that all `Signature Verifier` services are updated with the new endorsement policy for that namespace.
129
129
This update is not performed immediately via a separate call. Instead, for efficiency, the policy update is "piggybacked" onto the next validation
130
130
request sent to each verifier. The verifier then updates its internal policy list before processing the accompanying transactions.
An edge from a later transaction ($T_j$) to an earlier transaction ($T_i$) indicates that $T_i$ must be finalized before $T_j$ can be validated. The graph tracks three types of dependencies:
140
140
141
-
1.**Read-Write Dependency ($T_{i}\xleftarrow{rw\mbox{(}k\mbox{)}}T_{j}$):** $T_i$ writes to key `k`, and a later transaction $T_j$ reads the *previous* version of `k`.
141
+
1.**Read-Write Dependency ($T_{i}\xleftarrow{rw(k)}T_{j}$):** $T_i$ writes to key `k`, and a later transaction $T_j$ reads the *previous* version of `k`.
142
142
If $T_i$ is valid, $T_j$ must be invalid because it read a stale value.
143
-
2.**Write-Read Dependency ($T_{i}\xleftarrow{wr\mbox{(}k\mbox{)}}T_{j}$):** $T_i$ reads key `k`, and a later transaction $T_j$ writes to `k`. This dependency is used
143
+
2.**Write-Read Dependency ($T_{i}\xleftarrow{wr(k)}T_{j}$):** $T_i$ reads key `k`, and a later transaction $T_j$ writes to `k`. This dependency is used
144
144
to enforce commit order. $T_j$ cannot be committed before $T_i$ is finalized, otherwise $T_i$'s read would become stale, violating the original block order.
145
-
3.**Write-Write Dependency ($T_{i}\xleftarrow{ww\mbox{(}k\mbox{)}}T_{j}$):** Both $T_i$ and $T_j$ write to the same key `k`. This dependency ensures $T_j$ is not
145
+
3.**Write-Write Dependency ($T_{i}\xleftarrow{ww(k)}T_{j}$):** Both $T_i$ and $T_j$ write to the same key `k`. This dependency ensures $T_j$ is not
146
146
committed before $T_i$, preventing $T_j$'s write from being overwritten and lost.
1.[errgroup - Synchronization, Error Propagation, and Context Cancellation for Goroutines](#errgroup-synchronization,-error-propagation,-and-context-cancellation-for-goroutines) - [Code](https://cs.opensource.google/go/x/sync)
9
-
2.[channel - Handling Communication within and across errgroup Tasks](#handling-channel-communication-within-errgroup-tasks) - [Code](https://github.com/hyperledger/fabric-x-committer//utils/channel/channel.go)
9
+
2.[channel - Handling Communication within and across errgroup Tasks](#handling-channel-communication-within-errgroup-tasks) - [Code](/utils/channel/channel.go)
10
10
11
11
## 1. errgroup - Synchronization, Error Propagation, and Context Cancellation for Goroutines
12
12
@@ -83,7 +83,7 @@ This pattern ensures that related long-running tasks are managed as a cohesive u
83
83
reliably shutting down together when any essential part fails, preventing partially
84
84
functioning states.
85
85
86
-
Example from [https://github.com/hyperledger/fabric-x-committer/service/sidecar/relay.go](https://github.com/hyperledger/fabric-x-committer/blob/main/service/sidecar/relay.go#L64)
86
+
Example from [service/sidecar/relay.go](/service/sidecar/relay.go#L64)
Copy file name to clipboardExpand all lines: docs/sidecar.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -74,7 +74,7 @@ In this section, we provide an overview of each of these tasks in detail.
74
74
The following code runs *Task 1* which fetches blocks from the ordering service and
75
75
enqueue to `blockToBeCommitted` queues.
76
76
77
-
From [https://github.com/hyperledger/fabric-x-committer/blob/main/service/sidecar/sidecar.go](https://github.com/hyperledger/fabric-x-committer/service/sidecar/sidecar.go#L138)
77
+
From [service/sidecar/sidecar.go](/service/sidecar/sidecar.go#L138)
78
78
79
79
```go
80
80
g.Go(func() error {
@@ -101,7 +101,7 @@ using the IP addresses provided in the configuration. It creates an `AtomicBroad
101
101
(a.k.a Orderer Client) using the `NewAtomicBroadcastClient()` function provided by the `fabric-protos-go` library
102
102
(`orderer/ab.pb.go`).
103
103
104
-
From [https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go)
104
+
From [hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/blob/main/orderer/ab.pb.go)
@@ -121,7 +121,7 @@ The Sidecar calls the `Deliver()` method on the `AtomicBroadcastClient` to estab
121
121
gRPC stream (`AtomicBroadcast_DeliverClient`) with the Ordering Service specifically
122
122
for receiving blocks.
123
123
124
-
From [https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go)
124
+
From [hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/blob/main/orderer/ab.pb.go)
125
125
126
126
```go
127
127
typeAtomicBroadcast_DeliverClientinterface {
@@ -140,7 +140,7 @@ set the start position to block `0` and the stop position to `MaxUint64`. When t
140
140
restarts after a failure, it determines the next needed block number and sets the start block
141
141
number accordingly. It always sets the `SeekBehavior` to `BLOCK_UNTIL_READY`.
142
142
143
-
From [https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/orderer/ab.pb.go) (illustrative fields)
143
+
From [hyperledger/fabric-protos-go/orderer/ab.pb.go](https://github.com/hyperledger/fabric-protos-go/blob/main/orderer/ab.pb.go) (illustrative fields)
144
144
```go
145
145
message SeekInfo {
146
146
SeekPosition start = 1; // The position to start the deliver from
@@ -183,7 +183,7 @@ Sidecar responsible for relaying them to the Coordinator.
183
183
The following code runs *Task 2* which relays the blocks present in
184
184
`blocksToBeCommitted` to coordinator and receive statuses.
185
185
186
-
From [https://github.com/hyperledger/fabric-x-committer/blob/main/service/sidecar/sidecar.go](https://github.com/hyperledger/fabric-x-committer/service/sidecar/sidecar.go#L152)
186
+
From [service/sidecar/sidecar.go](/service/sidecar/sidecar.go#L152)
@@ -277,7 +277,7 @@ Subsequently, this modified block is enqueued onto the `committedBlocks` output
277
277
The following code runs *Task 3* which reads blocks present in the `committedBlocks`
278
278
queue and store the block in the file system.
279
279
280
-
From [https://github.com/hyperledger/fabric-x-committer/blob/main/service/sidecar/sidecar.go](https://github.com/hyperledger/fabric-x-committer/service/sidecar/sidecar.go#L103)
280
+
From [service/sidecar/sidecar.go](/service/sidecar/sidecar.go#L103)
281
281
282
282
```go
283
283
g.Go(func() error {
@@ -333,7 +333,7 @@ using the IP:Port on which the ledger service (the Sidecar's block delivery serv
333
333
is listening. It creates a `DeliverClient` using the `NewDeliverClient()` function
334
334
provided by the `fabric-protos-go`` library (`peer/events.pb.go`).
335
335
336
-
From [https://github.com/hyperledger/fabric-protos-go/peer/events.pb.go](https://github.com/hyperledger/fabric-protos-go/peer/events.pb.go)
336
+
From [hyperledger/fabric-protos-go/peer/events.pb.go](https://github.com/hyperledger/fabric-protos-go/blob/main/peer/events.pb.go)
The [Validator-Committer (VC) service](https://github.com/hyperledger/fabric-x-committer/tree/main/service/vc) is a core component responsible for the final stages of transaction processing.
21
+
The [Validator-Committer (VC) service](/service/vc) is a core component responsible for the final stages of transaction processing.
22
22
It operates downstream from the Coordinator, receiving batches of transactions that are internally conflict-free.
23
23
This means that these conflict-free transactions do not conflict with any other active transactions being processed concurrently.
24
24
Its primary function is to perform optimistic concurrency control by validating each transaction's read-set against the current state in
@@ -37,7 +37,7 @@ write to the state database, ensuring consistent data access.
37
37
38
38
## 3. Configuration
39
39
40
-
The VC service requires the following configuration settings, provided in a YAML [configuration file](https://github.com/hyperledger/fabric-x-committer/blob/main/cmd/config/samples/vcservice.yaml):
40
+
The VC service requires the following configuration settings, provided in a YAML [configuration file](/cmd/config/samples/vcservice.yaml):
41
41
42
42
**Database Endpoints*: Address(es) of the state database nodes that the service will connect to.
43
43
**Listen Address*: The local address and port (e.g., 0.0.0.0:7056) where the VC service will host its gRPC service, enabling
@@ -125,7 +125,7 @@ Communication among these tasks is managed by a series of internal channels:
125
125
This task consumes transaction batches from the `toPrepareTxs` channel and transforms them into a structured format optimized for validation.
126
126
The primary goal is to organize the reads and writes from all transactions in the batch for efficient processing in the subsequent stages.
127
127
128
-
**a. Dequeueing Transaction Batches:** The [preparer](https://github.com/hyperledger/fabric-x-committer/blob/main/service/vc/preparer.go) reads a batch of transactions from the `toPrepareTxs` input queue.
128
+
**a. Dequeueing Transaction Batches:** The [preparer](/service/vc/preparer.go) reads a batch of transactions from the `toPrepareTxs` input queue.
129
129
130
130
**b. Data Structuring:** It iterates through each transaction in the batch and populates a `preparedTransactions` struct. This involves:
131
131
@@ -197,7 +197,7 @@ is known from the read phase, the new version is calculated by simply incrementi
197
197
This task performs the core MVCC logic. It consumes `preparedTransactions` from its input channel, checks the read-sets against the database, and
198
198
filters out any transactions that are invalidated by the check.
199
199
200
-
**a. Dequeueing Prepared Transactions:** The [validator](https://github.com/hyperledger/fabric-x-committer/blob/main/service/vc/validator.go) reads a `preparedTransactions` object from the `preparedTxs` queue.
200
+
**a. Dequeueing Prepared Transactions:** The [validator](/service/vc/validator.go) reads a `preparedTransactions` object from the `preparedTxs` queue.
201
201
202
202
**b. Validating Reads:** For performance, instead of fetching each key's version individually, the validator invokes a stored procedure for each namespace.
203
203
This procedure takes the entire read-set for that namespace as input and performs the version validation within the database itself, returning only the
@@ -229,7 +229,7 @@ type validatedTransactions struct {
229
229
This final task consumes `validatedTransactions`, performs final write preparations, commits the valid data to the database, and
230
230
reports the final status of all transactions back to the Coordinator.
231
231
232
-
**a. Dequeueing Validated Transactions:** The [committer](https://github.com/hyperledger/fabric-x-committer/blob/main/service/vc/committer.go) reads a `validatedTransactions` object from the `validatedTxs` queue.
232
+
**a. Dequeueing Validated Transactions:** The [committer](/service/vc/committer.go) reads a `validatedTransactions` object from the `validatedTxs` queue.
233
233
234
234
**b. Resolving Blind Writes:** Before committing, the committer must resolve the "blind writes". It fetches the current versions of keys in
235
235
the `validTxBlindWrites` set from the database. This allows it to categorize each blind write as either an update to an existing key or
@@ -262,7 +262,7 @@ back to the Coordinator, completing the workflow for the transaction batch.
262
262
263
263
## 6. gRPC Service API
264
264
265
-
The VC service exposes a [gRPC API](https://github.com/hyperledger/fabric-x-committer/blob/main/service/vc/validator_committer_service.go) (`ValidationAndCommitServiceClient`) for the Coordinator and other system components. The methods are:
265
+
The VC service exposes a [gRPC API](/service/vc/validator_committer_service.go) (`ValidationAndCommitServiceClient`) for the Coordinator and other system components. The methods are:
0 commit comments