You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(selector): describe the actual selection algorithm
The selector page described a configurable, amount-aware selection
strategy (smallest-first, FIFO) that the code does not have. There is no
strategy abstraction, no configuration key for one, and token amounts
never influence which candidate is picked - only when the accumulation
loop stops.
Describe what selectInternal actually does: a randomized greedy
first-fit that walks the candidate tokens in randomized order, locks
each one as it is encountered, and returns as soon as the running sum
covers the request. State that the randomization is deliberate, since it
is what spreads concurrent selectors across different candidates and
keeps lock contention down, and note that the shuffle lives in the
sherdlock fetcher while the simple driver walks the database order.
Correct the driver key comment: it selects the selector implementation
and its locking backend, not a selection algorithm. Keep smallest-first
and FIFO on the page only under an explicit note marking them as not
implemented, pointing at the issue that tracks amount-aware selection.
Signed-off-by: Hayim.Shaul@ibm.com <hayimsha@fhe03.vpc.cloud9.ibm.com>
fix doc
Signed-off-by: Hayim.Shaul@ibm.com <hayimsha@fhe03.vpc.cloud9.ibm.com>
fix doc
Signed-off-by: Hayim.Shaul@ibm.com <hayimsha@fhe03.vpc.cloud9.ibm.com>
fix doc
Signed-off-by: Hayim.Shaul@ibm.com <hayimsha@fhe03.vpc.cloud9.ibm.com>
Copy file name to clipboardExpand all lines: docs/services.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -88,7 +88,7 @@ The [Storage Service](./services/storage.md) encapsulates all data persistence m
88
88
The [Tokens Service](./services/tokens.md) provides advanced operations on tokens that go beyond basic UTXO management. This includes de-obfuscating token metadata for authorized parties and handling token upgrades (e.g., migrating from one driver implementation to another).
89
89
90
90
### Selector Service
91
-
The [Selector Service](./services/selector.md)implements strategic token selection algorithms. It is responsible for selecting the optimal set of UTXOs for a given transaction while mitigating the risk of double-spending by temporarily locking tokens in use.
91
+
The [Selector Service](./services/selector.md)selects the UTXOs that fund a transaction, using a randomized greedy first-fit over the wallet's candidate tokens, and mitigates the risk of double-spending by temporarily locking the tokens in use.
92
92
93
93
### Auditor Service
94
94
The [Auditor Service](./services/auditor.md) provides tools for oversight and compliance. It allows authorized auditors to inspect transactions, verify public parameters, and ensure that the system adheres to established rules without compromising the privacy of non-audited users.
Copy file name to clipboardExpand all lines: docs/services/selector.md
+87-14Lines changed: 87 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,13 @@
1
1
# Selector Service
2
2
3
-
The **Selector Service** (`token/services/selector`) implements strategic token selection algorithms to ensure that Panurus can efficiently and correctly select the best set of unspent tokens (UTXOs) for any given transaction.
3
+
The **Selector Service** (`token/services/selector`) picks the unspent tokens (UTXOs) that fund a transaction and holds them under a temporary lock while the transaction is assembled, so that concurrent transactions of the same wallet do not try to spend the same tokens.
4
4
5
5
## Core Responsibilities
6
6
7
7
The Selector Service is responsible for:
8
8
***UTXO Selection**: Finding a set of spendable tokens that cover the total quantity required for a transfer operation.
9
9
***Double-Spending Mitigation**: Temporarily locking selected tokens during the transaction assembly phase to prevent multiple concurrent transactions from attempting to spend the same tokens.
10
-
***Selection Strategy Implementation**: Providing different algorithms (e.g., First-In-First-Out, smallest-first) to optimize for transaction size, cost, or privacy.
10
+
***Candidate Enumeration**: Walking the wallet's candidate tokens in randomized order, locking each one as it is encountered, and stopping as soon as the accumulated amount covers the request. Token amounts do not order or rank the candidates.
11
11
12
12
## Interaction with TTX and Storage
13
13
@@ -25,43 +25,98 @@ graph LR
25
25
26
26
subgraph "Selection Logic"
27
27
Query[Query Spendable Tokens]
28
-
Strategy[Apply Selection Strategy]
28
+
Pick[Take Next Candidate - randomized order]
29
29
Lock[Acquire Temporary Lock]
30
+
Done[Return Locked Tokens]
30
31
end
31
32
32
33
Selector --> Fetcher
33
34
Selector --> Query
34
-
Query --> Strategy
35
-
Strategy --> Lock
35
+
Query --> Pick
36
+
Pick --> Lock
37
+
Lock -->|locked by another process, or sum still below target| Pick
38
+
Lock -->|requested amount covered| Done
36
39
```
37
40
38
41
**How the components interact:**
39
42
-**Selector Service**: Creates a selector instance per transaction and orchestrates the Selection Logic steps
40
43
-**Query Spendable Tokens**: Selector calls the Fetcher to retrieve available tokens
41
44
-**Fetcher Logic**: Checks cache first (fast path), queries Token Store - TokenDB on cache miss (slow path)
-**Acquire Temporary Lock**: Selector locks each selected token in storage to prevent concurrent selection
45
+
-**Take Next Candidate**: Selector takes the next token from the randomized candidate set; the token's amount plays no part in the choice
46
+
-**Acquire Temporary Lock**: Selector locks each candidate as it is encountered, before it knows whether the request can be covered at all; a candidate already locked by another process is skipped and the loop moves on
44
47
45
48
## Key Components
46
49
47
50
### Selector Manager
48
51
The `SelectorManager` is the entry point for obtaining a `Selector` instance anchored to a specific transaction. It ensures that the selection process is consistent and tied to the lifecycle of a single token request.
49
52
50
-
### Token Selection Strategy
51
-
The service supports various strategies for picking tokens (see "Strategy" box in diagram above). A common strategy is to pick the smallest number of tokens that cover the requested amount to minimize the transaction size and the associated verification overhead on the ledger.
53
+
### Token Selection Algorithm
54
+
55
+
Selection is a **randomized greedy first-fit**. It is not configurable, and it is not
1. the candidate tokens of the wallet and token type are enumerated in randomized order,
60
+
2. each candidate is locked as it is encountered — a candidate already locked by another
61
+
process is skipped; a lock failure wrapping `token.SelectorRateLimited` is a hard abort
62
+
(not a skip),
63
+
3. the amounts of the successfully locked tokens are added up, and
64
+
4. the selector returns as soon as the running sum reaches the requested quantity.
65
+
66
+
A token's amount therefore only decides *when* the loop stops, never *which* candidate is
67
+
picked. Two consequences worth planning for:
68
+
69
+
***The number and size of the inputs is not minimized.** A request that a single large
70
+
token could have covered may well be funded by several small ones.
71
+
***The result is not deterministic.** The same request against the same wallet can select
72
+
a different set of tokens, and a different number of inputs, on each run.
73
+
74
+
**The randomization is deliberate.** It is what spreads concurrent selectors of the same
75
+
wallet across different candidates: walking a fixed order would make every selector contend
76
+
for the same first tokens, driving up lock failures and, with them, the immediate-retry path
77
+
that gives up with `token.SelectorSufficientButLockedFunds`, and beyond it the backoff path
78
+
that ends in `token.SelectorInsufficientFunds`.
79
+
80
+
The shuffle lives in the sherdlock fetcher, not in the selection loop
81
+
(`token/services/selector/sherdlock/fetcher.go`): the lazy fetcher wraps the database
82
+
iterator in `collections.NewPermutatedIterator`, and the cached fetcher hands out a fresh
83
+
permutation of the cached slice on every query. The `simple` driver does **not** shuffle — it
84
+
walks the database iterator in the order the token store returns it
85
+
(`token/services/selector/simple/selector.go`) — so concurrent selectors under `simple` are
86
+
more exposed to colliding on the same leading candidates.
52
87
53
88
**How it works in the flow (see "Selection Logic" subgraph in diagram):**
54
89
1.**TTX Request**: TTX Service requests token selection for a transfer operation
55
90
2.**Query Spendable Tokens**: Selector queries via Fetcher (Cache Hit → fast path, Cache Miss → Token Store - TokenDB)
56
-
3.**Apply Selection Strategy**: Algorithm picks optimal tokens based on configured strategy (e.g., smallest-first)
57
-
4.**Acquire Temporary Lock**: Selected tokens are locked in TokenLocks table to prevent double-spending
91
+
3.**Take Next Candidate**: Selector takes the next token from the randomized candidate set
92
+
4.**Acquire Temporary Lock**: The candidate is locked to prevent double-spending (in the `TokenLocks` table under the `sherdlock` driver, in memory under `simple`); on success its amount is added to the running sum, on failure the loop moves to the next candidate
93
+
5.**Return or Retry**: The selector returns as soon as the sum covers the request; if the
94
+
candidate set is exhausted while other processes hold locks, it retries in two distinct
95
+
layers:
96
+
-**Immediate-retry layer** (`sherdlock` only): the inner loop refetches — refreshing the
97
+
sherdlock token cache via the fetcher — up to a hardcoded `maxImmediateRetries = 5` times
98
+
without releasing its already-acquired locks, then gives up with
99
+
`token.SelectorSufficientButLockedFunds`. Under `simple`, there is no equivalent cache
100
+
layer; the outer retry loop re-queries the query service directly on every attempt.
101
+
-**Backoff layer**: a configurable `numRetries` / `retryInterval` outer loop (the
102
+
`StubbornSelector` wrapper in `sherdlock`; the `numRetry` / `timeout` loop in `simple`)
103
+
releases locks, sleeps, and re-runs the whole selection from scratch. Exhausting this
104
+
layer returns `token.SelectorInsufficientFunds`.
105
+
106
+
#### Strategies that are not implemented
107
+
108
+
Amount-aware strategies — smallest-first, largest-first, First-In-First-Out, or minimizing
109
+
the number of inputs — are **not** implemented and cannot be configured. There is no
110
+
strategy abstraction in the code and no configuration key that selects one. Making selection
To prevent double-spending *before* the transaction is committed to the ledger, the Selector Service uses a local `TokenLocks` table in the **Storage Service** (see "TokenLocks" box in diagram above).
61
116
62
117
**Lock lifecycle:**
63
-
1.**Lock Acquisition**: When a token is selected by the Strategy, the service attempts to insert a record in the `TokenLocks` table.
64
-
2.**Concurrency Control**: If another concurrent process has already locked that token, the insertion fails, and the selector picks a different token.
118
+
1.**Lock Acquisition**: When the selector takes a candidate token, it attempts to insert a record in the `TokenLocks` table.
119
+
2.**Concurrency Control**: If another concurrent process has already locked that token, the insertion fails, and the selector moves on to the next candidate.
65
120
3.**Lock Release**: Locks are released either when the transaction reaches finality (success/failure) or when a timeout occurs, ensuring that tokens do not remain permanently inaccessible due to crashed or abandoned transactions.
66
121
67
122
### In-Memory Locker Internals
@@ -119,7 +174,7 @@ Configure the selector service in your `core.yaml`:
0 commit comments