-
Notifications
You must be signed in to change notification settings - Fork 652
Add design docs for reference #1029
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
yyforyongyu
wants to merge
5
commits into
btcsuite:master
Choose a base branch
from
yyforyongyu:add-design-docs
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # Chain Package Architecture | ||
|
|
||
| This document outlines the architecture for the `chain` package, which serves as the foundational layer for interacting with the Bitcoin blockchain. | ||
|
|
||
| ## Core Principle: A Two-Tiered Architecture | ||
|
|
||
| The `btcwallet`'s interaction with the blockchain is defined by a two-tiered architecture that creates a clean separation of concerns: | ||
|
|
||
| 1. **`chain` Package (The Driver Layer)**: This package is responsible for **low-level, stateless communication** with different blockchain backends (e.g., `btcd`, `bitcoind`). It provides a clean, abstracted API for direct blockchain queries and transaction broadcasting. It **does not** manage any long-lived state related to notifications. | ||
|
|
||
| 2. **`ntfn` Package (The Notification Layer)**: A separate, high-level package is the **stateful notification manager**. It is built *on top of* the `chain` package and consumes its raw events. All complex logic for handling reorgs and managing notifications resides in the `ntfn` package. | ||
|
|
||
| This document focuses exclusively on the design of the `chain` package. | ||
|
|
||
| ## Architectural Design: The Actor Model | ||
|
|
||
| To adhere to the project's core philosophy, the `chain` package is designed around the **Actor Model**. Each backend connection (`btcd`, `bitcoind`, etc.) is managed by a dedicated actor (a long-running goroutine). This actor owns all connection state and processes requests sequentially, eliminating the need for complex locking. | ||
|
|
||
| ## Interface-Driven Design | ||
|
|
||
| The `chain` package exposes a set of small, role-based interfaces. This adheres to the Interface Segregation Principle and allows consuming packages to depend only on the functionality they need. A single concrete `ChainDriver` struct (e.g., `BitcoindDriver`) will implement the interfaces it supports. | ||
|
|
||
| ### `ChainQuery` Interface | ||
| *For all methods that **fetch** data from the backend.* | ||
| ```go | ||
| // ChainQuery provides methods for querying the blockchain and mempool. | ||
| type ChainQuery interface { | ||
| // GetBestBlock returns the hash and height of the best block known to | ||
| // the backend. | ||
| GetBestBlock(ctx context.Context) (*chainhash.Hash, int32, error) | ||
|
|
||
| // GetBlock returns the block for the given hash. | ||
| GetBlock(ctx context.Context, hash *chainhash.Hash) (*wire.MsgBlock, error) | ||
|
|
||
| // GetBlockHash returns the hash of the block at the given height. | ||
| GetBlockHash(ctx context.Context, height int64) (*chainhash.Hash, error) | ||
|
|
||
| // GetBlockHeader returns the header for the given block hash. | ||
| GetBlockHeader(ctx context.Context, hash *chainhash.Hash) (*wire.BlockHeader, error) | ||
|
|
||
| // GetRawTransaction returns the transaction for the given hash. | ||
| // | ||
| // NOTE: The behavior of this method may differ between backends. | ||
| // Full nodes can query the mempool for unconfirmed transactions, | ||
| // while light clients like Neutrino may only return confirmed | ||
| // transactions. | ||
| GetRawTransaction(ctx context.Context, hash *chainhash.Hash) (*btcutil.Tx, error) | ||
| } | ||
| ``` | ||
|
|
||
| ### `ChainIO` Interface | ||
| *For all methods that **send** data to the network.* | ||
| ```go | ||
| // ChainIO provides methods for interacting with the blockchain, such as | ||
| // broadcasting transactions. | ||
| type ChainIO interface { | ||
| // SendRawTransaction broadcasts a transaction to the network. A | ||
| // successful call does not mean the transaction has been confirmed, | ||
| // only that it has been accepted by the backend for relay. | ||
| SendRawTransaction(ctx context.Context, tx *wire.MsgTx, allowHighFees bool) (*chainhash.Hash, error) | ||
| } | ||
| ``` | ||
|
|
||
| ### `ChainEventReceiver` Interface | ||
| *The key to our dependency inversion, this interface is implemented by the `ntfn` package and passed to the `chain` driver.* | ||
| ```go | ||
| // ChainEventReceiver is an interface that a consumer of the chain.Driver | ||
| // can implement to receive raw blockchain event notifications. | ||
| type ChainEventReceiver interface { | ||
| // OnBlockConnected is called when a new block is connected to the | ||
| // main chain. | ||
| OnBlockConnected(hash *chainhash.Hash, height int32, t time.Time) | ||
|
|
||
| // OnBlockDisconnected is called when a block is disconnected from the | ||
| // main chain. | ||
| OnBlockDisconnected(hash *chainhash.Hash, height int32, t time.Time) | ||
|
|
||
| // OnRelevantTx is called when a transaction relevant to the wallet is | ||
| // discovered. | ||
| OnRelevantTx(tx *btcutil.Tx, details *btcjson.BlockDetails) | ||
| } | ||
| ``` | ||
|
|
||
| ### `MempoolObserver` (Optional) Interface | ||
| *For full-node-specific mempool functionality.* | ||
| ```go | ||
| // MempoolObserver provides methods for observing the mempool of a full node. | ||
| // This is an optional interface that a ChainDriver may implement. | ||
| type MempoolObserver interface { | ||
| // GetRawMempool returns the hashes of all transactions in the mempool. | ||
| GetRawMempool(ctx context.Context) ([]*chainhash.Hash, error) | ||
|
|
||
| // TestMempoolAccept checks whether a transaction would be accepted | ||
| // into the mempool. | ||
| TestMempoolAccept(ctx context.Context, txns []*wire.MsgTx) ([]*btcjson.TestMempoolAcceptResult, error) | ||
| } | ||
| ``` | ||
|
|
||
| ### Driver Implementation Principles | ||
|
|
||
| All concrete `chain.Driver` implementations **MUST** adhere to the following principles: | ||
|
|
||
| 1. **Drivers MUST Be Stateless**: A driver's responsibility is to be a stateless I/O adapter. All stateful logic (queues, subscription management, reorg tracking) is pushed up to the `ntfn` layer. | ||
| 2. **Notification Structs MUST Be Self-Contained**: Event structs passed from the driver to the receiver must not depend on higher-level packages like `wtxmgr`. | ||
| 3. **Drivers MUST Return Typed, Exported Errors**: Drivers are responsible for mapping backend-specific errors to a standard set of exported errors in the `chain` package. | ||
| 4. **Implementations MUST Be Opaque**: Complex, backend-specific logic (like the `PrunedBlockDispatcher`) must be an opaque, internal implementation detail of a driver and must not leak into the public API. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # chain | ||
|
|
||
| ## Overview | ||
|
|
||
| The `chain` package is the foundational layer of `btcwallet` responsible for interacting with the Bitcoin blockchain. Its primary role is to act as a low-level, stateless I/O adapter over various blockchain backends, such as `btcd`, `bitcoind`, and `neutrino`. This ensures that the core wallet logic remains agnostic to the specific chain source being used. | ||
|
|
||
| For a detailed overview of the package's design and architecture, please see the [architecture documentation](./ARCHITECTURE.md). | ||
|
|
||
| ## Architecture | ||
|
|
||
| The package is designed to be a stateless I/O layer. All stateful logic (such as notification queues, subscription management, and reorg tracking) is handled by a higher-level package, `ntfn`. | ||
|
|
||
| ### Provided Interfaces | ||
|
|
||
| - **`ChainQuery`**: A read-only interface for fetching data from the blockchain (e.g., `GetBlock`, `GetBestBlock`). | ||
| - **`ChainIO`**: A write-only interface for broadcasting transactions (`SendRawTransaction`). | ||
| - **`MempoolObserver` (Optional)**: An optional interface that a driver may implement to provide functionality specific to full nodes that have a view of the transaction mempool. | ||
|
|
||
| ### Required Interfaces | ||
|
|
||
| The `chain` package's drivers depend on the following interface, which must be implemented by a consumer (such as the `ntfn` package): | ||
|
|
||
| - **`ChainEventReceiver`**: An interface that the driver uses to forward raw, low-level blockchain events to a higher-level, stateful consumer. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # `ntfn` Package Architecture | ||
|
|
||
| This document outlines the architecture of the `ntfn` (notification) package. This package provides a high-level, stateful notification service for on-chain events. | ||
|
|
||
| ## Core Responsibilities | ||
|
|
||
| The `ntfn` package is the stateful counterpart to the stateless `chain` package. Its responsibilities are: | ||
|
|
||
| 1. **Stateful Notification Management**: To manage the lifecycle of client subscriptions for transaction confirmations, UTXO spends, and new block epochs. | ||
| 2. **Reorg Handling**: To provide a robust, reorg-aware notification service. It contains all the complex logic for tracking confirmation counts across reorgs and notifying clients of chain reorganizations. | ||
| 3. **Historical Dispatch**: To scan the historical chain for events that occurred before a client subscribed, ensuring no events are missed. | ||
| 4. **Decoupling**: To abstract the complexity of on-chain event handling, providing a simple and powerful API to its clients (like `wallet` and `lnd`). | ||
|
|
||
| ## Architectural Design | ||
|
|
||
| The `ntfn` package contains a central component, the `Notifier`, which implements the primary public interface. | ||
|
|
||
| - **Dependency on `chain`**: The `Notifier` is built on top of the `chain` package. It takes a `chain.Driver` as a dependency and implements the `chain.ChainEventReceiver` interface to consume a raw stream of blockchain events. | ||
| - **State Management**: The `Notifier` is responsible for all state related to notifications, including client subscriptions, confirmation heights, and its view of the chain tip. | ||
| - **Proven Design**: The `Notifier` interface is adopted directly from `@lnd/chainntnfs`, which is a battle-tested design known to meet the needs of its primary consumer, `lnd`. | ||
|
|
||
| ## `Notifier` Interface Design | ||
|
|
||
| The interface and its supporting data structures are migrated directly from `@lnd/chainntnfs/interface.go`, ensuring a seamless transition for `lnd`. | ||
|
|
||
| ```go | ||
| package ntfn | ||
|
|
||
| // Notifier represents a trusted source to receive notifications concerning | ||
| // targeted events on the Bitcoin blockchain. | ||
| type Notifier interface { | ||
| // RegisterConfirmationsNtfn registers an intent to be notified once a | ||
| // txid reaches a specified number of confirmations. | ||
| RegisterConfirmationsNtfn(txid *chainhash.Hash, pkScript []byte, | ||
| numConfs, heightHint uint32) (*ConfirmationEvent, error) | ||
|
|
||
| // RegisterSpendNtfn registers an intent to be notified once a target | ||
| // outpoint is successfully spent. | ||
| RegisterSpendNtfn(outpoint *wire.OutPoint, pkScript []byte, | ||
| heightHint uint32) (*SpendEvent, error) | ||
|
|
||
| // RegisterBlockEpochNtfn registers an intent to be notified of each | ||
| // new block connected to the tip of the main chain. | ||
| RegisterBlockEpochNtfn(*BlockEpoch) (*BlockEpochEvent, error) | ||
|
|
||
| // Start the Notifier. | ||
| Start() error | ||
|
|
||
| // Stop the Notifier. | ||
| Stop() error | ||
| } | ||
|
|
||
| // TxConfirmation carries details of the block that confirmed a transaction. | ||
| type TxConfirmation struct { | ||
| BlockHash *chainhash.Hash | ||
| BlockHeight uint32 | ||
| TxIndex uint32 | ||
| Tx *wire.MsgTx | ||
| Block *wire.MsgBlock | ||
| } | ||
|
|
||
| // ConfirmationEvent encapsulates a confirmation notification. | ||
| type ConfirmationEvent struct { | ||
| Confirmed chan *TxConfirmation | ||
| NegativeConf chan int32 // For reorgs | ||
| Done chan struct{} | ||
| Cancel func() | ||
| } | ||
|
|
||
| // SpendDetail contains details pertaining to a spent output. | ||
| type SpendDetail struct { | ||
| SpentOutPoint *wire.OutPoint | ||
| SpenderTxHash *chainhash.Hash | ||
| SpendingTx *wire.MsgTx | ||
| SpenderInputIndex uint32 | ||
| SpendingHeight int32 | ||
| } | ||
|
|
||
| // SpendEvent encapsulates a spentness notification. | ||
| type SpendEvent struct { | ||
| Spend chan *SpendDetail | ||
| Reorg chan struct{} // For reorgs | ||
| Done chan struct{} | ||
| Cancel func() | ||
| } | ||
|
|
||
| // BlockEpoch represents metadata for each new block. | ||
| type BlockEpoch struct { | ||
| Hash *chainhash.Hash | ||
| Height int32 | ||
| BlockHeader *wire.BlockHeader | ||
| } | ||
|
|
||
| // BlockEpochEvent encapsulates a stream of block epoch notifications. | ||
| type BlockEpochEvent struct { | ||
| Epochs <-chan *BlockEpoch | ||
| Cancel func() | ||
| } | ||
| ``` | ||
|
|
||
| ## Key Architectural Decisions | ||
|
|
||
| The design and interaction of the `ntfn` package are governed by several key architectural decisions: | ||
|
|
||
| - **Hybrid "Push" Notification Model**: The `Notifier` uses a primarily asynchronous "push" model. However, to support linear client workflows, registration methods (`Register...Ntfn`) provide an immediate, synchronous check of the notifier's current, in-memory state. If an event has already occurred, the returned channel will be pre-populated. | ||
|
|
||
| - **"Subscribe-First, Sync-Later" Startup**: To eliminate race conditions, all subsystems must subscribe to the `ntfn.Notifier` *before* the `chain.Driver` is started and begins processing blocks. The `Notifier`'s `Start()` method is non-blocking, and historical scans are performed asynchronously in the background. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # ntfn | ||
|
|
||
| ## Overview | ||
|
|
||
| The `ntfn` package provides a high-level, stateful notification service for on-chain events. It is built on top of the low-level, stateless `chain` package and is responsible for all the complex logic of managing reorgs, historical dispatches, and ensuring exactly-once delivery of notifications to clients like `wallet` and `lnd`. | ||
|
|
||
| For a detailed overview of the package's design and architecture, please see the [architecture documentation](./ARCHITECTURE.md). | ||
|
|
||
| ## Architecture | ||
|
|
||
| The `ntfn` package contains a central component, the `Notifier`, which consumes raw events from a `chain.Driver` and provides a robust, reorg-aware notification service to its clients. | ||
|
|
||
| ### Provided Interfaces | ||
|
|
||
| - **`Notifier`**: The primary public interface of the `ntfn` package. It provides a robust, reorg-aware notification service with a type-safe, battle-tested API for registering for confirmation, spend, and block epoch notifications. | ||
|
|
||
| ### Required Interfaces | ||
|
|
||
| The `ntfn` package depends on the following interfaces for its operation: | ||
|
|
||
| - **`chain.Driver`**: The `Notifier` takes a `chain.Driver` (which implements `ChainQuery`, `ChainIO`, etc.) as a dependency in its constructor to interact with the blockchain. | ||
| - **`chain.ChainEventReceiver`**: The `Notifier` implements this interface to receive a raw stream of blockchain events from the `chain.Driver`. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The observer name here reminds me of the Observer Pattern, but the purpose seems to be a bit different, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
correct - this is more like providing a mempool IO, tho the whole
ntftuses the publish-subscribe pattern, which I believe is the same as the observer pattern you linked.