feat(starfish): send full block #6803
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Description of change This PR introduces a `starfish` crate in which Starfish consensus is going to be developed. It's merged to a feature branch which will aggregate all the changes before the crate is in usable state and can be merged to develop. ### Release Notes - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: ilyavorobev <[email protected]>
# Description of change PR removes garbage collection(gc) feature from Starfish. It may be reimplemented later. ## Links to any relevant issues fixes #6420. ## Type of change Choose a type of change, and delete any options that are not relevant. - Cleanup ## How the change has been tested All tests from consensus crate except the gc-related ones are passed Make sure to provide instructions for the maintainer as well as any relevant configurations. ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [ ] I have followed the contribution guidelines for this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes
# Description of change This PR implements an encoder and decoder for Starfish consensus ## Links to any relevant issues fixes #6163 ## Type of change Choose a type of change, and delete any options that are not relevant. - Enhancement (a non-breaking change which adds functionality) ## How the change has been tested Describe the tests that you ran to verify your changes. Make sure to provide instructions for the maintainer as well as any relevant configurations. ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [x] I have followed the contribution guidelines for this project - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have checked that new and existing unit tests pass locally with my changes
# Description of change
With Starfish, we decouple transaction data from block headers. This
requires changes in the block structure. Specifically, the block header
now contains the following fields:
- epoch: Epoch,
- round: Round,
- author: AuthorityIndex,
- timestamp_ms: BlockTimestampMs,
- ancestors: Vec(BlockRef), (at least 2f+1 references to blocks from the
previous round)
- acknowledgments: Vec(BlockRef), (acknowledging data availability of
transactions from prior blocks)
- transactions_commitment: TransactionDigest, (hash digest of
transaction data associated with this given block)
- commit_votes: Vec(CommitVote)
This PR touches only basic functionalities of this structure (e.g.
BlockHeaderAPI). Most of the tests within the starfish crate do not
require consuming transactions, but those that require transaction
inclusion were changed to pass the tests.
Further PRs should enable a full block consisting of a block header and
a block body (transaction data).
## Links to any relevant issues
Closes #6426
## Type of change
Choose a type of change, and delete any options that are not relevant.
- Breaking change (new block header structure affects everything in the
starfish crate)
## How the change has been tested
- [x ] Basic tests (linting, compilation, formatting, unit/integration
tests)
### Infrastructure QA (only required for crates that are maintained by
@iotaledger/infrastructure)
- [ ] Synchronization of the indexer from genesis for a network
including migration objects.
- [ ] Restart of indexer synchronization locally without resetting the
database.
- [ ] Restart of indexer synchronization on a production-like database.
- [ ] Deployment of services using Docker.
- [ ] Verification of API backward compatibility.
## Change checklist
Tick the boxes that are relevant to your changes, and delete any items
that are not.
- [ ] I have followed the contribution guidelines for this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have checked that new and existing unit tests pass locally with
my changes
### Release Notes
<!--
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
-->
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
) # Description of change PR removes the smart ancestor selection feature. Tests related to this feature are left in the code. ## Links to any relevant issues Closes #6422. ## Type of change Cleanup ## How the change has been tested All tests from consensus crate are passed. ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [ ] I have followed the contribution guidelines for this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: Smart ancestor selection is removed - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
# Description of change This PR removes the round prober feature from Starfish. ## Links to any relevant issues Closes #6421 ## Type of change - Cleanup ## How the change has been tested CI ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [x] I have followed the contribution guidelines for this project - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: The PR removes round prober feature from starfish - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
# Description of change This PR removes the leader schedule v1 from Starfish. ## Links to any relevant issues Closes #6424. ## Type of change Cleanup ## How the change has been tested CI ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [x] I have followed the contribution guidelines for this project - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [x] Protocol: PR removes leader schedule v1 from Starfish and renames v2 to leader schedule - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
…h env variable (#6488) # Description of change This PR introduces a possibility to run IOTA node with Starfish consensus enabled. It's going to be used for testing purposes initially and later we'll re-evaluate its use. This does not break the existing use of Mysticeti, which is still used by default. ## Links to any relevant issues Closes #6417 ## Type of change Choose a type of change, and delete any options that are not relevant. - Enhancement (a non-breaking change which adds functionality) ## How the change has been tested CI ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [ ] I have followed the contribution guidelines for this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
…der per round (#6515) # Description of change This PR reduces the number of constants used in the Starfish protocol. The wavelength is hard-coded to be 3. The pipeline in the universal committer is always enabled. While we leave the opportunity for a multileader universal committer, all current tests and the consensus logic assume exactly one leader per round. The PR fixes many tests and replaces with new ones, and, in addition, changes the name of the so-called "decision" round to "certifying" round to align with the Starfish paper and the underlying meaning of this name. ## Links to any relevant issues Closes #6425 ## Type of change Choose a type of change, and delete any options that are not relevant. - Bug fix (a non-breaking change which fixes an issue) ## How the change has been tested CI ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [ ] I have followed the contribution guidelines for this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [x] Protocol: make a constant 3-round wave length - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
# Description of change This PR removes linearizer_v2. Linearizer_v2 works only when gc is enabled. We removed gc, so linearizer v2 is also removed. ## Links to any relevant issues Be sure to reference any related issues by adding `fixes #(issue)`. ## Type of change Cleanup ## How the change has been tested CI ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [x] I have followed the contribution guidelines for this project - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
# Description of change In this PR, we add two more steps in the creation of the block - We compute the commitment of transaction data to be included in block header (serialization+hash) - We take pending acknowledgments for received transaction data from other peers In addition, we wrap transaction data into VerifiedTransactionData to be used for storing and broadcasting ## Links to any relevant issues Closes #6430 ## Type of change Choose a type of change, and delete any options that are not relevant. - Bug fix (a non-breaking change which fixes an issue) ## How the change has been tested CI ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. ## Change checklist Tick the boxes that are relevant to your changes, and delete any items that are not. - [ ] I have followed the contribution guidelines for this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have checked that new and existing unit tests pass locally with my changes ### Release Notes - [x] Protocol: Updated the block creation function to include acknowledgments and transaction commitment - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 4 Skipped Deployments
|
VorobyevIlya
force-pushed
the
consensus/feat/starfish/send-full-block
branch
from
7a9fb27 to
c1e421a
Compare
…nager (#6794) # Description of change This PR simplifies the implementation of managing network setup in the Starfish consensus implementation. It removes unnecessary `NetworkManager` trait, because there is currently one supported network type - Tonic. Since Tonic supports streaming, Broadcaster has also been removed as it was only used for network types that don't support streaming. ## Links to any relevant issues Fixes #6423 ## Type of change Choose a type of change, and delete any options that are not relevant. - Enhancement (a non-breaking change which adds functionality) ## How the change has been tested Unable to test because starfish implementation is not yet in a working condition. Make sure to provide instructions for the maintainer as well as any relevant configurations. - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage)
fix warnings fmt create add block headers command it compiles! fixing wrapper continues fix decoupling header from transactions decouple serialized bytes fix typo implement a wrapper around serialized header and transactions -- SerializedBlock (not finished) implement write and read functions for blocks for RocksDB fix test compilation fix compilation errors add a new struct VerifiedBlock
VorobyevIlya
force-pushed
the
consensus/feat/starfish/send-full-block
branch
from
6ecd338 to
6c73112
Compare
piotrm50
force-pushed
the
consensus/feat/starfish-consensus
branch
from
6b470f7 to
3d6f76d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
This PR introduces a full block structure, which contains a header and transactions.
In many places, we change the header into the full block, for example, in send_block functions.
In some places, we implement structures and functions to work with headers and blocks.
Links to any relevant issues
closes #6707.
Type of change
Choose a type of change, and delete any options that are not relevant.
How the change has been tested
Change checklist
Tick the boxes that are relevant to your changes, and delete any items that are not.
Release Notes